diff options
114 files changed, 33127 insertions, 6963 deletions
@@ -1,6 +1,23 @@ -Written by: Philip Hazel <ph10@cam.ac.uk> +THE MAIN PCRE LIBRARY +--------------------- + +Written by: Philip Hazel +Email local part: ph10 +Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. Phone: +44 1223 334714. -Copyright (c) 1997-2004 University of Cambridge +Copyright (c) 1997-2005 University of Cambridge +All rights reserved + + +THE C++ WRAPPER LIBRARY +----------------------- + +Written by: Google Inc. + +Copyright (c) 2005 Google Inc +All rights reserved + +#### @@ -4,18 +4,40 @@ PCRE LICENCE PCRE is a library of functions to support regular expressions whose syntax and semantics are as close as possible to those of the Perl 5 language. -Release 5 of PCRE is distributed under the terms of the "BSD" licence, as +Release 6 of PCRE is distributed under the terms of the "BSD" licence, as specified below. The documentation for PCRE, supplied in the "doc" directory, is distributed under the same terms as the software itself. -Written by: Philip Hazel <ph10@cam.ac.uk> +The basic library functions are written in C and are freestanding. Also +included in the distribution is a set of C++ wrapper functions. + + +THE BASIC LIBRARY FUNCTIONS +--------------------------- + +Written by: Philip Hazel +Email local part: ph10 +Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. Phone: +44 1223 334714. -Copyright (c) 1997-2004 University of Cambridge +Copyright (c) 1997-2005 University of Cambridge +All rights reserved. + + +THE C++ WRAPPER FUNCTIONS +------------------------- + +Contributed by: Google Inc. + +Copyright (c) 2005, Google Inc. All rights reserved. + +THE "BSD" LICENCE +----------------- + Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: @@ -26,9 +48,10 @@ modification, are permitted provided that the following conditions are met: notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - * Neither the name of the University of Cambridge nor the names of its - contributors may be used to endorse or promote products derived from - this software without specific prior written permission. + * Neither the name of the University of Cambridge nor the name of Google + Inc. nor the names of their contributors may be used to endorse or + promote products derived from this software without specific prior + written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE @@ -1,6 +1,158 @@ ChangeLog for PCRE ------------------ +Version 6.0 07-Jun-05 +--------------------- + + 1. Some minor internal re-organization to help with my DFA experiments. + + 2. Some missing #ifdef SUPPORT_UCP conditionals in pcretest and printint that + didn't matter for the library itself when fully configured, but did matter + when compiling without UCP support, or within Exim, where the ucp files are + not imported. + + 3. Refactoring of the library code to split up the various functions into + different source modules. The addition of the new DFA matching code (see + below) to a single monolithic source would have made it really too + unwieldy, quite apart from causing all the code to be include in a + statically linked application, when only some functions are used. This is + relevant even without the DFA addition now that patterns can be compiled in + one application and matched in another. + + The downside of splitting up is that there have to be some external + functions and data tables that are used internally in different modules of + the library but which are not part of the API. These have all had their + names changed to start with "_pcre_" so that they are unlikely to clash + with other external names. + + 4. Added an alternate matching function, pcre_dfa_exec(), which matches using + a different (DFA) algorithm. Although it is slower than the original + function, it does have some advantages for certain types of matching + problem. + + 5. Upgrades to pcretest in order to test the features of pcre_dfa_exec(), + including restarting after a partial match. + + 6. A patch for pcregrep that defines INVALID_FILE_ATTRIBUTES if it is not + defined when compiling for Windows was sent to me. I have put it into the + code, though I have no means of testing or verifying it. + + 7. Added the pcre_refcount() auxiliary function. + + 8. Added the PCRE_FIRSTLINE option. This constrains an unanchored pattern to + match before or at the first newline in the subject string. In pcretest, + the /f option on a pattern can be used to set this. + + 9. A repeated \w when used in UTF-8 mode with characters greater than 256 + would behave wrongly. This has been present in PCRE since release 4.0. + +10. A number of changes to the pcregrep command: + + (a) Refactored how -x works; insert ^(...)$ instead of setting + PCRE_ANCHORED and checking the length, in preparation for adding + something similar for -w. + + (b) Added the -w (match as a word) option. + + (c) Refactored the way lines are read and buffered so as to have more + than one at a time available. + + (d) Implemented a pcregrep test script. + + (e) Added the -M (multiline match) option. This allows patterns to match + over several lines of the subject. The buffering ensures that at least + 8K, or the rest of the document (whichever is the shorter) is available + for matching (and similarly the previous 8K for lookbehind assertions). + + (f) Changed the --help output so that it now says + + -w, --word-regex(p) + + instead of two lines, one with "regex" and the other with "regexp" + because that confused at least one person since the short forms are the + same. (This required a bit of code, as the output is generated + automatically from a table. It wasn't just a text change.) + + (g) -- can be used to terminate pcregrep options if the next thing isn't an + option but starts with a hyphen. Could be a pattern or a path name + starting with a hyphen, for instance. + + (h) "-" can be given as a file name to represent stdin. + + (i) When file names are being printed, "(standard input)" is used for + the standard input, for compatibility with GNU grep. Previously + "<stdin>" was used. + + (j) The option --label=xxx can be used to supply a name to be used for + stdin when file names are being printed. There is no short form. + + (k) Re-factored the options decoding logic because we are going to add + two more options that take data. Such options can now be given in four + different ways, e.g. "-fname", "-f name", "--file=name", "--file name". + + (l) Added the -A, -B, and -C options for requesting that lines of context + around matches be printed. + + (m) Added the -L option to print the names of files that do not contain + any matching lines, that is, the complement of -l. + + (n) The return code is 2 if any file cannot be opened, but pcregrep does + continue to scan other files. + + (o) The -s option was incorrectly implemented. For compatibility with other + greps, it now suppresses the error message for a non-existent or non- + accessible file (but not the return code). There is a new option called + -q that suppresses the output of matching lines, which was what -s was + previously doing. + + (p) Added --include and --exclude options to specify files for inclusion + and exclusion when recursing. + +11. The Makefile was not using the Autoconf-supported LDFLAGS macro properly. + Hopefully, it now does. + +12. Missing cast in pcre_study(). + +13. Added an "uninstall" target to the makefile. + +14. Replaced "extern" in the function prototypes in Makefile.in with + "PCRE_DATA_SCOPE", which defaults to 'extern' or 'extern "C"' in the Unix + world, but is set differently for Windows. + +15. Added a second compiling function called pcre_compile2(). The only + difference is that it has an extra argument, which is a pointer to an + integer error code. When there is a compile-time failure, this is set + non-zero, in addition to the error test pointer being set to point to an + error message. The new argument may be NULL if no error number is required + (but then you may as well call pcre_compile(), which is now just a + wrapper). This facility is provided because some applications need a + numeric error indication, but it has also enabled me to tidy up the way + compile-time errors are handled in the POSIX wrapper. + +16. Added VPATH=.libs to the makefile; this should help when building with one + prefix path and installing with another. (Or so I'm told by someone who + knows more about this stuff than I do.) + +17. Added a new option, REG_DOTALL, to the POSIX function regcomp(). This + passes PCRE_DOTALL to the pcre_compile() function, making the "." character + match everything, including newlines. This is not POSIX-compatible, but + somebody wanted the feature. From pcretest it can be activated by using + both the P and the s flags. + +18. AC_PROG_LIBTOOL appeared twice in Makefile.in. Removed one. + +19. libpcre.pc was being incorrectly installed as executable. + +20. A couple of places in pcretest check for end-of-line by looking for '\n'; + it now also looks for '\r' so that it will work unmodified on Windows. + +21. Added Google's contributed C++ wrapper to the distribution. + +22. Added some untidy missing memory free() calls in pcretest, to keep + Electric Fence happy when testing. + + + Version 5.0 13-Sep-04 --------------------- @@ -4,18 +4,40 @@ PCRE LICENCE PCRE is a library of functions to support regular expressions whose syntax and semantics are as close as possible to those of the Perl 5 language. -Release 5 of PCRE is distributed under the terms of the "BSD" licence, as +Release 6 of PCRE is distributed under the terms of the "BSD" licence, as specified below. The documentation for PCRE, supplied in the "doc" directory, is distributed under the same terms as the software itself. -Written by: Philip Hazel <ph10@cam.ac.uk> +The basic library functions are written in C and are freestanding. Also +included in the distribution is a set of C++ wrapper functions. + + +THE BASIC LIBRARY FUNCTIONS +--------------------------- + +Written by: Philip Hazel +Email local part: ph10 +Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. Phone: +44 1223 334714. -Copyright (c) 1997-2004 University of Cambridge +Copyright (c) 1997-2005 University of Cambridge +All rights reserved. + + +THE C++ WRAPPER FUNCTIONS +------------------------- + +Contributed by: Google Inc. + +Copyright (c) 2005, Google Inc. All rights reserved. + +THE "BSD" LICENCE +----------------- + Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: @@ -26,9 +48,10 @@ modification, are permitted provided that the following conditions are met: notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - * Neither the name of the University of Cambridge nor the names of its - contributors may be used to endorse or promote products derived from - this software without specific prior written permission. + * Neither the name of the University of Cambridge nor the name of Google + Inc. nor the names of their contributors may be used to endorse or + promote products derived from this software without specific prior + written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE diff --git a/Makefile.in b/Makefile.in index 5354f39..5af2cd2 100644 --- a/Makefile.in +++ b/Makefile.in @@ -18,6 +18,13 @@ PCREGREP_OSTYPE=-DIS_UNIX ############################################################################# +# Libtool places .o files in the .libs directory; this can mean that "make" +# thinks is it not up-to-date when in fact it is. This setting helps when +# GNU "make" is being used. It presumably does no harm in other cases. + +VPATH=.libs + + #---------------------------------------------------------------------------# # The following lines are modified by "configure" to insert data that it is # # given in its arguments, or which it finds out for itself. # @@ -61,12 +68,16 @@ BUILD_OBJEXT = @BUILD_OBJEXT@ # The compiler, C flags, preprocessor flags, etc CC = @CC@ +CXX = @CXX@ CFLAGS = @CFLAGS@ -CPPFLAGS = @CPPFLAGS@ +CXXFLAGS = @CXXFLAGS@ +LDFLAGS = @LDFLAGS@ CC_FOR_BUILD = @CC_FOR_BUILD@ CFLAGS_FOR_BUILD = @CFLAGS_FOR_BUILD@ -CPPFLAGS_FOR_BUILD = @CPPFLAGS_FOR_BUILD@ +CXX_FOR_BUILD = @CXX_FOR_BUILD@ +CXXFLAGS_FOR_BUILD = @CXXFLAGS_FOR_BUILD@ +LDFLAGS_FOR_BUILD = $(LDFLAGS) UCP = @UCP@ UTF8 = @UTF8@ @@ -85,23 +96,82 @@ INSTALL_DATA = @INSTALL_DATA@ LIBTOOL = @LIBTOOL@ LTCOMPILE = $(LIBTOOL) --mode=compile $(CC) -c $(CFLAGS) -I. -I$(top_srcdir) $(NEWLINE) $(LINK_SIZE) $(MATCH_LIMIT) $(NO_RECURSE) $(EBCDIC) -@ON_WINDOWS@LINK = $(CC) $(CFLAGS) -I. -I$(top_srcdir) -L.libs -@NOT_ON_WINDOWS@LINK = $(LIBTOOL) --mode=link $(CC) $(CFLAGS) -I. -I$(top_srcdir) -LINKLIB = $(LIBTOOL) --mode=link $(CC) $(CFLAGS) -I. -I$(top_srcdir) -LINK_FOR_BUILD = $(LIBTOOL) --mode=link $(CC_FOR_BUILD) $(CFLAGS_FOR_BUILD) -I. -I$(top_srcdir) +LTCXXCOMPILE = $(LIBTOOL) --mode=compile $(CXX) -c $(CXXFLAGS) -I. -I$(top_srcdir) $(NEWLINE) $(LINK_SIZE) $(MATCH_LIMIT) $(NO_RECURSE) $(EBCDIC) +@ON_WINDOWS@LINK = $(CC) $(LDFLAGS) -I. -I$(top_srcdir) -L.libs +@NOT_ON_WINDOWS@LINK = $(LIBTOOL) --mode=link $(CC) $(LDFLAGS) -I. -I$(top_srcdir) +LINKLIB = $(LIBTOOL) --mode=link $(CC) $(LDFLAGS) -I. -I$(top_srcdir) +LINK_FOR_BUILD = $(LIBTOOL) --mode=link $(CC_FOR_BUILD) $(LDFLAGS_FOR_BUILD) -I. -I$(top_srcdir) +@ON_WINDOWS@CXXLINK = $(CXX) $(LDFLAGS) -I. -I$(top_srcdir) -L.libs +@NOT_ON_WINDOWS@CXXLINK = $(LIBTOOL) --mode=link $(CXX) $(LDFLAGS) -I. -I$(top_srcdir) +CXXLINKLIB = $(LIBTOOL) --mode=link $(CXX) $(LDFLAGS) -I. -I$(top_srcdir) # These are the version numbers for the shared libraries PCRELIBVERSION = @PCRE_LIB_VERSION@ PCREPOSIXLIBVERSION = @PCRE_POSIXLIB_VERSION@ +PCRECPPLIBVERSION = @PCRE_CPPLIB_VERSION@ ############################################################################## -OBJ = maketables.@OBJEXT@ get.@OBJEXT@ study.@OBJEXT@ pcre.@OBJEXT@ @POSIX_OBJ@ -LOBJ = maketables.lo get.lo study.lo pcre.lo @POSIX_LOBJ@ - -all: libpcre.la @POSIX_LIB@ pcretest@EXEEXT@ pcregrep@EXEEXT@ @ON_WINDOWS@ winshared +OBJ = pcre_chartables.@OBJEXT@ \ + pcre_compile.@OBJEXT@ \ + pcre_config.@OBJEXT@ \ + pcre_dfa_exec.@OBJEXT@ \ + pcre_exec.@OBJEXT@ \ + pcre_fullinfo.@OBJEXT@ \ + pcre_get.@OBJEXT@ \ + pcre_globals.@OBJEXT@ \ + pcre_info.@OBJEXT@ \ + pcre_maketables.@OBJEXT@ \ + pcre_ord2utf8.@OBJEXT@ \ + pcre_printint.@OBJEXT@ \ + pcre_refcount.@OBJEXT@ \ + pcre_study.@OBJEXT@ \ + pcre_tables.@OBJEXT@ \ + pcre_try_flipped.@OBJEXT@ \ + pcre_ucp_findchar.@OBJEXT@ \ + pcre_valid_utf8.@OBJEXT@ \ + pcre_version.@OBJEXT@ \ + pcre_xclass.@OBJEXT@ \ + @POSIX_OBJ@ + +LOBJ = pcre_chartables.lo \ + pcre_compile.lo \ + pcre_config.lo \ + pcre_dfa_exec.lo \ + pcre_exec.lo \ + pcre_fullinfo.lo \ + pcre_get.lo \ + pcre_globals.lo \ + pcre_info.lo \ + pcre_maketables.lo \ + pcre_ord2utf8.lo \ + pcre_printint.lo \ + pcre_refcount.lo \ + pcre_study.lo \ + pcre_tables.lo \ + pcre_try_flipped.lo \ + pcre_ucp_findchar.lo \ + pcre_valid_utf8.lo \ + pcre_version.lo \ + pcre_xclass.lo \ + @POSIX_LOBJ@ + +CPPOBJ = pcrecpp.@OBJEXT@ \ + pcre_scanner.@OBJEXT@ \ + pcre_stringpiece.@OBJEXT@ + +CPPLOBJ = pcrecpp.lo \ + pcre_scanner.lo \ + pcre_stringpiece.lo + +CPP_TARGETS = libpcrecpp.la \ + pcrecpp_unittest@EXEEXT@ \ + pcre_scanner_unittest@EXEEXT@ \ + pcre_stringpiece_unittest@EXEEXT@ + +all: libpcre.la @POSIX_LIB@ @MAYBE_CPP_TARGETS@ pcretest@EXEEXT@ pcregrep@EXEEXT@ @ON_WINDOWS@ winshared pcregrep@EXEEXT@: libpcre.la pcregrep.@OBJEXT@ @ON_WINDOWS@ winshared $(LINK) -o pcregrep@EXEEXT@ pcregrep.@OBJEXT@ libpcre.la @@ -110,6 +180,18 @@ pcretest@EXEEXT@: libpcre.la @POSIX_LIB@ pcretest.@OBJEXT@ @ON_WINDOWS@ winshare $(LINK) $(PURIFY) $(EFENCE) -o pcretest@EXEEXT@ pcretest.@OBJEXT@ \ libpcre.la @POSIX_LIB@ +pcrecpp_unittest@EXEEXT@: libpcrecpp.la pcrecpp_unittest.@OBJEXT@ @ON_WINDOWS@ winshared + $(CXXLINK) $(PURIFY) $(EFENCE) -o pcrecpp_unittest@EXEEXT@ pcrecpp_unittest.@OBJEXT@ \ + libpcrecpp.la @POSIX_LIB@ + +pcre_scanner_unittest@EXEEXT@: libpcrecpp.la pcre_scanner_unittest.@OBJEXT@ @ON_WINDOWS@ winshared + $(CXXLINK) $(PURIFY) $(EFENCE) -o pcre_scanner_unittest@EXEEXT@ pcre_scanner_unittest.@OBJEXT@ \ + libpcrecpp.la @POSIX_LIB@ + +pcre_stringpiece_unittest@EXEEXT@: libpcrecpp.la pcre_stringpiece_unittest.@OBJEXT@ @ON_WINDOWS@ winshared + $(CXXLINK) $(PURIFY) $(EFENCE) -o pcre_stringpiece_unittest@EXEEXT@ pcre_stringpiece_unittest.@OBJEXT@ \ + libpcrecpp.la @POSIX_LIB@ + libpcre.la: $(OBJ) -rm -f libpcre.la $(LINKLIB) -rpath $(LIBDIR) -version-info \ @@ -120,42 +202,155 @@ libpcreposix.la: libpcre.la pcreposix.@OBJEXT@ $(LINKLIB) -rpath $(LIBDIR) libpcre.la -version-info \ '$(PCREPOSIXLIBVERSION)' -o libpcreposix.la pcreposix.lo -# Note that chartables.c is in the current directory, not the source directory. +libpcrecpp.la: libpcre.la $(CPPOBJ) + -rm -f libpcrecpp.la + $(CXXLINKLIB) -rpath $(LIBDIR) libpcre.la -version-info \ + '$(PCRECPPLIBVERSION)' -o libpcrecpp.la $(CPPLOBJ) + +# Note that files generated by ./configure and by dftables are in the current +# directory, not the source directory. + +pcre_chartables.@OBJEXT@: pcre_chartables.c + $(LTCOMPILE) $(top_srcdir)/pcre_chartables.c + +pcre_compile.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_compile.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_compile.c + +pcre_config.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_config.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_config.c + +pcre_dfa_exec.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_dfa_exec.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_dfa_exec.c + +pcre_exec.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_exec.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_exec.c + +pcre_fullinfo.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_fullinfo.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_fullinfo.c + +pcre_get.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_get.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_get.c + +pcre_globals.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_globals.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_globals.c + +pcre_info.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_info.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_info.c + +pcre_maketables.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_maketables.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_maketables.c + +pcre_ord2utf8.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_ord2utf8.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_ord2utf8.c + +pcre_printint.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_printint.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_printint.c + +pcre_refcount.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_refcount.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_refcount.c + +pcre_study.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_study.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_study.c + +pcre_tables.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_tables.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_tables.c + +pcre_try_flipped.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_try_flipped.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_try_flipped.c + +pcre_ucp_findchar.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_ucp_findchar.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_ucp_findchar.c + +pcre_valid_utf8.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_valid_utf8.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_valid_utf8.c + +pcre_version.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_version.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_version.c + +pcre_xclass.@OBJEXT@: Makefile config.h pcre.h \ + $(top_srcdir)/pcre_internal.h $(top_srcdir)/pcre_xclass.c + $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_xclass.c + +pcreposix.@OBJEXT@: $(top_srcdir)/pcreposix.c $(top_srcdir)/pcreposix.h \ + $(top_srcdir)/pcre_internal.h pcre.h config.h Makefile + $(LTCOMPILE) $(POSIX_MALLOC_THRESHOLD) $(top_srcdir)/pcreposix.c -pcre.@OBJEXT@: chartables.c $(top_srcdir)/pcre.c \ - $(top_srcdir)/internal.h $(top_srcdir)/printint.c \ - $(top_srcdir)/ucp.c $(top_srcdir)/ucp.h $(top_srcdir)/ucpinternal.h \ - $(top_srcdir)/ucptable.c $(top_srcdir)/ucptypetable.c \ - pcre.h config.h Makefile - $(LTCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) $(top_srcdir)/pcre.c +pcrecpp.@OBJEXT@: $(top_srcdir)/pcrecpp.cc pcrecpp.h \ + pcre_stringpiece.h pcre.h config.h Makefile + $(LTCXXCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcrecpp.cc -pcreposix.@OBJEXT@: $(top_srcdir)/pcreposix.c $(top_srcdir)/pcreposix.h \ - $(top_srcdir)/internal.h pcre.h config.h Makefile - $(LTCOMPILE) $(POSIX_MALLOC_THRESHOLD) $(top_srcdir)/pcreposix.c +pcre_scanner.@OBJEXT@: $(top_srcdir)/pcre_scanner.cc $(top_srcdir)/pcre_scanner.h \ + pcrecpp.h pcre_stringpiece.h pcre.h config.h Makefile + $(LTCXXCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_scanner.cc -maketables.@OBJEXT@: $(top_srcdir)/maketables.c $(top_srcdir)/internal.h \ - pcre.h config.h Makefile - $(LTCOMPILE) $(top_srcdir)/maketables.c +pcre_stringpiece.@OBJEXT@: $(top_srcdir)/pcre_stringpiece.cc pcre_stringpiece.h \ + config.h Makefile + $(LTCXXCOMPILE) $(UTF8) $(UCP) $(POSIX_MALLOC_THRESHOLD) \ + $(top_srcdir)/pcre_stringpiece.cc -get.@OBJEXT@: $(top_srcdir)/get.c $(top_srcdir)/internal.h \ +pcretest.@OBJEXT@: $(top_srcdir)/pcretest.c $(top_srcdir)/pcre_internal.h \ pcre.h config.h Makefile - $(LTCOMPILE) $(top_srcdir)/get.c + $(CC) -c $(CFLAGS) -I. $(UTF8) $(UCP) $(LINK_SIZE) $(top_srcdir)/pcretest.c -study.@OBJEXT@: $(top_srcdir)/study.c $(top_srcdir)/internal.h \ - pcre.h config.h Makefile - $(LTCOMPILE) $(UTF8) $(UCP) $(top_srcdir)/study.c +pcrecpp_unittest.@OBJEXT@: $(top_srcdir)/pcrecpp_unittest.cc pcrecpp.h \ + pcre_stringpiece.h pcre.h config.h Makefile + $(CXX) -c $(CXXFLAGS) -I. $(UTF8) $(UCP) $(LINK_SIZE) $(top_srcdir)/pcrecpp_unittest.cc + +pcre_stringpiece_unittest.@OBJEXT@: $(top_srcdir)/pcre_stringpiece_unittest.cc \ + pcre_stringpiece.h config.h Makefile + $(CXX) -c $(CXXFLAGS) -I. $(UTF8) $(UCP) $(LINK_SIZE) $(top_srcdir)/pcre_stringpiece_unittest.cc -pcretest.@OBJEXT@: $(top_srcdir)/pcretest.c $(top_srcdir)/internal.h \ - $(top_srcdir)/printint.c \ +pcre_scanner_unittest.@OBJEXT@: $(top_srcdir)/pcre_scanner_unittest.cc \ + $(top_srcdir)/pcre_scanner.h pcrecpp.h pcre_stringpiece.h \ pcre.h config.h Makefile - $(CC) -c $(CFLAGS) -I. $(UTF8) $(UCP) $(LINK_SIZE) $(top_srcdir)/pcretest.c + $(CXX) -c $(CXXFLAGS) -I. $(UTF8) $(UCP) $(LINK_SIZE) $(top_srcdir)/pcre_scanner_unittest.cc pcregrep.@OBJEXT@: $(top_srcdir)/pcregrep.c pcre.h Makefile config.h $(CC) -c $(CFLAGS) -I. $(UTF8) $(UCP) $(PCREGREP_OSTYPE) $(top_srcdir)/pcregrep.c # Some Windows-specific targets for MinGW. Do not use for Cygwin. -winshared : .libs/@WIN_PREFIX@pcre.dll .libs/@WIN_PREFIX@pcreposix.dll +winshared : .libs/@WIN_PREFIX@pcre.dll .libs/@WIN_PREFIX@pcreposix.dll \ + .libs/@WIN_PREFIX@pcrecpp.dll .libs/@WIN_PREFIX@pcre.dll : libpcre.la $(CC) $(CFLAGS) -shared -o $@ \ @@ -167,11 +362,11 @@ winshared : .libs/@WIN_PREFIX@pcre.dll .libs/@WIN_PREFIX@pcreposix.dll sed -e "s#dlname=''#dlname='../bin/@WIN_PREFIX@pcre.dll'#" \ -e "s#library_names=''#library_names='libpcre.dll.a'#" \ < .libs/libpcre.lai > .libs/libpcre.lai.tmp && \ - mv .libs/libpcre.lai.tmp .libs/libpcre.lai + mv -f .libs/libpcre.lai.tmp .libs/libpcre.lai sed -e "s#dlname=''#dlname='../bin/@WIN_PREFIX@pcre.dll'#" \ -e "s#library_names=''#library_names='libpcre.dll.a'#" \ < libpcre.la > libpcre.la.tmp && \ - mv libpcre.la.tmp libpcre.la + mv -f libpcre.la.tmp libpcre.la .libs/@WIN_PREFIX@pcreposix.dll: libpcreposix.la libpcre.la @@ -184,11 +379,27 @@ winshared : .libs/@WIN_PREFIX@pcre.dll .libs/@WIN_PREFIX@pcreposix.dll sed -e "s#dlname=''#dlname='../bin/@WIN_PREFIX@pcreposix.dll'#" \ -e "s#library_names=''#library_names='libpcreposix.dll.a'#"\ < .libs/libpcreposix.lai > .libs/libpcreposix.lai.tmp && \ - mv .libs/libpcreposix.lai.tmp .libs/libpcreposix.lai + mv -f .libs/libpcreposix.lai.tmp .libs/libpcreposix.lai sed -e "s#dlname=''#dlname='../bin/@WIN_PREFIX@pcreposix.dll'#" \ -e "s#library_names=''#library_names='libpcreposix.dll.a'#"\ < libpcreposix.la > libpcreposix.la.tmp && \ - mv libpcreposix.la.tmp libpcreposix.la + mv -f libpcreposix.la.tmp libpcreposix.la + +.libs/@WIN_PREFIX@pcrecpp.dll: libpcrecpp.la libpcre.la + $(CXX) $(CXXFLAGS) -shared -o $@ \ + -Wl,--whole-archive .libs/libpcrecpp.a \ + -Wl,--out-implib,.libs/@WIN_PREFIX@pcrecpp.dll.a \ + -Wl,--output-def,.libs/@WIN_PREFIX@libpcrecpp.dll-def \ + -Wl,--export-all-symbols \ + -Wl,--no-whole-archive .libs/libpcre.a + sed -e "s#dlname=''#dlname='../bin/@WIN_PREFIX@pcrecpp.dll'#" \ + -e "s#library_names=''#library_names='libpcrecpp.dll.a'#"\ + < .libs/libpcrecpp.lai > .libs/libpcrecpp.lai.tmp && \ + mv -f .libs/libpcrecpp.lai.tmp .libs/libpcrecpp.lai + sed -e "s#dlname=''#dlname='../bin/@WIN_PREFIX@pcrecpp.dll'#" \ + -e "s#library_names=''#library_names='libpcrecpp.dll.a'#"\ + < libpcrecpp.la > libpcrecpp.la.tmp && \ + mv -f libpcrecpp.la.tmp libpcrecpp.la wininstall : winshared @@ -198,23 +409,27 @@ wininstall : winshared $(INSTALL) .libs/@WIN_PREFIX@pcreposix.dll $(DESTDIR)$(BINDIR)/@WIN_PREFIX@pcreposix.dll $(INSTALL) .libs/@WIN_PREFIX@libpcreposix.dll.a $(DESTDIR)$(LIBDIR)/@WIN_PREFIX@libpcreposix.dll.a $(INSTALL) .libs/@WIN_PREFIX@libpcre.dll.a $(DESTDIR)$(LIBDIR)/@WIN_PREFIX@libpcre.dll.a + $(INSTALL) .libs/@WIN_PREFIX@pcrecpp.dll $(DESTDIR)$(BINDIR)/@WIN_PREFIX@pcrecpp.dll + $(INSTALL) .libs/@WIN_PREFIX@libpcrecpp.dll.a $(DESTDIR)$(LIBDIR)/@WIN_PREFIX@libpcrecpp.dll.a -strip -g $(DESTDIR)$(BINDIR)/@WIN_PREFIX@pcre.dll -strip -g $(DESTDIR)$(BINDIR)/@WIN_PREFIX@pcreposix.dll + -strip -g $(DESTDIR)$(BINDIR)/@WIN_PREFIX@pcrecpp.dll -strip $(DESTDIR)$(BINDIR)/pcregrep@EXEEXT@ -strip $(DESTDIR)$(BINDIR)/pcretest@EXEEXT@ # An auxiliary program makes the default character table source. This is put # in the current directory, NOT the $top_srcdir directory. -chartables.c: dftables - ./dftables chartables.c +pcre_chartables.c: dftables@BUILD_EXEEXT@ + ./dftables@BUILD_EXEEXT@ pcre_chartables.c -dftables.@BUILD_OBJEXT@: $(top_srcdir)/dftables.c $(top_srcdir)/maketables.c \ - $(top_srcdir)/internal.h pcre.h config.h Makefile +dftables.@BUILD_OBJEXT@: $(top_srcdir)/dftables.c \ + $(top_srcdir)/pcre_maketables.c $(top_srcdir)/pcre_internal.h \ + pcre.h config.h Makefile $(CC_FOR_BUILD) -c $(CFLAGS_FOR_BUILD) -I. $(top_srcdir)/dftables.c -dftables: dftables.@BUILD_OBJEXT@ - $(LINK_FOR_BUILD) -o dftables dftables.@OBJEXT@ +dftables@BUILD_EXEEXT@: dftables.@BUILD_OBJEXT@ + $(LINK_FOR_BUILD) -o dftables@BUILD_EXEEXT@ dftables.@OBJEXT@ install: all @ON_WINDOWS@ wininstall @NOT_ON_WINDOWS@ $(mkinstalldirs) $(DESTDIR)$(LIBDIR) @@ -222,16 +437,23 @@ install: all @ON_WINDOWS@ wininstall @NOT_ON_WINDOWS@ $(LIBTOOL) --mode=install $(INSTALL) libpcre.la $(DESTDIR)$(LIBDIR)/libpcre.la @NOT_ON_WINDOWS@ echo "$(LIBTOOL) --mode=install $(INSTALL) libpcreposix.la $(DESTDIR)$(LIBDIR)/libpcreposix.la" @NOT_ON_WINDOWS@ $(LIBTOOL) --mode=install $(INSTALL) libpcreposix.la $(DESTDIR)$(LIBDIR)/libpcreposix.la +@NOT_ON_WINDOWS@ echo "$(LIBTOOL) --mode=install $(INSTALL) libpcrecpp.la $(DESTDIR)$(LIBDIR)/libpcrecpp.la" +@NOT_ON_WINDOWS@ $(LIBTOOL) --mode=install $(INSTALL) libpcrecpp.la $(DESTDIR)$(LIBDIR)/libpcrecpp.la @NOT_ON_WINDOWS@ $(LIBTOOL) --finish $(DESTDIR)$(LIBDIR) $(mkinstalldirs) $(DESTDIR)$(INCDIR) $(INSTALL_DATA) pcre.h $(DESTDIR)$(INCDIR)/pcre.h $(INSTALL_DATA) $(top_srcdir)/pcreposix.h $(DESTDIR)$(INCDIR)/pcreposix.h + $(INSTALL_DATA) pcrecpp.h $(DESTDIR)$(INCDIR)/pcrecpp.h + $(INSTALL_DATA) pcre_stringpiece.h $(DESTDIR)$(INCDIR)/pcre_stringpiece.h + $(INSTALL_DATA) $(top_srcdir)/pcre_scanner.h $(DESTDIR)$(INCDIR)/pcre_scanner.h $(mkinstalldirs) $(DESTDIR)$(MANDIR)/man3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre.3 $(DESTDIR)$(MANDIR)/man3/pcre.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcreapi.3 $(DESTDIR)$(MANDIR)/man3/pcreapi.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcrebuild.3 $(DESTDIR)$(MANDIR)/man3/pcrebuild.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcrecallout.3 $(DESTDIR)$(MANDIR)/man3/pcrecallout.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcrecompat.3 $(DESTDIR)$(MANDIR)/man3/pcrecompat.3 + $(INSTALL_DATA) $(top_srcdir)/doc/pcrecpp.3 $(DESTDIR)$(MANDIR)/man3/pcrecpp.3 + $(INSTALL_DATA) $(top_srcdir)/doc/pcrematching.3 $(DESTDIR)$(MANDIR)/man3/pcrematching.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcrepattern.3 $(DESTDIR)$(MANDIR)/man3/pcrepattern.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcreperform.3 $(DESTDIR)$(MANDIR)/man3/pcreperform.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcreposix.3 $(DESTDIR)$(MANDIR)/man3/pcreposix.3 @@ -240,6 +462,7 @@ install: all @ON_WINDOWS@ wininstall $(INSTALL_DATA) $(top_srcdir)/doc/pcre_config.3 $(DESTDIR)$(MANDIR)/man3/pcre_config.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_copy_named_substring.3 $(DESTDIR)$(MANDIR)/man3/pcre_copy_named_substring.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_copy_substring.3 $(DESTDIR)$(MANDIR)/man3/pcre_copy_substring.3 + $(INSTALL_DATA) $(top_srcdir)/doc/pcre_dfa_exec.3 $(DESTDIR)$(MANDIR)/man3/pcre_dfa_exec.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_exec.3 $(DESTDIR)$(MANDIR)/man3/pcre_exec.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_free_substring.3 $(DESTDIR)$(MANDIR)/man3/pcre_free_substring.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_free_substring_list.3 $(DESTDIR)$(MANDIR)/man3/pcre_free_substring_list.3 @@ -250,6 +473,7 @@ install: all @ON_WINDOWS@ wininstall $(INSTALL_DATA) $(top_srcdir)/doc/pcre_get_substring_list.3 $(DESTDIR)$(MANDIR)/man3/pcre_get_substring_list.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_info.3 $(DESTDIR)$(MANDIR)/man3/pcre_info.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_maketables.3 $(DESTDIR)$(MANDIR)/man3/pcre_maketables.3 + $(INSTALL_DATA) $(top_srcdir)/doc/pcre_refcount.3 $(DESTDIR)$(MANDIR)/man3/pcre_refcount.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_study.3 $(DESTDIR)$(MANDIR)/man3/pcre_study.3 $(INSTALL_DATA) $(top_srcdir)/doc/pcre_version.3 $(DESTDIR)$(MANDIR)/man3/pcre_version.3 $(mkinstalldirs) $(DESTDIR)$(MANDIR)/man1 @@ -260,18 +484,67 @@ install: all @ON_WINDOWS@ wininstall $(LIBTOOL) --mode=install $(INSTALL) pcretest@EXEEXT@ $(DESTDIR)$(BINDIR)/pcretest@EXEEXT@ $(INSTALL) pcre-config $(DESTDIR)$(BINDIR)/pcre-config $(mkinstalldirs) $(DESTDIR)$(LIBDIR)/pkgconfig - $(INSTALL) libpcre.pc $(DESTDIR)$(LIBDIR)/pkgconfig/libpcre.pc - -# We deliberately omit dftables and chartables.c from 'make clean'; once made -# chartables.c shouldn't change, and if people have edited the tables by hand, -# you don't want to throw them away. - -clean:; -rm -rf *.@OBJEXT@ *.lo *.a *.la .libs pcretest@EXEEXT@ pcregrep@EXEEXT@ testtry + $(INSTALL_DATA) libpcre.pc $(DESTDIR)$(LIBDIR)/pkgconfig/libpcre.pc + +# The uninstall target removes all the files that were installed. + +uninstall:; -rm -rf \ + $(DESTDIR)$(LIBDIR)/libpcre.* \ + $(DESTDIR)$(LIBDIR)/libpcreposix.* \ + $(DESTDIR)$(LIBDIR)/libpcrecpp.* \ + $(DESTDIR)$(INCDIR)/pcre.h \ + $(DESTDIR)$(INCDIR)/pcreposix.h \ + $(DESTDIR)$(INCDIR)/pcrecpp.h \ + $(DESTDIR)$(INCDIR)/pcre_scanner.h \ + $(DESTDIR)$(INCDIR)/pcre_stringpiece.h \ + $(DESTDIR)$(MANDIR)/man3/pcre.3 \ + $(DESTDIR)$(MANDIR)/man3/pcreapi.3 \ + $(DESTDIR)$(MANDIR)/man3/pcrebuild.3 \ + $(DESTDIR)$(MANDIR)/man3/pcrecallout.3 \ + $(DESTDIR)$(MANDIR)/man3/pcrecompat.3 \ + $(DESTDIR)$(MANDIR)/man3/pcrecpp.3 \ + $(DESTDIR)$(MANDIR)/man3/pcrematching.3 \ + $(DESTDIR)$(MANDIR)/man3/pcrepattern.3 \ + $(DESTDIR)$(MANDIR)/man3/pcreperform.3 \ + $(DESTDIR)$(MANDIR)/man3/pcreposix.3 \ + $(DESTDIR)$(MANDIR)/man3/pcresample.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_compile.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_config.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_copy_named_substring.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_copy_substring.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_dfa_exec.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_exec.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_free_substring.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_free_substring_list.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_fullinfo.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_get_named_substring.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_get_stringnumber.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_get_substring.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_get_substring_list.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_info.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_maketables.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_refcount.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_study.3 \ + $(DESTDIR)$(MANDIR)/man3/pcre_version.3 \ + $(DESTDIR)$(MANDIR)/man1/pcregrep.1 \ + $(DESTDIR)$(MANDIR)/man1/pcretest.1 \ + $(DESTDIR)$(BINDIR)/pcregrep@EXEEXT@ \ + $(DESTDIR)$(BINDIR)/pcretest@EXEEXT@ \ + $(DESTDIR)$(BINDIR)/pcre-config \ + $(DESTDIR)$(LIBDIR)/pkgconfig/libpcre.pc + +# We deliberately omit dftables and pcre_chartables.c from 'make clean'; once +# made pcre_chartables.c shouldn't change, and if people have edited the tables +# by hand, you don't want to throw them away. + +clean:; -rm -rf *.@OBJEXT@ *.lo *.a *.la .libs pcretest@EXEEXT@ pcre_stringpiece_unittest@EXEEXT@ pcrecpp_unittest@EXEEXT@ pcre_scanner_unittest@EXEEXT@ pcregrep@EXEEXT@ testtry # But "make distclean" should get back to a virgin distribution distclean: clean - -rm -f chartables.c libtool pcre-config pcre.h \ + -rm -f pcre_chartables.c libtool pcre-config libpcre.pc \ + pcre.h pcre_stringpiece.h pcrecpp.h \ + dftables@EXEEXT@ RunGrepTest RunTest \ Makefile config.h config.status config.log config.cache check: runtest @@ -283,5 +556,14 @@ test: runtest runtest: all @ON_WINDOWS@ @WIN_PREFIX@pcre.dll @./RunTest + @./RunGrepTest + @echo "" + @echo "Testing C++ wrapper" + @echo ""; echo "Test 1++: stringpiece" + @./pcre_stringpiece_unittest@EXEEXT@ + @echo ""; echo "Test 2++: RE class" + @./pcrecpp_unittest@EXEEXT@ + @echo ""; echo "Test 3++: Scanner class" + @./pcre_scanner_unittest@EXEEXT@ # End @@ -1,6 +1,38 @@ News about PCRE releases ------------------------ +Release 6.0 07-Jun-05 +--------------------- + +The release number has been increased to 6.0 because of the addition of several +major new pieces of functionality. + +A new function, pcre_dfa_exec(), which implements pattern matching using a DFA +algorithm, has been added. This has a number of advantages for certain cases, +though it does run more slowly, and lacks the ability to capture substrings. On +the other hand, it does find all matches, not just the first, and it works +better for partial matching. The pcrematching man page discusses the +differences. + +The pcretest program has been enhanced so that it can make use of the new +pcre_dfa_exec() matching function and the extra features it provides. + +The distribution now includes a C++ wrapper library. This is built +automatically if a C++ compiler is found. The pcrecpp man page discusses this +interface. + +The code itself has been re-organized into many more files, one for each +function, so it no longer requires everything to be linked in when static +linkage is used. As a consequence, some internal functions have had to have +their names exposed. These functions all have names starting with _pcre_. They +are undocumented, and are not intended for use by outside callers. + +The pcregrep program has been enhanced with new functionality such as +multiline-matching and options for output more matching context. See the +ChangeLog for a complete list of changes to the library and the utility +programs. + + Release 5.0 13-Sep-04 --------------------- diff --git a/NON-UNIX-USE b/NON-UNIX-USE index f6280af..6612ab2 100644 --- a/NON-UNIX-USE +++ b/NON-UNIX-USE @@ -51,8 +51,8 @@ character tables and writes them to that file. rem Mark Tetrode's commands rem Compile & run - cl -DSUPPORT_UTF8 dftables.c - dftables.exe > chartables.c + cl -DSUPPORT_UTF8 -DSUPPORT_UCP dftables.c + dftables.exe chartables.c (4) Compile maketables.c, get.c, study.c and pcre.c and link them all together into an object library in whichever form your system keeps such @@ -62,7 +62,7 @@ have to do this once for each type. rem Mark Tetrode's commands, for a static library rem Compile & lib - cl -DSUPPORT_UTF8 -DPOSIX_MALLOC_THRESHOLD=10 /c maketables.c get.c study.c pcre.c + cl -DSUPPORT_UTF8 -DSUPPORT_UCP -DPOSIX_MALLOC_THRESHOLD=10 /c maketables.c get.c study.c pcre.c lib /OUT:pcre.lib maketables.obj get.obj study.obj pcre.obj (5) Similarly, compile pcreposix.c and link it (on its own) as the pcreposix @@ -70,7 +70,7 @@ library. rem Mark Tetrode's commands, for a static library rem Compile & lib - cl -DSUPPORT_UTF8 -DPOSIX_MALLOC_THRESHOLD=10 /c pcreposix.c + cl -DSUPPORT_UTF8 -DSUPPORT_UCP -DPOSIX_MALLOC_THRESHOLD=10 /c pcreposix.c lib /OUT:pcreposix.lib pcreposix.obj (6) Compile the test program pcretest.c. This needs the functions in the @@ -78,7 +78,7 @@ pcre and pcreposix libraries when linking. rem Mark Tetrode's commands rem compile & link - cl pcretest.c pcre.lib pcreposix.lib + cl /F0x400000 pcretest.c pcre.lib pcreposix.lib (7) Run pcretest on the testinput files in the testdata directory, and check that the output matches the corresponding testoutput files. You must use the @@ -87,13 +87,6 @@ format, with just LF characters as line terminators. You may need to edit them to change this if your system uses a different convention. rem Mark Tetrode's commands - rem Make a change, i.e. space, backspace, and save again - do this for all - rem to change UNIX to Win, \n to \n\r - write testoutput1 - write testoutput2 - write testoutput3 - write testoutput4 - write testoutput5 pcretest testdata\testinput1 testdata\myoutput1 windiff testdata\testoutput1 testdata\myoutput1 pcretest -i testdata\testinput2 testdata\myoutput2 @@ -104,6 +97,8 @@ to change this if your system uses a different convention. windiff testdata\testoutput4 testdata\myoutput4 pcretest testdata\testinput5 testdata\myoutput5 windiff testdata\testoutput5 testdata\myoutput5 + pcretest testdata\testinput6 testdata\myoutput6 + windiff testdata\testoutput6 testdata\myoutput6 FURTHER REMARKS @@ -7,14 +7,22 @@ The latest release of PCRE is always available from Please read the NEWS file if you are upgrading from a previous release. -PCRE has its own native API, but a set of "wrapper" functions that are based on -the POSIX API are also supplied in the library libpcreposix. Note that this -just provides a POSIX calling interface to PCRE: the regular expressions -themselves still follow Perl syntax and semantics. The header file -for the POSIX-style functions is called pcreposix.h. The official POSIX name is -regex.h, but I didn't want to risk possible problems with existing files of -that name by distributing it that way. To use it with an existing program that -uses the POSIX API, it will have to be renamed or pointed at by a link. + +The PCRE APIs +------------- + +PCRE is written in C, and it has its own API. The distribution now includes a +set of C++ wrapper functions, courtesy of Google Inc. (see the pcrecpp man page +for details). + +Also included are a set of C wrapper functions that are based on the POSIX +API. These end up in the library called libpcreposix. Note that this just +provides a POSIX calling interface to PCRE: the regular expressions themselves +still follow Perl syntax and semantics. The header file for the POSIX-style +functions is called pcreposix.h. The official POSIX name is regex.h, but I +didn't want to risk possible problems with existing files of that name by +distributing it that way. To use it with an existing program that uses the +POSIX API, it will have to be renamed or pointed at by a link. If you are using the POSIX interface to PCRE and there is already a POSIX regex library installed on your system, you must take care when linking programs to @@ -112,7 +120,7 @@ library. You can read more about them in the pcrebuild man page. on the "configure" command. -. PCRE has a counter which can be set to limit the amount of resources it uses. +. PCRE has a counter that can be set to limit the amount of resources it uses. If the limit is exceeded during a match, the match fails. The default is ten million. You can change the default by setting, for example, @@ -130,31 +138,56 @@ library. You can read more about them in the pcrebuild man page. is a representation of the compiled pattern, and this changes with the link size. -. You can build PCRE so that its match() function does not call itself - recursively. Instead, it uses blocks of data from the heap via special - functions pcre_stack_malloc() and pcre_stack_free() to save data that would - otherwise be saved on the stack. To build PCRE like this, use +. You can build PCRE so that its internal match() function that is called from + pcre_exec() does not call itself recursively. Instead, it uses blocks of data + from the heap via special functions pcre_stack_malloc() and pcre_stack_free() + to save data that would otherwise be saved on the stack. To build PCRE like + this, use --disable-stack-for-recursion on the "configure" command. PCRE runs more slowly in this mode, but it may be - necessary in environments with limited stack sizes. + necessary in environments with limited stack sizes. This applies only to the + pcre_exec() function; it does not apply to pcre_dfa_exec(), which does not + use deeply nested recursion. -The "configure" script builds seven files: +The "configure" script builds eight files for the basic C library: -. pcre.h is build by copying pcre.in and making substitutions -. Makefile is built by copying Makefile.in and making substitutions. -. config.h is built by copying config.in and making substitutions. -. pcre-config is built by copying pcre-config.in and making substitutions. -. libpcre.pc is data for the pkg-config command, built from libpcre.pc.in +. pcre.h is the header file for C programs that call PCRE +. Makefile is the makefile that builds the library +. config.h contains build-time configuration options for the library +. pcre-config is a script that shows the settings of "configure" options +. libpcre.pc is data for the pkg-config command . libtool is a script that builds shared and/or static libraries -. RunTest is a script for running tests +. RunTest is a script for running tests on the library +. RunGrepTest is a script for running tests on the pcregrep command + +In addition, if a C++ compiler is found, the following are also built: -Once "configure" has run, you can run "make". It builds two libraries called +. pcrecpp.h is the header file for programs that call PCRE via the C++ wrapper +. pcre_stringpiece.h is the header for the C++ "stringpiece" functions + +The "configure" script also creates config.status, which is an executable +script that can be run to recreate the configuration, and config.log, which +contains compiler output from tests that "configure" runs. + +Once "configure" has run, you can run "make". It builds two libraries, called libpcre and libpcreposix, a test program called pcretest, and the pcregrep -command. You can use "make install" to copy these, the public header files -pcre.h and pcreposix.h, and the man pages to appropriate live directories on -your system, in the normal way. +command. If a C++ compiler was found on your system, it also builds the C++ +wrapper library, which is called libpcrecpp, and some test programs called +pcrecpp_unittest, pcre_scanner_unittest, and pcre_stringpiece_unittest. + +The command "make test" runs all the appropriate tests. Details of the PCRE +tests are given in a separate section of this document, below. + +You can use "make install" to copy the libraries, the public header files +pcre.h, pcreposix.h, pcrecpp.h, and pcre_stringpiece.h (the last two only if +the C++ wrapper was built), and the man pages to appropriate live directories +on your system, in the normal way. + +If you want to remove PCRE from your system, you can run "make uninstall". +This removes all the files that "make install" installed. However, it does not +remove any directories, because these are often shared with other programs. Retrieving configuration information on Unix-like systems @@ -187,9 +220,9 @@ pkgconfig. Shared libraries on Unix-like systems ------------------------------------- -The default distribution builds PCRE as two shared libraries and two static -libraries, as long as the operating system supports shared libraries. Shared -library support relies on the "libtool" script which is built as part of the +The default distribution builds PCRE as shared libraries and static libraries, +as long as the operating system supports shared libraries. Shared library +support relies on the "libtool" script which is built as part of the "configure" process. The libtool script is used to compile and link both shared and static @@ -218,7 +251,8 @@ order to cross-compile PCRE for some other host. However, during the building process, the dftables.c source file is compiled *and run* on the local host, in order to generate the default character tables (the chartables.c file). It therefore needs to be compiled with the local compiler, not the cross compiler. -You can do this by specifying CC_FOR_BUILD (and if necessary CFLAGS_FOR_BUILD) +You can do this by specifying CC_FOR_BUILD (and if necessary CFLAGS_FOR_BUILD; +there are also CXX_FOR_BUILD and CXXFLAGS_FOR_BUILD for the C++ wrapper) when calling the "configure" command. If they are not specified, they default to the values of CC and CFLAGS. @@ -240,13 +274,19 @@ Testing PCRE ------------ To test PCRE on a Unix system, run the RunTest script that is created by the -configuring process. (This can also be run by "make runtest", "make check", or -"make test".) For other systems, see the instructions in NON-UNIX-USE. - -The script runs the pcretest test program (which is documented in its own man -page) on each of the testinput files (in the testdata directory) in turn, -and compares the output with the contents of the corresponding testoutput file. -A file called testtry is used to hold the main output from pcretest +configuring process. There is also a script called RunGrepTest that tests the +options of the pcregrep command. If the C++ wrapper library is build, three +test programs called pcrecpp_unittest, pcre_scanner_unittest, and +pcre_stringpiece_unittest are provided. + +Both the scripts and all the program tests are run if you obey "make runtest", +"make check", or "make test". For other systems, see the instructions in +NON-UNIX-USE. + +The RunTest script runs the pcretest test program (which is documented in its +own man page) on each of the testinput files (in the testdata directory) in +turn, and compares the output with the contents of the corresponding testoutput +file. A file called testtry is used to hold the main output from pcretest (testsavedregex is also used as a working file). To run pcretest on just one of the test files, give its number as an argument to RunTest, for example: @@ -294,9 +334,14 @@ commented in the script, can be be used.) The fifth test checks error handling with UTF-8 encoding, and internal UTF-8 features of PCRE that are not relevant to Perl. -The sixth and final test checks the support for Unicode character properties. -It it not run automatically unless PCRE is built with Unicode property support. -To to this you must set --enable-unicode-properties when running "configure". +The sixth and test checks the support for Unicode character properties. It it +not run automatically unless PCRE is built with Unicode property support. To to +this you must set --enable-unicode-properties when running "configure". + +The seventh, eighth, and ninth tests check the pcre_dfa_exec() alternative +matching function, in non-UTF-8 mode, UTF-8 mode, and UTF-8 mode with Unicode +property support, respectively. The eighth and ninth tests are not run +automatically unless PCRE is build with the relevant support. Character tables @@ -348,14 +393,27 @@ The distribution should contain the following files: dftables.c auxiliary program for building chartables.c - get.c ) - maketables.c ) - study.c ) source of the functions - pcre.c ) in the library pcreposix.c ) - printint.c ) - - ucp.c ) + pcre_compile.c ) + pcre_config.c ) + pcre_dfa_exec.c ) + pcre_exec.c ) + pcre_fullinfo.c ) + pcre_get.c ) sources for the functions in the library, + pcre_globals.c ) and some internal functions that they use + pcre_info.c ) + pcre_maketables.c ) + pcre_ord2utf8.c ) + pcre_printint.c ) + pcre_study.c ) + pcre_tables.c ) + pcre_try_flipped.c ) + pcre_ucp_findchar.c ) + pcre_valid_utf8.c ) + pcre_version.c ) + pcre_xclass.c ) + + ucp_findchar.c ) ucp.h ) source for the code that is used for ucpinternal.h ) Unicode property handling ucptable.c ) @@ -364,9 +422,17 @@ The distribution should contain the following files: pcre.in "source" for the header for the external API; pcre.h is built from this by "configure" pcreposix.h header for the external POSIX wrapper API - internal.h header for internal use + pcre_internal.h header for internal use config.in template for config.h, which is built by configure + pcrecpp.h.in "source" for the header file for the C++ wrapper + pcrecpp.cc ) + pcre_scanner.cc ) source for the C++ wrapper library + + pcre_stringpiece.h.in "source" for pcre_stringpiece.h, the header for the + C++ stringpiece functions + pcre_stringpiece.cc source for the C++ stringpiece functions + (B) Auxiliary files: AUTHORS information about the author of PCRE @@ -379,6 +445,7 @@ The distribution should contain the following files: NON-UNIX-USE notes on building PCRE on non-Unix systems README this file RunTest.in template for a Unix shell script for running tests + RunGrepTest.in template for a Unix shell script for pcregrep tests config.guess ) files used by libtool, config.sub ) used only when building a shared library configure a configuring shell script (built by autoconf) @@ -399,22 +466,15 @@ The distribution should contain the following files: perltest Perl test program pcregrep.c source of a grep utility that uses PCRE pcre-config.in source of script which retains PCRE information - testdata/testinput1 test data, compatible with Perl - testdata/testinput2 test data for error messages and non-Perl things - testdata/testinput3 test data for locale-specific tests - testdata/testinput4 test data for UTF-8 tests compatible with Perl - testdata/testinput5 test data for other UTF-8 tests - testdata/testinput6 test data for Unicode property support tests - testdata/testoutput1 test results corresponding to testinput1 - testdata/testoutput2 test results corresponding to testinput2 - testdata/testoutput3 test results corresponding to testinput3 - testdata/testoutput4 test results corresponding to testinput4 - testdata/testoutput5 test results corresponding to testinput5 - testdata/testoutput6 test results corresponding to testinput6 + pcrecpp_unittest.c ) + pcre_scanner_unittest.c ) test programs for the C++ wrapper + pcre_stringpiece_unittest.c ) + testdata/testinput* test data for main library tests + testdata/testoutput* expected test results + testdata/grep* input and output for pcregrep tests (C) Auxiliary files for Win32 DLL - dll.mk libpcre.def libpcreposix.def pcre.def @@ -423,5 +483,7 @@ The distribution should contain the following files: makevp.bat -Philip Hazel <ph10@cam.ac.uk> -September 2004 +Philip Hazel +Email local part: ph10 +Email domain: cam.ac.uk +June 2005 diff --git a/RunGrepTest.in b/RunGrepTest.in new file mode 100644 index 0000000..57ac403 --- /dev/null +++ b/RunGrepTest.in @@ -0,0 +1,137 @@ +#! /bin/sh + +# This file is generated by configure from RunGrepTest.in. Make any changes +# to that file. + +echo "Testing pcregrep" + +# Run pcregrep tests. The assumption is that the PCRE tests check the library +# itself. What we are checking here is the file handling and options that are +# supported by pcregrep. + +cf=diff +testdata=@top_srcdir@/testdata + +./pcregrep -V 2>testtry + +echo "---------------------------- Test 1 ------------------------------" >>testtry +./pcregrep PATTERN $testdata/grepinput >>testtry + +echo "---------------------------- Test 2 ------------------------------" >>testtry +./pcregrep '^PATTERN' $testdata/grepinput >>testtry + +echo "---------------------------- Test 3 ------------------------------" >>testtry +./pcregrep -in PATTERN $testdata/grepinput >>testtry + +echo "---------------------------- Test 4 ------------------------------" >>testtry +./pcregrep -ic PATTERN $testdata/grepinput >>testtry + +echo "---------------------------- Test 5 ------------------------------" >>testtry +./pcregrep -in PATTERN $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 6 ------------------------------" >>testtry +./pcregrep -inh PATTERN $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 7 ------------------------------" >>testtry +./pcregrep -il PATTERN $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 8 ------------------------------" >>testtry +./pcregrep -l PATTERN $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 9 ------------------------------" >>testtry +./pcregrep -q PATTERN $testdata/grepinput $testdata/grepinputx >>testtry +echo "RC=$?" >>testtry + +echo "---------------------------- Test 10 -----------------------------" >>testtry +./pcregrep -q NEVER-PATTERN $testdata/grepinput $testdata/grepinputx >>testtry +echo "RC=$?" >>testtry + +echo "---------------------------- Test 11 -----------------------------" >>testtry +./pcregrep -vn pattern $testdata/grepinputx >>testtry + +echo "---------------------------- Test 12 -----------------------------" >>testtry +./pcregrep -ix pattern $testdata/grepinputx >>testtry + +echo "---------------------------- Test 13 -----------------------------" >>testtry +./pcregrep -f$testdata/greplist $testdata/grepinputx >>testtry + +echo "---------------------------- Test 14 -----------------------------" >>testtry +./pcregrep -w pat $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 15 -----------------------------" >>testtry +./pcregrep 'abc^*' $testdata/grepinput 2>>testtry >>testtry + +echo "---------------------------- Test 16 -----------------------------" >>testtry +./pcregrep abc $testdata/grepinput $testdata/nonexistfile 2>>testtry >>testtry + +echo "---------------------------- Test 17 -----------------------------" >>testtry +./pcregrep -M 'the\noutput' $testdata/grepinput >>testtry + +echo "---------------------------- Test 18 -----------------------------" >>testtry +./pcregrep -Mn '(the\noutput|dog\.\n--)' $testdata/grepinput >>testtry + +echo "---------------------------- Test 19 -----------------------------" >>testtry +./pcregrep -Mix 'Pattern' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 20 -----------------------------" >>testtry +./pcregrep -Mixn 'complete pair\nof lines' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 21 -----------------------------" >>testtry +./pcregrep -nA3 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 22 -----------------------------" >>testtry +./pcregrep -nB3 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 23 -----------------------------" >>testtry +./pcregrep -C3 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 24 -----------------------------" >>testtry +./pcregrep -A9 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 25 -----------------------------" >>testtry +./pcregrep -nB9 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 26 -----------------------------" >>testtry +./pcregrep -A9 -B9 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 27 -----------------------------" >>testtry +./pcregrep -A10 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 28 -----------------------------" >>testtry +./pcregrep -nB10 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 29 -----------------------------" >>testtry +./pcregrep -C12 -B10 'four' $testdata/grepinputx >>testtry + +echo "---------------------------- Test 30 -----------------------------" >>testtry +./pcregrep -inB3 'pattern' $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 31 -----------------------------" >>testtry +./pcregrep -inA3 'pattern' $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 32 -----------------------------" >>testtry +./pcregrep -L 'fox' $testdata/grepinput $testdata/grepinputx >>testtry + +echo "---------------------------- Test 33 -----------------------------" >>testtry +./pcregrep 'fox' $testdata/grepnonexist >>testtry 2>&1 +echo "RC=$?" >>testtry + +echo "---------------------------- Test 34 -----------------------------" >>testtry +./pcregrep -s 'fox' $testdata/grepnonexist >>testtry 2>&1 +echo "RC=$?" >>testtry + +echo "---------------------------- Test 35 -----------------------------" >>testtry +./pcregrep -L -r --include=grepinputx 'fox' $testdata >>testtry +echo "RC=$?" >>testtry + +echo "---------------------------- Test 36 -----------------------------" >>testtry +./pcregrep -L -r --include=grepinput --exclude 'grepinput$' 'fox' $testdata >>testtry +echo "RC=$?" >>testtry + + +# Now compare the results. + +$cf testtry $testdata/grepoutput +if [ $? != 0 ] ; then exit 1; else exit 0; fi + +# End @@ -16,6 +16,9 @@ do3=no do4=no do5=no do6=no +do7=no +do8=no +do9=no while [ $# -gt 0 ] ; do case $1 in @@ -23,8 +26,11 @@ while [ $# -gt 0 ] ; do 2) do2=yes;; 3) do3=yes;; 4) do4=yes;; - 5) do5=yes;; - 6) do6=yes;; + 5) do5=yes;; + 6) do6=yes;; + 7) do7=yes;; + 8) do8=yes;; + 9) do9=yes;; *) echo "Unknown test number $1"; exit 1;; esac shift @@ -33,48 +39,64 @@ done if [ "@LINK_SIZE@" != "" -a "@LINK_SIZE@" != "-DLINK_SIZE=2" ] ; then if [ $do2 = yes ] ; then echo "Can't run test 2 with an internal link size other than 2" - exit 1 - fi + exit 1 + fi if [ $do5 = yes ] ; then echo "Can't run test 5 with an internal link size other than 2" - exit 1 - fi + exit 1 + fi if [ $do6 = yes ] ; then echo "Can't run test 6 with an internal link size other than 2" - exit 1 - fi + exit 1 + fi fi if [ "@UTF8@" = "" ] ; then if [ $do4 = yes ] ; then echo "Can't run test 4 because UTF-8 support is not configured" exit 1 - fi + fi if [ $do5 = yes ] ; then echo "Can't run test 5 because UTF-8 support is not configured" exit 1 - fi + fi if [ $do6 = yes ] ; then echo "Can't run test 6 because UTF-8 support is not configured" exit 1 - fi -fi + fi + if [ $do8 = yes ] ; then + echo "Can't run test 8 because UTF-8 support is not configured" + exit 1 + fi + if [ $do9 = yes ] ; then + echo "Can't run test 9 because UTF-8 support is not configured" + exit 1 + fi +fi if [ "@UCP@" = "" ] ; then if [ $do6 = yes ] ; then echo "Can't run test 6 because Unicode property support is not configured" exit 1 fi -fi + if [ $do9 = yes ] ; then + echo "Can't run test 9 because Unicode property support is not configured" + exit 1 + fi +fi if [ $do1 = no -a $do2 = no -a $do3 = no -a $do4 = no -a \ - $do5 = no -a $do6 = no ] ; then + $do5 = no -a $do6 = no -a $do7 = no -a $do8 = no -a \ + $do9 = no ] ; then do1=yes - do2=yes + do2=yes do3=yes if [ "@UTF8@" != "" ] ; then do4=yes; fi if [ "@UTF8@" != "" ] ; then do5=yes; fi if [ "@UTF8@" != "" -a "@UCP@" != "" ] ; then do6=yes; fi + do7=yes + if [ "@UTF8@" != "" ] ; then do8=yes; fi + if [ "@UTF8@" != "" -a "@UCP@" != "" ] ; then do9=yes; fi fi # Show which release @@ -89,15 +111,16 @@ if [ $do1 = yes ] ; then if [ $? = 0 ] ; then $cf testtry $testdata/testoutput1 if [ $? != 0 ] ; then exit 1; fi - echo " " else exit 1 fi + echo "OK" + echo " " fi # PCRE tests that are not Perl-compatible - API & error tests, mostly if [ $do2 = yes ] ; then - if [ "@LINK_SIZE@" = "" -o "@LINK_SIZE@" = "-DLINK_SIZE=2" ] ; then + if [ "@LINK_SIZE@" = "" -o "@LINK_SIZE@" = "-DLINK_SIZE=2" ] ; then echo "Test 2: API and error handling (not Perl compatible)" ./pcretest -i $testdata/testinput2 testtry if [ $? = 0 ] ; then @@ -105,15 +128,12 @@ if [ $do2 = yes ] ; then if [ $? != 0 ] ; then exit 1; fi else exit 1 fi + echo "OK" + echo " " else - echo Test 2 skipped for link size other than 2 \(@LINK_SIZE@\) - fi -fi - -if [ $do1 = yes -a $do2 = yes ] ; then - echo " " - echo "The two main tests ran OK" - echo " " + echo Test 2 skipped for link size other than 2 \(@LINK_SIZE@\) + echo " " + fi fi # Locale-specific tests, provided the "fr_FR" locale is available @@ -125,21 +145,21 @@ if [ $do3 = yes ] ; then ./pcretest $testdata/testinput3 testtry if [ $? = 0 ] ; then $cf testtry $testdata/testoutput3 - if [ $? != 0 ] ; then + if [ $? != 0 ] ; then echo " " echo "Locale test did not run entirely successfully." echo "This usually means that there is a problem with the locale" - echo "settings rather than a bug in PCRE." + echo "settings rather than a bug in PCRE." else - echo "Locale test ran OK" - fi - echo " " + echo "OK" + fi + echo " " else exit 1 fi else echo "Cannot test locale-specific features - 'fr_FR' locale not found," echo "or the \"locale\" command is not available to check for it." - echo " " + echo " " fi fi @@ -147,46 +167,86 @@ fi if [ $do4 = yes ] ; then echo "Test 4: UTF-8 support (Perl compatible)" - ./pcretest $testdata/testinput4 testtry + ./pcretest $testdata/testinput4 testtry if [ $? = 0 ] ; then $cf testtry $testdata/testoutput4 if [ $? != 0 ] ; then exit 1; fi else exit 1 fi - echo "UTF8 test ran OK" + echo "OK" echo " " fi if [ $do5 = yes ] ; then - if [ "@LINK_SIZE@" = "" -o "@LINK_SIZE@" = "-DLINK_SIZE=2" ] ; then + if [ "@LINK_SIZE@" = "" -o "@LINK_SIZE@" = "-DLINK_SIZE=2" ] ; then echo "Test 5: API and internals for UTF-8 support (not Perl compatible)" - ./pcretest $testdata/testinput5 testtry + ./pcretest $testdata/testinput5 testtry if [ $? = 0 ] ; then $cf testtry $testdata/testoutput5 if [ $? != 0 ] ; then exit 1; fi else exit 1 fi - echo "UTF8 internals test ran OK" + echo "OK" echo " " else - echo Test 5 skipped for link size other than 2 \(@LINK_SIZE@\) - fi + echo Test 5 skipped for link size other than 2 \(@LINK_SIZE@\) + echo " " + fi fi if [ $do6 = yes ] ; then - if [ "@LINK_SIZE@" = "" -o "@LINK_SIZE@" = "-DLINK_SIZE=2" ] ; then + if [ "@LINK_SIZE@" = "" -o "@LINK_SIZE@" = "-DLINK_SIZE=2" ] ; then echo "Test 6: Unicode property support" - ./pcretest $testdata/testinput6 testtry + ./pcretest $testdata/testinput6 testtry if [ $? = 0 ] ; then $cf testtry $testdata/testoutput6 if [ $? != 0 ] ; then exit 1; fi else exit 1 fi - echo "Unicode properties test ran OK" + echo "OK" echo " " - else - echo Test 6 skipped for link size other than 2 \(@LINK_SIZE@\) - fi + else + echo Test 6 skipped for link size other than 2 \(@LINK_SIZE@\) + echo " " + fi +fi + +# Tests for DFA matching support + +if [ $do7 = yes ] ; then + echo "Test 7: DFA matching" + ./pcretest -dfa $testdata/testinput7 testtry + if [ $? = 0 ] ; then + $cf testtry $testdata/testoutput7 + if [ $? != 0 ] ; then exit 1; fi + else exit 1 + fi + echo "OK" + echo " " +fi + +if [ $do8 = yes ] ; then + echo "Test 8: DFA matching with UTF-8" + ./pcretest -dfa $testdata/testinput8 testtry + if [ $? = 0 ] ; then + $cf testtry $testdata/testoutput8 + if [ $? != 0 ] ; then exit 1; fi + else exit 1 + fi + echo "OK" + echo " " +fi + +if [ $do9 = yes ] ; then + echo "Test 9: DFA matching with Unicode properties" + ./pcretest -dfa $testdata/testinput9 testtry + if [ $? = 0 ] ; then + $cf testtry $testdata/testoutput9 + if [ $? != 0 ] ; then exit 1; fi + else exit 1 + fi + echo "OK" + echo " " fi # End @@ -463,7 +463,7 @@ ac_includes_default="\ # include <unistd.h> #endif" -ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS CC CFLAGS LDFLAGS CPPFLAGS ac_ct_CC EXEEXT OBJEXT INSTALL_PROGRAM INSTALL_SCRIPT INSTALL_DATA build build_cpu build_vendor build_os host host_cpu host_vendor host_os EGREP LN_S ECHO AR ac_ct_AR RANLIB ac_ct_RANLIB STRIP ac_ct_STRIP DLLTOOL ac_ct_DLLTOOL AS ac_ct_AS OBJDUMP ac_ct_OBJDUMP CPP CXX CXXFLAGS ac_ct_CXX CXXCPP F77 FFLAGS ac_ct_F77 LIBTOOL BUILD_EXEEXT BUILD_OBJEXT CC_FOR_BUILD CFLAGS_FOR_BUILD EBCDIC HAVE_MEMMOVE HAVE_STRERROR LINK_SIZE MATCH_LIMIT NEWLINE NO_RECURSE PCRE_MAJOR PCRE_MINOR PCRE_DATE PCRE_VERSION PCRE_LIB_VERSION PCRE_POSIXLIB_VERSION POSIX_MALLOC_THRESHOLD UCP UTF8 WIN_PREFIX ON_WINDOWS NOT_ON_WINDOWS POSIX_OBJ POSIX_LOBJ POSIX_LIB LIBOBJS LTLIBOBJS' +ac_subst_vars='SHELL PATH_SEPARATOR PACKAGE_NAME PACKAGE_TARNAME PACKAGE_VERSION PACKAGE_STRING PACKAGE_BUGREPORT exec_prefix prefix program_transform_name bindir sbindir libexecdir datadir sysconfdir sharedstatedir localstatedir libdir includedir oldincludedir infodir mandir build_alias host_alias target_alias DEFS ECHO_C ECHO_N ECHO_T LIBS CC CFLAGS LDFLAGS CPPFLAGS ac_ct_CC EXEEXT OBJEXT CXX CXXFLAGS ac_ct_CXX MAYBE_CPP_TARGETS INSTALL_PROGRAM INSTALL_SCRIPT INSTALL_DATA build build_cpu build_vendor build_os host host_cpu host_vendor host_os EGREP LN_S ECHO AR ac_ct_AR RANLIB ac_ct_RANLIB STRIP ac_ct_STRIP DLLTOOL ac_ct_DLLTOOL AS ac_ct_AS OBJDUMP ac_ct_OBJDUMP CPP CXXCPP F77 FFLAGS ac_ct_F77 LIBTOOL pcre_has_bits_type_traits pcre_has_type_traits pcre_has_long_long pcre_has_ulong_long BUILD_EXEEXT BUILD_OBJEXT CC_FOR_BUILD CXX_FOR_BUILD CFLAGS_FOR_BUILD CXXFLAGS_FOR_BUILD EBCDIC HAVE_MEMMOVE HAVE_STRERROR LINK_SIZE MATCH_LIMIT NEWLINE NO_RECURSE PCRE_MAJOR PCRE_MINOR PCRE_DATE PCRE_VERSION PCRE_LIB_VERSION PCRE_POSIXLIB_VERSION PCRE_CPPLIB_VERSION POSIX_MALLOC_THRESHOLD UCP UTF8 WIN_PREFIX ON_WINDOWS NOT_ON_WINDOWS POSIX_OBJ POSIX_LOBJ POSIX_LIB LIBOBJS LTLIBOBJS' ac_subst_files='' # Initialize some variables set by options. @@ -920,10 +920,6 @@ ac_env_CPPFLAGS_set=${CPPFLAGS+set} ac_env_CPPFLAGS_value=$CPPFLAGS ac_cv_env_CPPFLAGS_set=${CPPFLAGS+set} ac_cv_env_CPPFLAGS_value=$CPPFLAGS -ac_env_CPP_set=${CPP+set} -ac_env_CPP_value=$CPP -ac_cv_env_CPP_set=${CPP+set} -ac_cv_env_CPP_value=$CPP ac_env_CXX_set=${CXX+set} ac_env_CXX_value=$CXX ac_cv_env_CXX_set=${CXX+set} @@ -932,6 +928,10 @@ ac_env_CXXFLAGS_set=${CXXFLAGS+set} ac_env_CXXFLAGS_value=$CXXFLAGS ac_cv_env_CXXFLAGS_set=${CXXFLAGS+set} ac_cv_env_CXXFLAGS_value=$CXXFLAGS +ac_env_CPP_set=${CPP+set} +ac_env_CPP_value=$CPP +ac_cv_env_CPP_set=${CPP+set} +ac_cv_env_CPP_value=$CPP ac_env_CXXCPP_set=${CXXCPP+set} ac_env_CXXCPP_value=$CXXCPP ac_cv_env_CXXCPP_set=${CXXCPP+set} @@ -950,24 +950,6 @@ ac_cv_env_FFLAGS_value=$FFLAGS # if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. - # The list generated by autoconf has been trimmed to remove many - # options that are totally irrelevant to PCRE (e.g. relating to X), - # or are not supported by its Makefile. - # The list generated by autoconf has been trimmed to remove many - # options that are totally irrelevant to PCRE (e.g. relating to X), - # or are not supported by its Makefile. - # The list generated by autoconf has been trimmed to remove many - # options that are totally irrelevant to PCRE (e.g. relating to X), - # or are not supported by its Makefile. - # The list generated by autoconf has been trimmed to remove many - # options that are totally irrelevant to PCRE (e.g. relating to X), - # or are not supported by its Makefile. - # The list generated by autoconf has been trimmed to remove many - # options that are totally irrelevant to PCRE (e.g. relating to X), - # or are not supported by its Makefile. - # The list generated by autoconf has been trimmed to remove many - # options that are totally irrelevant to PCRE (e.g. relating to X), - # or are not supported by its Makefile. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF \`configure' configures this package to adapt to many kinds of systems. @@ -1008,13 +990,24 @@ For better control, use the options below. Fine tuning of the installation directories: --bindir=DIR user executables [EPREFIX/bin] + --sbindir=DIR system admin executables [EPREFIX/sbin] + --libexecdir=DIR program executables [EPREFIX/libexec] + --datadir=DIR read-only architecture-independent data [PREFIX/share] + --sysconfdir=DIR read-only single-machine data [PREFIX/etc] + --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] + --localstatedir=DIR modifiable single-machine data [PREFIX/var] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] + --oldincludedir=DIR C header files for non-gcc [/usr/include] + --infodir=DIR info documentation [PREFIX/info] --mandir=DIR man documentation [PREFIX/man] _ACEOF cat <<\_ACEOF +System types: + --build=BUILD configure for building on BUILD [guessed] + --host=HOST cross-compile to build programs to run on HOST [BUILD] _ACEOF fi @@ -1025,11 +1018,11 @@ if test -n "$ac_init_help"; then Optional Features: --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] - --enable-shared[ ] + --enable-shared[=PKGS] build shared libraries [default=yes] - --enable-static[ ] + --enable-static[=PKGS] build static libraries [default=yes] - --enable-fast-install[ ] + --enable-fast-install[=PKGS] optimize for fast installation [default=yes] --disable-libtool-lock avoid locking (might break parallel builds) --enable-utf8 enable UTF8 support @@ -1040,6 +1033,8 @@ Optional Features: --disable-stack-for-recursion disable use of stack recursion when matching Optional Packages: + --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] + --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-gnu-ld assume the C compiler uses GNU ld [default=no] --with-pic try to use only PIC/non-PIC objects [default=use both] @@ -1056,9 +1051,9 @@ Some influential environment variables: nonstandard directory <lib dir> CPPFLAGS C/C++ preprocessor flags, e.g. -I<include dir> if you have headers in a nonstandard directory <include dir> - CPP C preprocessor CXX C++ compiler command CXXFLAGS C++ compiler flags + CPP C preprocessor CXXCPP C++ preprocessor F77 Fortran 77 compiler command FFLAGS Fortran 77 compiler flags @@ -1509,9 +1504,9 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu -PCRE_MAJOR=5 +PCRE_MAJOR=6 PCRE_MINOR=0 -PCRE_DATE=13-Sep-2004 +PCRE_DATE=07-Jun-2005 PCRE_VERSION=${PCRE_MAJOR}.${PCRE_MINOR} @@ -1520,6 +1515,7 @@ POSIX_MALLOC_THRESHOLD=-DPOSIX_MALLOC_THRESHOLD=10 PCRE_LIB_VERSION=0:1:0 PCRE_POSIXLIB_VERSION=0:0:0 +PCRE_CPPLIB_VERSION=0:0:0 ac_ext=c @@ -2454,6 +2450,369 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu +ac_ext=cc +ac_cpp='$CXXCPP $CPPFLAGS' +ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' +ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' +ac_compiler_gnu=$ac_cv_cxx_compiler_gnu +if test -n "$ac_tool_prefix"; then + for ac_prog in $CCC g++ c++ gpp aCC CC cxx cc++ cl FCC KCC RCC xlC_r xlC + do + # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. +set dummy $ac_tool_prefix$ac_prog; ac_word=$2 +echo "$as_me:$LINENO: checking for $ac_word" >&5 +echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6 +if test "${ac_cv_prog_CXX+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + if test -n "$CXX"; then + ac_cv_prog_CXX="$CXX" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_prog_CXX="$ac_tool_prefix$ac_prog" + echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done +done + +fi +fi +CXX=$ac_cv_prog_CXX +if test -n "$CXX"; then + echo "$as_me:$LINENO: result: $CXX" >&5 +echo "${ECHO_T}$CXX" >&6 +else + echo "$as_me:$LINENO: result: no" >&5 +echo "${ECHO_T}no" >&6 +fi + + test -n "$CXX" && break + done +fi +if test -z "$CXX"; then + ac_ct_CXX=$CXX + for ac_prog in $CCC g++ c++ gpp aCC CC cxx cc++ cl FCC KCC RCC xlC_r xlC +do + # Extract the first word of "$ac_prog", so it can be a program name with args. +set dummy $ac_prog; ac_word=$2 +echo "$as_me:$LINENO: checking for $ac_word" >&5 +echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6 +if test "${ac_cv_prog_ac_ct_CXX+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + if test -n "$ac_ct_CXX"; then + ac_cv_prog_ac_ct_CXX="$ac_ct_CXX" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_prog_ac_ct_CXX="$ac_prog" + echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done +done + +fi +fi +ac_ct_CXX=$ac_cv_prog_ac_ct_CXX +if test -n "$ac_ct_CXX"; then + echo "$as_me:$LINENO: result: $ac_ct_CXX" >&5 +echo "${ECHO_T}$ac_ct_CXX" >&6 +else + echo "$as_me:$LINENO: result: no" >&5 +echo "${ECHO_T}no" >&6 +fi + + test -n "$ac_ct_CXX" && break +done +test -n "$ac_ct_CXX" || ac_ct_CXX="g++" + + CXX=$ac_ct_CXX +fi + + +# Provide some information about the compiler. +echo "$as_me:$LINENO:" \ + "checking for C++ compiler version" >&5 +ac_compiler=`set X $ac_compile; echo $2` +{ (eval echo "$as_me:$LINENO: \"$ac_compiler --version </dev/null >&5\"") >&5 + (eval $ac_compiler --version </dev/null >&5) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } +{ (eval echo "$as_me:$LINENO: \"$ac_compiler -v </dev/null >&5\"") >&5 + (eval $ac_compiler -v </dev/null >&5) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } +{ (eval echo "$as_me:$LINENO: \"$ac_compiler -V </dev/null >&5\"") >&5 + (eval $ac_compiler -V </dev/null >&5) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } + +echo "$as_me:$LINENO: checking whether we are using the GNU C++ compiler" >&5 +echo $ECHO_N "checking whether we are using the GNU C++ compiler... $ECHO_C" >&6 +if test "${ac_cv_cxx_compiler_gnu+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ + +int +main () +{ +#ifndef __GNUC__ + choke me +#endif + + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_cxx_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_compiler_gnu=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +ac_compiler_gnu=no +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +ac_cv_cxx_compiler_gnu=$ac_compiler_gnu + +fi +echo "$as_me:$LINENO: result: $ac_cv_cxx_compiler_gnu" >&5 +echo "${ECHO_T}$ac_cv_cxx_compiler_gnu" >&6 +GXX=`test $ac_compiler_gnu = yes && echo yes` +ac_test_CXXFLAGS=${CXXFLAGS+set} +ac_save_CXXFLAGS=$CXXFLAGS +CXXFLAGS="-g" +echo "$as_me:$LINENO: checking whether $CXX accepts -g" >&5 +echo $ECHO_N "checking whether $CXX accepts -g... $ECHO_C" >&6 +if test "${ac_cv_prog_cxx_g+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_cxx_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_cv_prog_cxx_g=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +ac_cv_prog_cxx_g=no +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +fi +echo "$as_me:$LINENO: result: $ac_cv_prog_cxx_g" >&5 +echo "${ECHO_T}$ac_cv_prog_cxx_g" >&6 +if test "$ac_test_CXXFLAGS" = set; then + CXXFLAGS=$ac_save_CXXFLAGS +elif test $ac_cv_prog_cxx_g = yes; then + if test "$GXX" = yes; then + CXXFLAGS="-g -O2" + else + CXXFLAGS="-g" + fi +else + if test "$GXX" = yes; then + CXXFLAGS="-O2" + else + CXXFLAGS= + fi +fi +for ac_declaration in \ + '' \ + 'extern "C" void std::exit (int) throw (); using std::exit;' \ + 'extern "C" void std::exit (int); using std::exit;' \ + 'extern "C" void exit (int) throw ();' \ + 'extern "C" void exit (int);' \ + 'void exit (int);' +do + cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_declaration +#include <stdlib.h> +int +main () +{ +exit (42); + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_cxx_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + : +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +continue +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext + cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_declaration +int +main () +{ +exit (42); + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_cxx_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + break +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +done +rm -f conftest* +if test -n "$ac_declaration"; then + echo '#ifdef __cplusplus' >>confdefs.h + echo $ac_declaration >>confdefs.h + echo '#endif' >>confdefs.h +fi + +ac_ext=c +ac_cpp='$CPP $CPPFLAGS' +ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' +ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' +ac_compiler_gnu=$ac_cv_c_compiler_gnu + + +if test -n "$ac_cv_prog_ac_ct_CXX"; then + MAYBE_CPP_TARGETS='$(CPP_TARGETS)' +fi +echo "Maybe-cpp-targets: '$ac_cv_prog_CXX' '$MAYBE_CPP_TARGETS'" #!! + + + ac_aux_dir= for ac_dir in $srcdir $srcdir/.. $srcdir/../..; do if test -f $ac_dir/install-sh; then @@ -3175,7 +3534,7 @@ ia64-*-hpux*) ;; *-*-irix6*) # Find out which ABI we are using. - echo '#line 3173 "configure"' > conftest.$ac_ext + echo '#line 3537 "configure"' > conftest.$ac_ext if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 (eval $ac_compile) 2>&5 ac_status=$? @@ -4207,361 +4566,6 @@ ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu -if test -n "$ac_tool_prefix"; then - for ac_prog in $CCC g++ c++ gpp aCC CC cxx cc++ cl FCC KCC RCC xlC_r xlC - do - # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. -set dummy $ac_tool_prefix$ac_prog; ac_word=$2 -echo "$as_me:$LINENO: checking for $ac_word" >&5 -echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6 -if test "${ac_cv_prog_CXX+set}" = set; then - echo $ECHO_N "(cached) $ECHO_C" >&6 -else - if test -n "$CXX"; then - ac_cv_prog_CXX="$CXX" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_CXX="$ac_tool_prefix$ac_prog" - echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done -done - -fi -fi -CXX=$ac_cv_prog_CXX -if test -n "$CXX"; then - echo "$as_me:$LINENO: result: $CXX" >&5 -echo "${ECHO_T}$CXX" >&6 -else - echo "$as_me:$LINENO: result: no" >&5 -echo "${ECHO_T}no" >&6 -fi - - test -n "$CXX" && break - done -fi -if test -z "$CXX"; then - ac_ct_CXX=$CXX - for ac_prog in $CCC g++ c++ gpp aCC CC cxx cc++ cl FCC KCC RCC xlC_r xlC -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -echo "$as_me:$LINENO: checking for $ac_word" >&5 -echo $ECHO_N "checking for $ac_word... $ECHO_C" >&6 -if test "${ac_cv_prog_ac_ct_CXX+set}" = set; then - echo $ECHO_N "(cached) $ECHO_C" >&6 -else - if test -n "$ac_ct_CXX"; then - ac_cv_prog_ac_ct_CXX="$ac_ct_CXX" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if $as_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_CXX="$ac_prog" - echo "$as_me:$LINENO: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done -done - -fi -fi -ac_ct_CXX=$ac_cv_prog_ac_ct_CXX -if test -n "$ac_ct_CXX"; then - echo "$as_me:$LINENO: result: $ac_ct_CXX" >&5 -echo "${ECHO_T}$ac_ct_CXX" >&6 -else - echo "$as_me:$LINENO: result: no" >&5 -echo "${ECHO_T}no" >&6 -fi - - test -n "$ac_ct_CXX" && break -done -test -n "$ac_ct_CXX" || ac_ct_CXX="g++" - - CXX=$ac_ct_CXX -fi - - -# Provide some information about the compiler. -echo "$as_me:$LINENO:" \ - "checking for C++ compiler version" >&5 -ac_compiler=`set X $ac_compile; echo $2` -{ (eval echo "$as_me:$LINENO: \"$ac_compiler --version </dev/null >&5\"") >&5 - (eval $ac_compiler --version </dev/null >&5) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } -{ (eval echo "$as_me:$LINENO: \"$ac_compiler -v </dev/null >&5\"") >&5 - (eval $ac_compiler -v </dev/null >&5) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } -{ (eval echo "$as_me:$LINENO: \"$ac_compiler -V </dev/null >&5\"") >&5 - (eval $ac_compiler -V </dev/null >&5) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } - -echo "$as_me:$LINENO: checking whether we are using the GNU C++ compiler" >&5 -echo $ECHO_N "checking whether we are using the GNU C++ compiler... $ECHO_C" >&6 -if test "${ac_cv_cxx_compiler_gnu+set}" = set; then - echo $ECHO_N "(cached) $ECHO_C" >&6 -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - -int -main () -{ -#ifndef __GNUC__ - choke me -#endif - - ; - return 0; -} -_ACEOF -rm -f conftest.$ac_objext -if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 - (eval $ac_compile) 2>conftest.er1 - ac_status=$? - grep -v '^ *+' conftest.er1 >conftest.err - rm -f conftest.er1 - cat conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && - { ac_try='test -z "$ac_cxx_werror_flag" - || test ! -s conftest.err' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; } && - { ac_try='test -s conftest.$ac_objext' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_compiler_gnu=yes -else - echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -ac_compiler_gnu=no -fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext -ac_cv_cxx_compiler_gnu=$ac_compiler_gnu - -fi -echo "$as_me:$LINENO: result: $ac_cv_cxx_compiler_gnu" >&5 -echo "${ECHO_T}$ac_cv_cxx_compiler_gnu" >&6 -GXX=`test $ac_compiler_gnu = yes && echo yes` -ac_test_CXXFLAGS=${CXXFLAGS+set} -ac_save_CXXFLAGS=$CXXFLAGS -CXXFLAGS="-g" -echo "$as_me:$LINENO: checking whether $CXX accepts -g" >&5 -echo $ECHO_N "checking whether $CXX accepts -g... $ECHO_C" >&6 -if test "${ac_cv_prog_cxx_g+set}" = set; then - echo $ECHO_N "(cached) $ECHO_C" >&6 -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -rm -f conftest.$ac_objext -if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 - (eval $ac_compile) 2>conftest.er1 - ac_status=$? - grep -v '^ *+' conftest.er1 >conftest.err - rm -f conftest.er1 - cat conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && - { ac_try='test -z "$ac_cxx_werror_flag" - || test ! -s conftest.err' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; } && - { ac_try='test -s conftest.$ac_objext' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_prog_cxx_g=yes -else - echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -ac_cv_prog_cxx_g=no -fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext -fi -echo "$as_me:$LINENO: result: $ac_cv_prog_cxx_g" >&5 -echo "${ECHO_T}$ac_cv_prog_cxx_g" >&6 -if test "$ac_test_CXXFLAGS" = set; then - CXXFLAGS=$ac_save_CXXFLAGS -elif test $ac_cv_prog_cxx_g = yes; then - if test "$GXX" = yes; then - CXXFLAGS="-g -O2" - else - CXXFLAGS="-g" - fi -else - if test "$GXX" = yes; then - CXXFLAGS="-O2" - else - CXXFLAGS= - fi -fi -for ac_declaration in \ - '' \ - 'extern "C" void std::exit (int) throw (); using std::exit;' \ - 'extern "C" void std::exit (int); using std::exit;' \ - 'extern "C" void exit (int) throw ();' \ - 'extern "C" void exit (int);' \ - 'void exit (int);' -do - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ -$ac_declaration -#include <stdlib.h> -int -main () -{ -exit (42); - ; - return 0; -} -_ACEOF -rm -f conftest.$ac_objext -if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 - (eval $ac_compile) 2>conftest.er1 - ac_status=$? - grep -v '^ *+' conftest.er1 >conftest.err - rm -f conftest.er1 - cat conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && - { ac_try='test -z "$ac_cxx_werror_flag" - || test ! -s conftest.err' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; } && - { ac_try='test -s conftest.$ac_objext' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - : -else - echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -continue -fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ -$ac_declaration -int -main () -{ -exit (42); - ; - return 0; -} -_ACEOF -rm -f conftest.$ac_objext -if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 - (eval $ac_compile) 2>conftest.er1 - ac_status=$? - grep -v '^ *+' conftest.er1 >conftest.err - rm -f conftest.er1 - cat conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && - { ac_try='test -z "$ac_cxx_werror_flag" - || test ! -s conftest.err' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; } && - { ac_try='test -s conftest.$ac_objext' - { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 - (eval $ac_try) 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - break -else - echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext -done -rm -f conftest* -if test -n "$ac_declaration"; then - echo '#ifdef __cplusplus' >>confdefs.h - echo $ac_declaration >>confdefs.h - echo '#endif' >>confdefs.h -fi - -ac_ext=cc -ac_cpp='$CXXCPP $CPPFLAGS' -ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_cxx_compiler_gnu - -ac_ext=cc -ac_cpp='$CXXCPP $CPPFLAGS' -ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_cxx_compiler_gnu echo "$as_me:$LINENO: checking how to run the C++ preprocessor" >&5 echo $ECHO_N "checking how to run the C++ preprocessor... $ECHO_C" >&6 if test -z "$CXXCPP"; then @@ -4880,7 +4884,7 @@ fi # Provide some information about the compiler. -echo "$as_me:4878:" \ +echo "$as_me:4887:" \ "checking for Fortran 77 compiler version" >&5 ac_compiler=`set X $ac_compile; echo $2` { (eval echo "$as_me:$LINENO: \"$ac_compiler --version </dev/null >&5\"") >&5 @@ -5934,11 +5938,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:5932: $lt_compile\"" >&5) + (eval echo "\"\$as_me:5941: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:5936: \$? = $ac_status" >&5 + echo "$as_me:5945: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -6177,11 +6181,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:6175: $lt_compile\"" >&5) + (eval echo "\"\$as_me:6184: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:6179: \$? = $ac_status" >&5 + echo "$as_me:6188: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -6237,11 +6241,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:6235: $lt_compile\"" >&5) + (eval echo "\"\$as_me:6244: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:6239: \$? = $ac_status" >&5 + echo "$as_me:6248: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -8422,7 +8426,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<EOF -#line 8420 "configure" +#line 8429 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -8520,7 +8524,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<EOF -#line 8518 "configure" +#line 8527 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10713,11 +10717,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:10711: $lt_compile\"" >&5) + (eval echo "\"\$as_me:10720: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:10715: \$? = $ac_status" >&5 + echo "$as_me:10724: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -10773,11 +10777,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:10771: $lt_compile\"" >&5) + (eval echo "\"\$as_me:10780: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:10775: \$? = $ac_status" >&5 + echo "$as_me:10784: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -12134,7 +12138,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<EOF -#line 12132 "configure" +#line 12141 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -12232,7 +12236,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<EOF -#line 12230 "configure" +#line 12239 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13069,11 +13073,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:13067: $lt_compile\"" >&5) + (eval echo "\"\$as_me:13076: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:13071: \$? = $ac_status" >&5 + echo "$as_me:13080: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -13129,11 +13133,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:13127: $lt_compile\"" >&5) + (eval echo "\"\$as_me:13136: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:13131: \$? = $ac_status" >&5 + echo "$as_me:13140: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -15168,11 +15172,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:15166: $lt_compile\"" >&5) + (eval echo "\"\$as_me:15175: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:15170: \$? = $ac_status" >&5 + echo "$as_me:15179: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -15411,11 +15415,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:15409: $lt_compile\"" >&5) + (eval echo "\"\$as_me:15418: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 - echo "$as_me:15413: \$? = $ac_status" >&5 + echo "$as_me:15422: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings @@ -15471,11 +15475,11 @@ else -e 's:.*FLAGS}? :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:15469: $lt_compile\"" >&5) + (eval echo "\"\$as_me:15478: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 - echo "$as_me:15473: \$? = $ac_status" >&5 + echo "$as_me:15482: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized @@ -17656,7 +17660,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<EOF -#line 17654 "configure" +#line 17663 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -17754,7 +17758,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<EOF -#line 17752 "configure" +#line 17761 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -18771,7 +18775,10 @@ LIBTOOL='$(SHELL) $(top_builddir)/libtool' CC_FOR_BUILD=${CC_FOR_BUILD:-'$(CC)'} +CXX_FOR_BUILD=${CXX_FOR_BUILD:-'$(CXX)'} CFLAGS_FOR_BUILD=${CFLAGS_FOR_BUILD:-'$(CFLAGS)'} +CPPFLAGS_FOR_BUILD=${CFLAGS_FOR_BUILD:-'$(CPPFLAGS)'} +CXXFLAGS_FOR_BUILD=${CXXFLAGS_FOR_BUILD:-'$(CXXFLAGS)'} BUILD_EXEEXT=${BUILD_EXEEXT:-'$(EXEEXT)'} BUILD_OBJEXT=${BUILD_OBJEXT:-'$(OBJEXT)'} @@ -19094,6 +19101,329 @@ done + + +ac_ext=cc +ac_cpp='$CXXCPP $CPPFLAGS' +ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' +ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' +ac_compiler_gnu=$ac_cv_cxx_compiler_gnu + + + +for ac_header in bits/type_traits.h +do +as_ac_Header=`echo "ac_cv_header_$ac_header" | $as_tr_sh` +if eval "test \"\${$as_ac_Header+set}\" = set"; then + echo "$as_me:$LINENO: checking for $ac_header" >&5 +echo $ECHO_N "checking for $ac_header... $ECHO_C" >&6 +if eval "test \"\${$as_ac_Header+set}\" = set"; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +fi +echo "$as_me:$LINENO: result: `eval echo '${'$as_ac_Header'}'`" >&5 +echo "${ECHO_T}`eval echo '${'$as_ac_Header'}'`" >&6 +else + # Is the header compilable? +echo "$as_me:$LINENO: checking $ac_header usability" >&5 +echo $ECHO_N "checking $ac_header usability... $ECHO_C" >&6 +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_includes_default +#include <$ac_header> +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_cxx_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_header_compiler=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +ac_header_compiler=no +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 +echo "${ECHO_T}$ac_header_compiler" >&6 + +# Is the header present? +echo "$as_me:$LINENO: checking $ac_header presence" >&5 +echo $ECHO_N "checking $ac_header presence... $ECHO_C" >&6 +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include <$ac_header> +_ACEOF +if { (eval echo "$as_me:$LINENO: \"$ac_cpp conftest.$ac_ext\"") >&5 + (eval $ac_cpp conftest.$ac_ext) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } >/dev/null; then + if test -s conftest.err; then + ac_cpp_err=$ac_cxx_preproc_warn_flag + ac_cpp_err=$ac_cpp_err$ac_cxx_werror_flag + else + ac_cpp_err= + fi +else + ac_cpp_err=yes +fi +if test -z "$ac_cpp_err"; then + ac_header_preproc=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + + ac_header_preproc=no +fi +rm -f conftest.err conftest.$ac_ext +echo "$as_me:$LINENO: result: $ac_header_preproc" >&5 +echo "${ECHO_T}$ac_header_preproc" >&6 + +# So? What about this header? +case $ac_header_compiler:$ac_header_preproc:$ac_cxx_preproc_warn_flag in + yes:no: ) + { echo "$as_me:$LINENO: WARNING: $ac_header: accepted by the compiler, rejected by the preprocessor!" >&5 +echo "$as_me: WARNING: $ac_header: accepted by the compiler, rejected by the preprocessor!" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: proceeding with the compiler's result" >&5 +echo "$as_me: WARNING: $ac_header: proceeding with the compiler's result" >&2;} + ac_header_preproc=yes + ;; + no:yes:* ) + { echo "$as_me:$LINENO: WARNING: $ac_header: present but cannot be compiled" >&5 +echo "$as_me: WARNING: $ac_header: present but cannot be compiled" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: check for missing prerequisite headers?" >&5 +echo "$as_me: WARNING: $ac_header: check for missing prerequisite headers?" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: see the Autoconf documentation" >&5 +echo "$as_me: WARNING: $ac_header: see the Autoconf documentation" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: section \"Present But Cannot Be Compiled\"" >&5 +echo "$as_me: WARNING: $ac_header: section \"Present But Cannot Be Compiled\"" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: proceeding with the preprocessor's result" >&5 +echo "$as_me: WARNING: $ac_header: proceeding with the preprocessor's result" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: in the future, the compiler will take precedence" >&5 +echo "$as_me: WARNING: $ac_header: in the future, the compiler will take precedence" >&2;} + ( + cat <<\_ASBOX +## ------------------------------------------ ## +## Report this to the AC_PACKAGE_NAME lists. ## +## ------------------------------------------ ## +_ASBOX + ) | + sed "s/^/$as_me: WARNING: /" >&2 + ;; +esac +echo "$as_me:$LINENO: checking for $ac_header" >&5 +echo $ECHO_N "checking for $ac_header... $ECHO_C" >&6 +if eval "test \"\${$as_ac_Header+set}\" = set"; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + eval "$as_ac_Header=\$ac_header_preproc" +fi +echo "$as_me:$LINENO: result: `eval echo '${'$as_ac_Header'}'`" >&5 +echo "${ECHO_T}`eval echo '${'$as_ac_Header'}'`" >&6 + +fi +if test `eval echo '${'$as_ac_Header'}'` = yes; then + cat >>confdefs.h <<_ACEOF +#define `echo "HAVE_$ac_header" | $as_tr_cpp` 1 +_ACEOF + pcre_has_bits_type_traits="1" +else + pcre_has_bits_type_traits="0" +fi + +done + + +for ac_header in type_traits.h +do +as_ac_Header=`echo "ac_cv_header_$ac_header" | $as_tr_sh` +if eval "test \"\${$as_ac_Header+set}\" = set"; then + echo "$as_me:$LINENO: checking for $ac_header" >&5 +echo $ECHO_N "checking for $ac_header... $ECHO_C" >&6 +if eval "test \"\${$as_ac_Header+set}\" = set"; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +fi +echo "$as_me:$LINENO: result: `eval echo '${'$as_ac_Header'}'`" >&5 +echo "${ECHO_T}`eval echo '${'$as_ac_Header'}'`" >&6 +else + # Is the header compilable? +echo "$as_me:$LINENO: checking $ac_header usability" >&5 +echo $ECHO_N "checking $ac_header usability... $ECHO_C" >&6 +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_includes_default +#include <$ac_header> +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_cxx_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_header_compiler=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +ac_header_compiler=no +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 +echo "${ECHO_T}$ac_header_compiler" >&6 + +# Is the header present? +echo "$as_me:$LINENO: checking $ac_header presence" >&5 +echo $ECHO_N "checking $ac_header presence... $ECHO_C" >&6 +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include <$ac_header> +_ACEOF +if { (eval echo "$as_me:$LINENO: \"$ac_cpp conftest.$ac_ext\"") >&5 + (eval $ac_cpp conftest.$ac_ext) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } >/dev/null; then + if test -s conftest.err; then + ac_cpp_err=$ac_cxx_preproc_warn_flag + ac_cpp_err=$ac_cpp_err$ac_cxx_werror_flag + else + ac_cpp_err= + fi +else + ac_cpp_err=yes +fi +if test -z "$ac_cpp_err"; then + ac_header_preproc=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + + ac_header_preproc=no +fi +rm -f conftest.err conftest.$ac_ext +echo "$as_me:$LINENO: result: $ac_header_preproc" >&5 +echo "${ECHO_T}$ac_header_preproc" >&6 + +# So? What about this header? +case $ac_header_compiler:$ac_header_preproc:$ac_cxx_preproc_warn_flag in + yes:no: ) + { echo "$as_me:$LINENO: WARNING: $ac_header: accepted by the compiler, rejected by the preprocessor!" >&5 +echo "$as_me: WARNING: $ac_header: accepted by the compiler, rejected by the preprocessor!" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: proceeding with the compiler's result" >&5 +echo "$as_me: WARNING: $ac_header: proceeding with the compiler's result" >&2;} + ac_header_preproc=yes + ;; + no:yes:* ) + { echo "$as_me:$LINENO: WARNING: $ac_header: present but cannot be compiled" >&5 +echo "$as_me: WARNING: $ac_header: present but cannot be compiled" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: check for missing prerequisite headers?" >&5 +echo "$as_me: WARNING: $ac_header: check for missing prerequisite headers?" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: see the Autoconf documentation" >&5 +echo "$as_me: WARNING: $ac_header: see the Autoconf documentation" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: section \"Present But Cannot Be Compiled\"" >&5 +echo "$as_me: WARNING: $ac_header: section \"Present But Cannot Be Compiled\"" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: proceeding with the preprocessor's result" >&5 +echo "$as_me: WARNING: $ac_header: proceeding with the preprocessor's result" >&2;} + { echo "$as_me:$LINENO: WARNING: $ac_header: in the future, the compiler will take precedence" >&5 +echo "$as_me: WARNING: $ac_header: in the future, the compiler will take precedence" >&2;} + ( + cat <<\_ASBOX +## ------------------------------------------ ## +## Report this to the AC_PACKAGE_NAME lists. ## +## ------------------------------------------ ## +_ASBOX + ) | + sed "s/^/$as_me: WARNING: /" >&2 + ;; +esac +echo "$as_me:$LINENO: checking for $ac_header" >&5 +echo $ECHO_N "checking for $ac_header... $ECHO_C" >&6 +if eval "test \"\${$as_ac_Header+set}\" = set"; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + eval "$as_ac_Header=\$ac_header_preproc" +fi +echo "$as_me:$LINENO: result: `eval echo '${'$as_ac_Header'}'`" >&5 +echo "${ECHO_T}`eval echo '${'$as_ac_Header'}'`" >&6 + +fi +if test `eval echo '${'$as_ac_Header'}'` = yes; then + cat >>confdefs.h <<_ACEOF +#define `echo "HAVE_$ac_header" | $as_tr_cpp` 1 +_ACEOF + pcre_has_type_traits="1" +else + pcre_has_type_traits="0" +fi + +done + + + +ac_ext=c +ac_cpp='$CPP $CPPFLAGS' +ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' +ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' +ac_compiler_gnu=$ac_cv_c_compiler_gnu + + + echo "$as_me:$LINENO: checking for an ANSI C-conforming const" >&5 echo $ECHO_N "checking for an ANSI C-conforming const... $ECHO_C" >&6 if test "${ac_cv_c_const+set}" = set; then @@ -19268,11 +19598,150 @@ _ACEOF fi +echo "$as_me:$LINENO: checking for long long" >&5 +echo $ECHO_N "checking for long long... $ECHO_C" >&6 +if test "${ac_cv_type_long_long+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_includes_default +int +main () +{ +if ((long long *) 0) + return 0; +if (sizeof (long long)) + return 0; + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_cv_type_long_long=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 +ac_cv_type_long_long=no +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +fi +echo "$as_me:$LINENO: result: $ac_cv_type_long_long" >&5 +echo "${ECHO_T}$ac_cv_type_long_long" >&6 +if test $ac_cv_type_long_long = yes; then +cat >>confdefs.h <<_ACEOF +#define HAVE_LONG_LONG 1 +_ACEOF +pcre_has_long_long="1" +else + pcre_has_long_long="0" +fi -for ac_func in bcopy memmove strerror +echo "$as_me:$LINENO: checking for unsigned long long" >&5 +echo $ECHO_N "checking for unsigned long long... $ECHO_C" >&6 +if test "${ac_cv_type_unsigned_long_long+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_includes_default +int +main () +{ +if ((unsigned long long *) 0) + return 0; +if (sizeof (unsigned long long)) + return 0; + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_cv_type_unsigned_long_long=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +ac_cv_type_unsigned_long_long=no +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +fi +echo "$as_me:$LINENO: result: $ac_cv_type_unsigned_long_long" >&5 +echo "${ECHO_T}$ac_cv_type_unsigned_long_long" >&6 +if test $ac_cv_type_unsigned_long_long = yes; then + +cat >>confdefs.h <<_ACEOF +#define HAVE_UNSIGNED_LONG_LONG 1 +_ACEOF + +pcre_has_ulong_long="1" +else + pcre_has_ulong_long="0" +fi + + + + + + + + + + +for ac_func in bcopy memmove strerror strtoq strtoll do as_ac_var=`echo "ac_cv_func_$ac_func" | $as_tr_sh` echo "$as_me:$LINENO: checking for $ac_func" >&5 @@ -19463,9 +19932,6 @@ if test "${with_match_limit+set}" = set; then fi; - - - if test "$UCP" != "" ; then UTF8=-DSUPPORT_UTF8 fi @@ -19493,6 +19959,9 @@ fi + + + case $host_os in mingw* ) POSIX_OBJ=pcreposix.o @@ -19526,7 +19995,7 @@ _ACEOF fi - ac_config_files="$ac_config_files Makefile pcre.h:pcre.in pcre-config:pcre-config.in libpcre.pc:libpcre.pc.in RunTest:RunTest.in" + ac_config_files="$ac_config_files Makefile pcre.h:pcre.in pcre-config:pcre-config.in libpcre.pc:libpcre.pc.in pcrecpp.h:pcrecpp.h.in pcre_stringpiece.h:pcre_stringpiece.h.in RunGrepTest:RunGrepTest.in RunTest:RunTest.in" ac_config_commands="$ac_config_commands default" cat >confcache <<\_ACEOF # This file is a shell script that caches the results of configure @@ -20060,6 +20529,9 @@ do "pcre.h" ) CONFIG_FILES="$CONFIG_FILES pcre.h:pcre.in" ;; "pcre-config" ) CONFIG_FILES="$CONFIG_FILES pcre-config:pcre-config.in" ;; "libpcre.pc" ) CONFIG_FILES="$CONFIG_FILES libpcre.pc:libpcre.pc.in" ;; + "pcrecpp.h" ) CONFIG_FILES="$CONFIG_FILES pcrecpp.h:pcrecpp.h.in" ;; + "pcre_stringpiece.h" ) CONFIG_FILES="$CONFIG_FILES pcre_stringpiece.h:pcre_stringpiece.h.in" ;; + "RunGrepTest" ) CONFIG_FILES="$CONFIG_FILES RunGrepTest:RunGrepTest.in" ;; "RunTest" ) CONFIG_FILES="$CONFIG_FILES RunTest:RunTest.in" ;; "default" ) CONFIG_COMMANDS="$CONFIG_COMMANDS default" ;; "config.h" ) CONFIG_HEADERS="$CONFIG_HEADERS config.h:config.in" ;; @@ -20155,6 +20627,10 @@ s,@CPPFLAGS@,$CPPFLAGS,;t t s,@ac_ct_CC@,$ac_ct_CC,;t t s,@EXEEXT@,$EXEEXT,;t t s,@OBJEXT@,$OBJEXT,;t t +s,@CXX@,$CXX,;t t +s,@CXXFLAGS@,$CXXFLAGS,;t t +s,@ac_ct_CXX@,$ac_ct_CXX,;t t +s,@MAYBE_CPP_TARGETS@,$MAYBE_CPP_TARGETS,;t t s,@INSTALL_PROGRAM@,$INSTALL_PROGRAM,;t t s,@INSTALL_SCRIPT@,$INSTALL_SCRIPT,;t t s,@INSTALL_DATA@,$INSTALL_DATA,;t t @@ -20182,18 +20658,21 @@ s,@ac_ct_AS@,$ac_ct_AS,;t t s,@OBJDUMP@,$OBJDUMP,;t t s,@ac_ct_OBJDUMP@,$ac_ct_OBJDUMP,;t t s,@CPP@,$CPP,;t t -s,@CXX@,$CXX,;t t -s,@CXXFLAGS@,$CXXFLAGS,;t t -s,@ac_ct_CXX@,$ac_ct_CXX,;t t s,@CXXCPP@,$CXXCPP,;t t s,@F77@,$F77,;t t s,@FFLAGS@,$FFLAGS,;t t s,@ac_ct_F77@,$ac_ct_F77,;t t s,@LIBTOOL@,$LIBTOOL,;t t +s,@pcre_has_bits_type_traits@,$pcre_has_bits_type_traits,;t t +s,@pcre_has_type_traits@,$pcre_has_type_traits,;t t +s,@pcre_has_long_long@,$pcre_has_long_long,;t t +s,@pcre_has_ulong_long@,$pcre_has_ulong_long,;t t s,@BUILD_EXEEXT@,$BUILD_EXEEXT,;t t s,@BUILD_OBJEXT@,$BUILD_OBJEXT,;t t s,@CC_FOR_BUILD@,$CC_FOR_BUILD,;t t +s,@CXX_FOR_BUILD@,$CXX_FOR_BUILD,;t t s,@CFLAGS_FOR_BUILD@,$CFLAGS_FOR_BUILD,;t t +s,@CXXFLAGS_FOR_BUILD@,$CXXFLAGS_FOR_BUILD,;t t s,@EBCDIC@,$EBCDIC,;t t s,@HAVE_MEMMOVE@,$HAVE_MEMMOVE,;t t s,@HAVE_STRERROR@,$HAVE_STRERROR,;t t @@ -20207,6 +20686,7 @@ s,@PCRE_DATE@,$PCRE_DATE,;t t s,@PCRE_VERSION@,$PCRE_VERSION,;t t s,@PCRE_LIB_VERSION@,$PCRE_LIB_VERSION,;t t s,@PCRE_POSIXLIB_VERSION@,$PCRE_POSIXLIB_VERSION,;t t +s,@PCRE_CPPLIB_VERSION@,$PCRE_CPPLIB_VERSION,;t t s,@POSIX_MALLOC_THRESHOLD@,$POSIX_MALLOC_THRESHOLD,;t t s,@UCP@,$UCP,;t t s,@UTF8@,$UTF8,;t t @@ -20793,7 +21273,7 @@ esac { echo "$as_me:$LINENO: executing $ac_dest commands" >&5 echo "$as_me: executing $ac_dest commands" >&6;} case $ac_dest in - default ) chmod a+x RunTest pcre-config ;; + default ) chmod a+x RunTest RunGrepTest pcre-config ;; esac done _ACEOF diff --git a/configure.in b/configure.in index f2e4efd..0bbee9a 100644 --- a/configure.in +++ b/configure.in @@ -20,9 +20,9 @@ dnl macro, and may be treated as octal constants. Stick to single dnl digits for minor numbers less than 10. There are unlikely to be dnl that many releases anyway. -PCRE_MAJOR=5 +PCRE_MAJOR=6 PCRE_MINOR=0 -PCRE_DATE=13-Sep-2004 +PCRE_DATE=07-Jun-2005 PCRE_VERSION=${PCRE_MAJOR}.${PCRE_MINOR} dnl Default values for miscellaneous macros @@ -34,11 +34,22 @@ dnl are built by default on Unix systems. PCRE_LIB_VERSION=0:1:0 PCRE_POSIXLIB_VERSION=0:0:0 +PCRE_CPPLIB_VERSION=0:0:0 dnl Checks for programs. AC_PROG_CC +dnl For the C++ wrapper libpcrecpp. + +AC_PROG_CXX + +if test -n "$ac_cv_prog_ac_ct_CXX"; then + MAYBE_CPP_TARGETS='$(CPP_TARGETS)' +fi +echo "Maybe-cpp-targets: '$ac_cv_prog_CXX' '$MAYBE_CPP_TARGETS'" #!! +AC_SUBST(MAYBE_CPP_TARGETS) + dnl The icc compiler has the same options as gcc, so let the rest of the dnl configure script think it has gcc when setting up dnl options etc. dnl This is a nasty hack which no longer seems necessary with the update @@ -60,7 +71,10 @@ dnl just have to adjust the Makefile by hand or set these values when they dnl run "configure". CC_FOR_BUILD=${CC_FOR_BUILD:-'$(CC)'} +CXX_FOR_BUILD=${CXX_FOR_BUILD:-'$(CXX)'} CFLAGS_FOR_BUILD=${CFLAGS_FOR_BUILD:-'$(CFLAGS)'} +CPPFLAGS_FOR_BUILD=${CFLAGS_FOR_BUILD:-'$(CPPFLAGS)'} +CXXFLAGS_FOR_BUILD=${CXXFLAGS_FOR_BUILD:-'$(CXXFLAGS)'} BUILD_EXEEXT=${BUILD_EXEEXT:-'$(EXEEXT)'} BUILD_OBJEXT=${BUILD_OBJEXT:-'$(OBJEXT)'} @@ -69,14 +83,35 @@ dnl Checks for header files. AC_HEADER_STDC AC_CHECK_HEADERS(limits.h) +dnl These are C++ header files + +AC_LANG_SAVE +AC_LANG_CPLUSPLUS +dnl I could be more clever here, given I'm doing AC_SUBST with this +dnl (eg set a var to be the name of the include file I want). But I'm not +dnl so it's easy to change back to 'regular' autoconf vars if we needed to. +AC_CHECK_HEADERS(bits/type_traits.h, [pcre_has_bits_type_traits="1"], + [pcre_has_bits_type_traits="0"]) +AC_CHECK_HEADERS(type_traits.h, [pcre_has_type_traits="1"], + [pcre_has_type_traits="0"]) +dnl Using AC_SUBST eliminates the need to include config.h in a public .h file +AC_SUBST(pcre_has_bits_type_traits) +AC_SUBST(pcre_has_type_traits) +AC_LANG_RESTORE + dnl Checks for typedefs, structures, and compiler characteristics. AC_C_CONST AC_TYPE_SIZE_T +AC_CHECK_TYPES([long long], [pcre_has_long_long="1"], [pcre_has_long_long="0"]) +AC_CHECK_TYPES([unsigned long long], [pcre_has_ulong_long="1"], [pcre_has_ulong_long="0"]) +AC_SUBST(pcre_has_long_long) +AC_SUBST(pcre_has_ulong_long) + dnl Checks for library functions. -AC_CHECK_FUNCS(bcopy memmove strerror) +AC_CHECK_FUNCS(bcopy memmove strerror strtoq strtoll) dnl Handle --enable-utf8 @@ -157,10 +192,6 @@ AC_ARG_WITH(match-limit, MATCH_LIMIT=-DMATCH_LIMIT=$withval ) -dnl Now arrange to build libtool - -AC_PROG_LIBTOOL - dnl Unicode character property support implies UTF-8 support if test "$UCP" != "" ; then @@ -172,7 +203,9 @@ dnl "Export" these variables AC_SUBST(BUILD_EXEEXT) AC_SUBST(BUILD_OBJEXT) AC_SUBST(CC_FOR_BUILD) +AC_SUBST(CXX_FOR_BUILD) AC_SUBST(CFLAGS_FOR_BUILD) +AC_SUBST(CXXFLAGS_FOR_BUILD) AC_SUBST(EBCDIC) AC_SUBST(HAVE_MEMMOVE) AC_SUBST(HAVE_STRERROR) @@ -186,6 +219,7 @@ AC_SUBST(PCRE_DATE) AC_SUBST(PCRE_VERSION) AC_SUBST(PCRE_LIB_VERSION) AC_SUBST(PCRE_POSIXLIB_VERSION) +AC_SUBST(PCRE_CPPLIB_VERSION) AC_SUBST(POSIX_MALLOC_THRESHOLD) AC_SUBST(UCP) AC_SUBST(UTF8) @@ -223,4 +257,4 @@ if test "x$enable_shared" = "xno" ; then fi dnl This must be last; it determines what files are written as well as config.h -AC_OUTPUT(Makefile pcre.h:pcre.in pcre-config:pcre-config.in libpcre.pc:libpcre.pc.in RunTest:RunTest.in,[chmod a+x RunTest pcre-config]) +AC_OUTPUT(Makefile pcre.h:pcre.in pcre-config:pcre-config.in libpcre.pc:libpcre.pc.in pcrecpp.h:pcrecpp.h.in pcre_stringpiece.h:pcre_stringpiece.h.in RunGrepTest:RunGrepTest.in RunTest:RunTest.in,[chmod a+x RunTest RunGrepTest pcre-config]) @@ -2,13 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -PCRE is a library of functions to support regular expressions whose syntax +/* PCRE is a library of functions to support regular expressions whose syntax and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2004 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -40,20 +38,19 @@ POSSIBILITY OF SUCH DAMAGE. */ -/* This is a support program to generate the file chartables.c, containing -character tables of various kinds. They are built according to the default C -locale and used as the default tables by PCRE. Now that pcre_maketables is -a function visible to the outside world, we make use of its code from here in -order to be consistent. */ +/* This is a freestanding support program to generate a file containing default +character tables for PCRE. The tables are built according to the default C +locale. Now that pcre_maketables is a function visible to the outside world, we +make use of its code from here in order to be consistent. */ #include <ctype.h> #include <stdio.h> #include <string.h> -#include "internal.h" +#include "pcre_internal.h" -#define DFTABLES /* maketables.c notices this */ -#include "maketables.c" +#define DFTABLES /* pcre_maketables.c notices this */ +#include "pcre_maketables.c" int main(int argc, char **argv) @@ -61,6 +58,7 @@ int main(int argc, char **argv) int i; FILE *f; const unsigned char *tables = pcre_maketables(); +const unsigned char *base_of_tables = tables; if (argc != 2) { @@ -86,10 +84,10 @@ fprintf(f, "program. If you edit it by hand, you might like to edit the Makefile to \n" "prevent its ever being regenerated.\n\n"); fprintf(f, - "This file is #included in the compilation of pcre.c to build the default\n" - "character tables which are used when no tables are passed to the compile\n" - "function. */\n\n" - "static unsigned char pcre_default_tables[] = {\n\n" + "This file contains the default tables for characters with codes less than\n" + "128 (ASCII characters). These tables are used when no external tables are\n" + "passed to PCRE. */\n\n" + "const unsigned char _pcre_default_tables[] = {\n\n" "/* This table is a lower casing table. */\n\n"); fprintf(f, " "); @@ -167,6 +165,7 @@ if (isprint(i-1)) fprintf(f, " %c ", i-1); fprintf(f, " */\n\n/* End of chartables.c */\n"); fclose(f); +free((void *)base_of_tables); return 0; } diff --git a/doc/Tech.Notes b/doc/Tech.Notes index 18eb72b..322cc2d 100644 --- a/doc/Tech.Notes +++ b/doc/Tech.Notes @@ -32,18 +32,42 @@ terminology. OK, here's the real stuff ------------------------- -For the set of functions that forms PCRE (which are unrelated to those -mentioned above), I tried at first to invent an algorithm that used an amount -of store bounded by a multiple of the number of characters in the pattern, to -save on compiling time. However, because of the greater complexity in Perl -regular expressions, I couldn't do this. In any case, a first pass through the -pattern is needed, for a number of reasons. PCRE works by running a very -degenerate first pass to calculate a maximum store size, and then a second pass -to do the real compile - which may use a bit less than the predicted amount of -store. The idea is that this is going to turn out faster because the first pass -is degenerate and the second pass can just store stuff straight into the -vector. It does make the compiling functions bigger, of course, but they have -got quite big anyway to handle all the Perl stuff. +For the set of functions that form the "basic" PCRE library (which are +unrelated to those mentioned above), I tried at first to invent an algorithm +that used an amount of store bounded by a multiple of the number of characters +in the pattern, to save on compiling time. However, because of the greater +complexity in Perl regular expressions, I couldn't do this. In any case, a +first pass through the pattern is needed, for a number of reasons. PCRE works +by running a very degenerate first pass to calculate a maximum store size, and +then a second pass to do the real compile - which may use a bit less than the +predicted amount of store. The idea is that this is going to turn out faster +because the first pass is degenerate and the second pass can just store stuff +straight into the vector, which it knows is big enough. It does make the +compiling functions bigger, of course, but they have got quite big anyway to +handle all the Perl stuff. + +Traditional matching function +----------------------------- + +The "traditional", and original, matching function is called pcre_exec(), and +it implements an NFA algorithm, similar to the original Henry Spencer algorithm +and the way that Perl works. Not surprising, since it is intended to be as +compatible with Perl as possible. This is the function most users of PCRE will +use most of the time. + +Supplementary matching function +------------------------------- + +From PCRE 6.0, there is also a supplementary matching function called +pcre_dfa_exec(). This implements a DFA matching algorithm that searches +simultaneously for all possible matches that start at one point in the subject +string. (Going back to my roots: see Historical Note 1 above.) This function +intreprets the same compiled pattern data as pcre_exec(); however, not all the +facilities are available, and those that are don't always work in quite the +same way. See the user documentation for details. + +Format of compiled patterns +--------------------------- The compiled form of a pattern is a vector of bytes, containing items of variable length. The first byte in an item is an opcode, and the length of the @@ -223,14 +247,14 @@ number. This opcode is ignored while matching, but is fished out when handling the bracket itself. (They could have all been done like this, but I was making minimal changes.) -A bracket opcode is followed by two bytes which give the offset to the next -alternative OP_ALT or, if there aren't any branches, to the matching OP_KET -opcode. Each OP_ALT is followed by two bytes giving the offset to the next one, -or to the OP_KET opcode. +A bracket opcode is followed by LINK_SIZE bytes which give the offset to the +next alternative OP_ALT or, if there aren't any branches, to the matching +OP_KET opcode. Each OP_ALT is followed by LINK_SIZE bytes giving the offset to +the next one, or to the OP_KET opcode. OP_KET is used for subpatterns that do not repeat indefinitely, while OP_KETRMIN and OP_KETRMAX are used for indefinite repetitions, minimally or -maximally respectively. All three are followed by two bytes giving (as a +maximally respectively. All three are followed by LINK_SIZE bytes giving (as a positive number) the offset back to the matching OP_BRA opcode. If a subpattern is quantified such that it is permitted to match zero times, it @@ -312,4 +336,4 @@ at compile time, and so does not cause anything to be put into the compiled data. Philip Hazel -September 2004 +March 2005 diff --git a/doc/html/index.html b/doc/html/index.html index c0dbf59..32664ff 100644 --- a/doc/html/index.html +++ b/doc/html/index.html @@ -24,9 +24,15 @@ The HTML documentation for PCRE comprises the following pages: <tr><td><a href="pcrecompat.html">pcrecompat</a></td> <td> Compability with Perl</td></tr> +<tr><td><a href="pcrecpp.html">pcrecpp</a></td> + <td> The C++ wrapper for the PCRE library</td></tr> + <tr><td><a href="pcregrep.html">pcregrep</a></td> <td> The <b>pcregrep</b> command</td></tr> +<tr><td><a href="pcrematching.html">pcrematching</a></td> + <td> Discussion of the two matching algorithms</td></tr> + <tr><td><a href="pcrepartial.html">pcrepartial</a></td> <td> Using PCRE for partial matching</td></tr> @@ -59,6 +65,9 @@ in the library: <tr><td><a href="pcre_compile.html">pcre_compile</a></td> <td> Compile a regular expression</td></tr> +<tr><td><a href="pcre_compile2.html">pcre_compile2</a></td> + <td> Compile a regular expression (alternate interface)</td></tr> + <tr><td><a href="pcre_config.html">pcre_config</a></td> <td> Show build-time configuration options</td></tr> @@ -68,8 +77,13 @@ in the library: <tr><td><a href="pcre_copy_substring.html">pcre_copy_substring</a></td> <td> Extract numbered substring into given buffer</td></tr> +<tr><td><a href="pcre_dfa_exec.html">pcre_dfa_exec</a></td> + <td> Match a compiled pattern to a subject string + (DFA algorithm; <i>not</i> Perl compatible)</td></tr> + <tr><td><a href="pcre_exec.html">pcre_exec</a></td> - <td> Match a compiled pattern to a subject string</td></tr> + <td> Match a compiled pattern to a subject string + (Perl compatible)</td></tr> <tr><td><a href="pcre_free_substring.html">pcre_free_substring</a></td> <td> Free extracted substring</td></tr> diff --git a/doc/html/pcre.html b/doc/html/pcre.html index b1caf80..c8db6b0 100644 --- a/doc/html/pcre.html +++ b/doc/html/pcre.html @@ -23,16 +23,27 @@ man page, in case the conversion went wrong. <P> The PCRE library is a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl, with just a few -differences. The current implementation of PCRE (release 5.x) corresponds +differences. The current implementation of PCRE (release 6.x) corresponds approximately with Perl 5.8, including support for UTF-8 encoded strings and Unicode general category properties. However, this support has to be explicitly enabled; it is not the default. </P> <P> +In addition to the Perl-compatible matching function, PCRE also contains an +alternative matching function that matches the same compiled patterns in a +different way. In certain circumstances, the alternative function has some +advantages. For a discussion of the two matching algorithms, see the +<a href="pcrematching.html"><b>pcrematching</b></a> +page. +</P> +<P> PCRE is written in C and released as a C library. A number of people have -written wrappers and interfaces of various kinds. A C++ class is included in -these contributions, which can be found in the <i>Contrib</i> directory at the -primary FTP site, which is: +written wrappers and interfaces of various kinds. In particular, Google Inc. +have provided a comprehensive C++ wrapper. This is now included as part of the +PCRE distribution. The +<a href="pcrecpp.html"><b>pcrecpp</b></a> +page has details of this interface. Other people's contributions can be found +in the <i>Contrib</i> directory at the primary FTP site, which is: <a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre">ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre</a> </P> <P> @@ -53,6 +64,12 @@ available. The features themselves are described in the page. Documentation about building PCRE for various operating systems can be found in the <b>README</b> file in the source distribution. </P> +<P> +The library contains a number of undocumented internal functions and data +tables that are used by more than one of the exported external functions, but +which are not intended for use by external callers. Their names all begin with +"_pcre_", which hopefully will not provoke any name clashes. +</P> <br><a name="SEC2" href="#TOC1">USER DOCUMENTATION</a><br> <P> The user documentation for PCRE comprises a number of different sections. In @@ -62,21 +79,23 @@ all the sections are concatenated, for ease of searching. The sections are as follows: <pre> pcre this document - pcreapi details of PCRE's native API + pcreapi details of PCRE's native C API pcrebuild options for building PCRE pcrecallout details of the callout feature pcrecompat discussion of Perl compatibility + pcrecpp details of the C++ wrapper pcregrep description of the <b>pcregrep</b> command + pcrematching discussion of the two matching algorithms pcrepartial details of the partial matching facility pcrepattern syntax and semantics of supported regular expressions pcreperform discussion of performance issues - pcreposix the POSIX-compatible API + pcreposix the POSIX-compatible C API pcreprecompile details of saving and re-using precompiled patterns pcresample discussion of the sample program pcretest description of the <b>pcretest</b> testing command </pre> In addition, in the "man" and HTML formats, there is a short page for each -library function, listing its arguments and results. +C library function, listing its arguments and results. </P> <br><a name="SEC3" href="#TOC1">LIMITATIONS</a><br> <P> @@ -104,9 +123,10 @@ subpatterns, assertions, and other types of subpattern, is 200. </P> <P> The maximum length of a subject string is the largest positive number that an -integer variable can hold. However, PCRE uses recursion to handle subpatterns -and indefinite repetition. This means that the available stack space may limit -the size of a subject string that can be processed by certain patterns. +integer variable can hold. However, when using the traditional matching +function, PCRE uses recursion to handle subpatterns and indefinite repetition. +This means that the available stack space may limit the size of a subject +string that can be processed by certain patterns. <a name="utf8support"></a></P> <br><a name="SEC4" href="#TOC1">UTF-8 AND UNICODE PROPERTY SUPPORT</a><br> <P> @@ -174,7 +194,8 @@ bytes, for example: \x{100}{3}. </P> <P> 6. The escape sequence \C can be used to match a single byte in UTF-8 mode, -but its use can lead to some strange effects. +but its use can lead to some strange effects. This facility is not available in +the alternative matching function, <b>pcre_dfa_exec()</b>. </P> <P> 7. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly @@ -199,16 +220,19 @@ values. </P> <br><a name="SEC5" href="#TOC1">AUTHOR</a><br> <P> -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel <br> University Computing Service, <br> Cambridge CB2 3QG, England. +</P> +<P> +Putting an actual email address here seems to have been a spam magnet, so I've +taken it away. If you want to email me, use my initial and surname, separated +by a dot, at the domain ucs.cam.ac.uk. +Last updated: 07 March 2005 <br> -Phone: +44 1223 334714 -Last updated: 09 September 2004 -<br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcre_compile.html b/doc/html/pcre_compile.html index 0d417a1..7ed2c32 100644 --- a/doc/html/pcre_compile.html +++ b/doc/html/pcre_compile.html @@ -48,6 +48,7 @@ The option bits are: PCRE_EXTENDED Ignore whitespace and # comments PCRE_EXTRA PCRE extra features (not much use currently) + PCRE_FIRSTLINE Force matching to be before newline PCRE_MULTILINE ^ and $ match newlines within data PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- theses (named ones available) diff --git a/doc/html/pcre_compile2.html b/doc/html/pcre_compile2.html new file mode 100644 index 0000000..fc39929 --- /dev/null +++ b/doc/html/pcre_compile2.html @@ -0,0 +1,81 @@ +<html> +<head> +<title>pcre_compile2 specification</title> +</head> +<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> +<h1>pcre_compile2 man page</h1> +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> +<p> +This page is part of the PCRE HTML documentation. It was generated automatically +from the original man page. If there is any nonsense in it, please consult the +man page, in case the conversion went wrong. +<br> +<br><b> +SYNOPSIS +</b><br> +<P> +<b>#include <pcre.h></b> +</P> +<P> +<b>pcre *pcre_compile2(const char *<i>pattern</i>, int <i>options</i>,</b> +<b>int *<i>errorcodeptr</i>,</b> +<b>const char **<i>errptr</i>, int *<i>erroffset</i>,</b> +<b>const unsigned char *<i>tableptr</i>);</b> +</P> +<br><b> +DESCRIPTION +</b><br> +<P> +This function compiles a regular expression into an internal form. It is the +same as <b>pcre_compile()</b>, except for the addition of the <i>errorcodeptr</i> +argument. The arguments are: +</P> +<P> +<pre> + <i>pattern</i> A zero-terminated string containing the + regular expression to be compiled + <i>options</i> Zero or more option bits + <i>errorcodeptr</i> Where to put an error code + <i>errptr</i> Where to put an error message + <i>erroffset</i> Offset in pattern where error was found + <i>tableptr</i> Pointer to character tables, or NULL to + use the built-in default +</pre> +The option bits are: +<pre> + PCRE_ANCHORED Force pattern anchoring + PCRE_AUTO_CALLOUT Compile automatic callouts + PCRE_CASELESS Do caseless matching + PCRE_DOLLAR_ENDONLY $ not to match newline at end + PCRE_DOTALL . matches anything including NL + PCRE_EXTENDED Ignore whitespace and # comments + PCRE_EXTRA PCRE extra features + (not much use currently) + PCRE_FIRSTLINE Force matching to be before newline + PCRE_MULTILINE ^ and $ match newlines within data + PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- + theses (named ones available) + PCRE_UNGREEDY Invert greediness of quantifiers + PCRE_UTF8 Run in UTF-8 mode + PCRE_NO_UTF8_CHECK Do not check the pattern for UTF-8 + validity (only relevant if + PCRE_UTF8 is set) +</pre> +PCRE must be built with UTF-8 support in order to use PCRE_UTF8 and +PCRE_NO_UTF8_CHECK. +</P> +<P> +The yield of the function is a pointer to a private data structure that +contains the compiled pattern, or NULL if an error was detected. +</P> +<P> +There is a complete description of the PCRE native API in the +<a href="pcreapi.html"><b>pcreapi</b></a> +page and a description of the POSIX API in the +<a href="pcreposix.html"><b>pcreposix</b></a> +page. +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> diff --git a/doc/html/pcre_dfa_exec.html b/doc/html/pcre_dfa_exec.html new file mode 100644 index 0000000..ceadb8b --- /dev/null +++ b/doc/html/pcre_dfa_exec.html @@ -0,0 +1,88 @@ +<html> +<head> +<title>pcre_dfa_exec specification</title> +</head> +<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> +<h1>pcre_dfa_exec man page</h1> +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> +<p> +This page is part of the PCRE HTML documentation. It was generated automatically +from the original man page. If there is any nonsense in it, please consult the +man page, in case the conversion went wrong. +<br> +<br><b> +SYNOPSIS +</b><br> +<P> +<b>#include <pcre.h></b> +</P> +<P> +<b>int pcre_dfa_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> +<b>const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> +<b>int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> +<b>int *<i>workspace</i>, int <i>wscount</i>);</b> +</P> +<br><b> +DESCRIPTION +</b><br> +<P> +This function matches a compiled regular expression against a given subject +string, using a DFA matching algorithm (<i>not</i> Perl-compatible). Note that +the main, Perl-compatible, matching function is <b>pcre_exec()</b>. The +arguments for this function are: +<pre> + <i>code</i> Points to the compiled pattern + <i>extra</i> Points to an associated <b>pcre_extra</b> structure, + or is NULL + <i>subject</i> Points to the subject string + <i>length</i> Length of the subject string, in bytes + <i>startoffset</i> Offset in bytes in the subject at which to + start matching + <i>options</i> Option bits + <i>ovector</i> Points to a vector of ints for result offsets + <i>ovecsize</i> Number of elements in the vector + <i>workspace</i> Points to a vector of ints used as working space + <i>wscount</i> Number of elements in the vector +</pre> +The options are: +<pre> + PCRE_ANCHORED Match only at the first position + PCRE_NOTBOL Subject is not the beginning of a line + PCRE_NOTEOL Subject is not the end of a line + PCRE_NOTEMPTY An empty string is not a valid match + PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 + validity (only relevant if PCRE_UTF8 + was set at compile time) + PCRE_PARTIAL Return PCRE_ERROR_PARTIAL for a partial match + PCRE_DFA_SHORTEST Return only the shortest match + PCRE_DFA_RESTART This is a restart after a partial match +</pre> +There are restrictions on what may appear in a pattern when matching using the +DFA algorithm is requested. Details are given in the +<a href="pcrematching.html"><b>pcrematching</b></a> +documentation. +</P> +<P> +A <b>pcre_extra</b> structure contains the following fields: +<pre> + <i>flags</i> Bits indicating which fields are set + <i>study_data</i> Opaque data from <b>pcre_study()</b> + <i>match_limit</i> Limit on internal recursion + <i>callout_data</i> Opaque data passed back to callouts + <i>tables</i> Points to character tables or is NULL +</pre> +The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, +PCRE_EXTRA_CALLOUT_DATA, and PCRE_EXTRA_TABLES. For DFA matching, the +<i>match_limit</i> field is not used, and must not be set. +</P> +<P> +There is a complete description of the PCRE native API in the +<a href="pcreapi.html"><b>pcreapi</b></a> +page and a description of the POSIX API in the +<a href="pcreposix.html"><b>pcreposix</b></a> +page. +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> diff --git a/doc/html/pcre_exec.html b/doc/html/pcre_exec.html index fc3d322..5fae92f 100644 --- a/doc/html/pcre_exec.html +++ b/doc/html/pcre_exec.html @@ -28,7 +28,8 @@ DESCRIPTION </b><br> <P> This function matches a compiled regular expression against a given subject -string, and returns offsets to capturing subexpressions. Its arguments are: +string, using a matching algorithm that is similar to Perl's. It returns +offsets to captured substrings. Its arguments are: <pre> <i>code</i> Points to the compiled pattern <i>extra</i> Points to an associated <b>pcre_extra</b> structure, diff --git a/doc/html/pcre_refcount.html b/doc/html/pcre_refcount.html new file mode 100644 index 0000000..ba53112 --- /dev/null +++ b/doc/html/pcre_refcount.html @@ -0,0 +1,45 @@ +<html> +<head> +<title>pcre_refcount specification</title> +</head> +<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> +<h1>pcre_refcount man page</h1> +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> +<p> +This page is part of the PCRE HTML documentation. It was generated automatically +from the original man page. If there is any nonsense in it, please consult the +man page, in case the conversion went wrong. +<br> +<br><b> +SYNOPSIS +</b><br> +<P> +<b>#include <pcre.h></b> +</P> +<P> +<b>int pcre_info(pcre *<i>code</i>, int <i>adjust</i>);</b> +</P> +<br><b> +DESCRIPTION +</b><br> +<P> +This function is used to maintain a reference count inside a data block that +contains a compiled pattern. Its arguments are: +<pre> + <i>code</i> Compiled regular expression + <i>adjust</i> Adjustment to reference value +</pre> +The yield of the function is the adjusted reference value, which is constrained +to lie between 0 and 65535. +</P> +<P> +There is a complete description of the PCRE native API in the +<a href="pcreapi.html"><b>pcreapi</b></a> +page and a description of the POSIX API in the +<a href="pcreposix.html"><b>pcreposix</b></a> +page. +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> diff --git a/doc/html/pcreapi.html b/doc/html/pcreapi.html index 72639f4..4d5f866 100644 --- a/doc/html/pcreapi.html +++ b/doc/html/pcreapi.html @@ -19,13 +19,17 @@ man page, in case the conversion went wrong. <li><a name="TOC4" href="#SEC4">SAVING PRECOMPILED PATTERNS FOR LATER USE</a> <li><a name="TOC5" href="#SEC5">CHECKING BUILD-TIME OPTIONS</a> <li><a name="TOC6" href="#SEC6">COMPILING A PATTERN</a> -<li><a name="TOC7" href="#SEC7">STUDYING A PATTERN</a> -<li><a name="TOC8" href="#SEC8">LOCALE SUPPORT</a> -<li><a name="TOC9" href="#SEC9">INFORMATION ABOUT A PATTERN</a> -<li><a name="TOC10" href="#SEC10">OBSOLETE INFO FUNCTION</a> -<li><a name="TOC11" href="#SEC11">MATCHING A PATTERN</a> -<li><a name="TOC12" href="#SEC12">EXTRACTING CAPTURED SUBSTRINGS BY NUMBER</a> -<li><a name="TOC13" href="#SEC13">EXTRACTING CAPTURED SUBSTRINGS BY NAME</a> +<li><a name="TOC7" href="#SEC7">COMPILATION ERROR CODES</a> +<li><a name="TOC8" href="#SEC8">STUDYING A PATTERN</a> +<li><a name="TOC9" href="#SEC9">LOCALE SUPPORT</a> +<li><a name="TOC10" href="#SEC10">INFORMATION ABOUT A PATTERN</a> +<li><a name="TOC11" href="#SEC11">OBSOLETE INFO FUNCTION</a> +<li><a name="TOC12" href="#SEC12">REFERENCE COUNTS</a> +<li><a name="TOC13" href="#SEC13">MATCHING A PATTERN: THE TRADITIONAL FUNCTION</a> +<li><a name="TOC14" href="#SEC14">EXTRACTING CAPTURED SUBSTRINGS BY NUMBER</a> +<li><a name="TOC15" href="#SEC15">EXTRACTING CAPTURED SUBSTRINGS BY NAME</a> +<li><a name="TOC16" href="#SEC16">FINDING ALL POSSIBLE MATCHES</a> +<li><a name="TOC17" href="#SEC17">MATCHING A PATTERN: THE ALTERNATIVE FUNCTION</a> </ul> <br><a name="SEC1" href="#TOC1">PCRE NATIVE API</a><br> <P> @@ -37,6 +41,12 @@ man page, in case the conversion went wrong. <b>const unsigned char *<i>tableptr</i>);</b> </P> <P> +<b>pcre *pcre_compile2(const char *<i>pattern</i>, int <i>options</i>,</b> +<b>int *<i>errorcodeptr</i>,</b> +<b>const char **<i>errptr</i>, int *<i>erroffset</i>,</b> +<b>const unsigned char *<i>tableptr</i>);</b> +</P> +<P> <b>pcre_extra *pcre_study(const pcre *<i>code</i>, int <i>options</i>,</b> <b>const char **<i>errptr</i>);</b> </P> @@ -46,6 +56,12 @@ man page, in case the conversion went wrong. <b>int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>);</b> </P> <P> +<b>int pcre_dfa_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> +<b>const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> +<b>int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> +<b>int *<i>workspace</i>, int <i>wscount</i>);</b> +</P> +<P> <b>int pcre_copy_named_substring(const pcre *<i>code</i>,</b> <b>const char *<i>subject</i>, int *<i>ovector</i>,</b> <b>int <i>stringcount</i>, const char *<i>stringname</i>,</b> @@ -93,6 +109,9 @@ man page, in case the conversion went wrong. <b>*<i>firstcharptr</i>);</b> </P> <P> +<b>int pcre_refcount(pcre *<i>code</i>, int <i>adjust</i>);</b> +</P> +<P> <b>int pcre_config(int <i>what</i>, void *<i>where</i>);</b> </P> <P> @@ -115,32 +134,46 @@ man page, in case the conversion went wrong. </P> <br><a name="SEC2" href="#TOC1">PCRE API OVERVIEW</a><br> <P> -PCRE has its own native API, which is described in this document. There is also -a set of wrapper functions that correspond to the POSIX regular expression API. -These are described in the +PCRE has its own native API, which is described in this document. There is +also a set of wrapper functions that correspond to the POSIX regular expression +API. These are described in the <a href="pcreposix.html"><b>pcreposix</b></a> -documentation. +documentation. Both of these APIs define a set of C function calls. A C++ +wrapper is distributed with PCRE. It is documented in the +<a href="pcrecpp.html"><b>pcrecpp</b></a> +page. </P> <P> -The native API function prototypes are defined in the header file <b>pcre.h</b>, -and on Unix systems the library itself is called <b>libpcre</b>. It can -normally be accessed by adding <b>-lpcre</b> to the command for linking an -application that uses PCRE. The header file defines the macros PCRE_MAJOR and -PCRE_MINOR to contain the major and minor release numbers for the library. +The native API C function prototypes are defined in the header file +<b>pcre.h</b>, and on Unix systems the library itself is called <b>libpcre</b>. +It can normally be accessed by adding <b>-lpcre</b> to the command for linking +an application that uses PCRE. The header file defines the macros PCRE_MAJOR +and PCRE_MINOR to contain the major and minor release numbers for the library. Applications can use these to include support for different releases of PCRE. </P> <P> -The functions <b>pcre_compile()</b>, <b>pcre_study()</b>, and <b>pcre_exec()</b> -are used for compiling and matching regular expressions. A sample program that -demonstrates the simplest way of using them is provided in the file called -<i>pcredemo.c</i> in the source distribution. The +The functions <b>pcre_compile()</b>, <b>pcre_compile2()</b>, <b>pcre_study()</b>, +and <b>pcre_exec()</b> are used for compiling and matching regular expressions +in a Perl-compatible manner. A sample program that demonstrates the simplest +way of using them is provided in the file called <i>pcredemo.c</i> in the source +distribution. The <a href="pcresample.html"><b>pcresample</b></a> documentation describes how to run it. </P> <P> +A second matching function, <b>pcre_dfa_exec()</b>, which is not +Perl-compatible, is also provided. This uses a different algorithm for the +matching. This allows it to find all possible matches (at a given point in the +subject), not just one. However, this algorithm does not return captured +substrings. A description of the two matching algorithms and their advantages +and disadvantages is given in the +<a href="pcrematching.html"><b>pcrematching</b></a> +documentation. +</P> +<P> In addition to the main compiling and matching functions, there are convenience -functions for extracting captured substrings from a matched subject string. -They are: +functions for extracting captured substrings from a subject string that is +matched by <b>pcre_exec()</b>. They are: <pre> <b>pcre_copy_substring()</b> <b>pcre_copy_named_substring()</b> @@ -154,10 +187,10 @@ provided, to free the memory used for extracted strings. </P> <P> The function <b>pcre_maketables()</b> is used to build a set of character tables -in the current locale for passing to <b>pcre_compile()</b> or <b>pcre_exec()</b>. -This is an optional facility that is provided for specialist use. Most -commonly, no special tables are passed, in which case internal tables that are -generated when PCRE is built are used. +in the current locale for passing to <b>pcre_compile()</b>, <b>pcre_exec()</b>, +or <b>pcre_dfa_exec()</b>. This is an optional facility that is provided for +specialist use. Most commonly, no special tables are passed, in which case +internal tables that are generated when PCRE is built are used. </P> <P> The function <b>pcre_fullinfo()</b> is used to find out information about a @@ -167,6 +200,11 @@ The function <b>pcre_version()</b> returns a pointer to a string containing the version of PCRE and its date of release. </P> <P> +The function <b>pcre_refcount()</b> maintains a reference count in a data block +containing a compiled pattern. This is provided for the benefit of +object-oriented applications. +</P> +<P> The global variables <b>pcre_malloc</b> and <b>pcre_free</b> initially contain the entry points of the standard <b>malloc()</b> and <b>free()</b> functions, respectively. PCRE calls the memory management functions via these variables, @@ -177,12 +215,13 @@ should be done before calling any PCRE functions. The global variables <b>pcre_stack_malloc</b> and <b>pcre_stack_free</b> are also indirections to memory management functions. These special functions are used only when PCRE is compiled to use the heap for remembering data, instead of -recursive function calls. This is a non-standard way of building PCRE, for use -in environments that have limited stacks. Because of the greater use of memory -management, it runs more slowly. Separate functions are provided so that -special-purpose external code can be used for this case. When used, these -functions are always called in a stack-like manner (last obtained, first -freed), and always for memory blocks of the same size. +recursive function calls, when running the <b>pcre_exec()</b> function. This is +a non-standard way of building PCRE, for use in environments that have limited +stacks. Because of the greater use of memory management, it runs more slowly. +Separate functions are provided so that special-purpose external code can be +used for this case. When used, these functions are always called in a +stack-like manner (last obtained, first freed), and always for memory blocks of +the same size. </P> <P> The global variable <b>pcre_callout</b> initially contains NULL. It can be set @@ -265,27 +304,37 @@ details are given with <b>pcre_exec()</b> below. <pre> PCRE_CONFIG_STACKRECURSE </pre> -The output is an integer that is set to one if internal recursion is -implemented by recursive function calls that use the stack to remember their -state. This is the usual way that PCRE is compiled. The output is zero if PCRE -was compiled to use blocks of data on the heap instead of recursive function -calls. In this case, <b>pcre_stack_malloc</b> and <b>pcre_stack_free</b> are -called to manage memory blocks on the heap, thus avoiding the use of the stack. +The output is an integer that is set to one if internal recursion when running +<b>pcre_exec()</b> is implemented by recursive function calls that use the stack +to remember their state. This is the usual way that PCRE is compiled. The +output is zero if PCRE was compiled to use blocks of data on the heap instead +of recursive function calls. In this case, <b>pcre_stack_malloc</b> and +<b>pcre_stack_free</b> are called to manage memory blocks on the heap, thus +avoiding the use of the stack. </P> <br><a name="SEC6" href="#TOC1">COMPILING A PATTERN</a><br> <P> <b>pcre *pcre_compile(const char *<i>pattern</i>, int <i>options</i>,</b> <b>const char **<i>errptr</i>, int *<i>erroffset</i>,</b> <b>const unsigned char *<i>tableptr</i>);</b> +<b>pcre *pcre_compile2(const char *<i>pattern</i>, int <i>options</i>,</b> +<b>int *<i>errorcodeptr</i>,</b> +<b>const char **<i>errptr</i>, int *<i>erroffset</i>,</b> +<b>const unsigned char *<i>tableptr</i>);</b> +</P> +<P> +Either of the functions <b>pcre_compile()</b> or <b>pcre_compile2()</b> can be +called to compile a pattern into an internal form. The only difference between +the two interfaces is that <b>pcre_compile2()</b> has an additional argument, +<i>errorcodeptr</i>, via which a numerical error code can be returned. </P> <P> -The function <b>pcre_compile()</b> is called to compile a pattern into an -internal form. The pattern is a C string terminated by a binary zero, and -is passed in the <i>pattern</i> argument. A pointer to a single block of memory -that is obtained via <b>pcre_malloc</b> is returned. This contains the compiled -code and related data. The <b>pcre</b> type is defined for the returned block; -this is a typedef for a structure whose contents are not externally defined. It -is up to the caller to free the memory when it is no longer required. +The pattern is a C string terminated by a binary zero, and is passed in the +<i>pattern</i> argument. A pointer to a single block of memory that is obtained +via <b>pcre_malloc</b> is returned. This contains the compiled code and related +data. The <b>pcre</b> type is defined for the returned block; this is a typedef +for a structure whose contents are not externally defined. It is up to the +caller to free the memory when it is no longer required. </P> <P> Although the compiled code of a PCRE regex is relocatable, that is, it does not @@ -314,6 +363,12 @@ the error was discovered is placed in the variable pointed to by <i>erroffset</i>, which must not be NULL. If it is, an immediate error is given. </P> <P> +If <b>pcre_compile2()</b> is used instead of <b>pcre_compile()</b>, and the +<i>errorcodeptr</i> argument is not NULL, a non-zero error code number is +returned via this argument in the event of an error. This is in addition to the +textual error message. Error codes and messages are listed below. +</P> +<P> If the final argument, <i>tableptr</i>, is NULL, PCRE uses a default set of character tables that are built when PCRE is compiled, using the default C locale. Otherwise, <i>tableptr</i> must be an address that is the result of a @@ -357,9 +412,13 @@ documentation. </pre> If this bit is set, letters in the pattern match both upper and lower case letters. It is equivalent to Perl's /i option, and it can be changed within a -pattern by a (?i) option setting. When running in UTF-8 mode, case support for -high-valued characters is available only when PCRE is built with Unicode -character property support. +pattern by a (?i) option setting. In UTF-8 mode, PCRE always understands the +concept of case for characters whose values are less than 128, so caseless +matching is always possible. For characters with higher values, the concept of +case is supported if PCRE is compiled with Unicode property support, but not +otherwise. If you want to use caseless matching for characters 128 and above, +you must ensure that PCRE is compiled with Unicode property support as well as +with UTF-8 support. <pre> PCRE_DOLLAR_ENDONLY </pre> @@ -404,6 +463,12 @@ special meaning is treated as a literal. There are at present no other features controlled by this option. It can also be set by a (?X) option setting within a pattern. <pre> + PCRE_FIRSTLINE +</pre> +If this option is set, an unanchored pattern is required to match before or at +the first newline character in the subject string, though the matched text may +continue over the newline. +<pre> PCRE_MULTILINE </pre> By default, PCRE treats the subject string as consisting of a single line of @@ -455,12 +520,69 @@ automatically checked. If an invalid UTF-8 sequence of bytes is found, valid, and you want to skip this check for performance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is set, the effect of passing an invalid UTF-8 string as a pattern is undefined. It may cause your program to crash. -Note that this option can also be passed to <b>pcre_exec()</b>, to suppress the -UTF-8 validity checking of subject strings. -</P> -<br><a name="SEC7" href="#TOC1">STUDYING A PATTERN</a><br> -<P> -<b>pcre_extra *pcre_study(const pcre *<i>code</i>, int <i>options</i>,</b> +Note that this option can also be passed to <b>pcre_exec()</b> and +<b>pcre_dfa_exec()</b>, to suppress the UTF-8 validity checking of subject +strings. +</P> +<br><a name="SEC7" href="#TOC1">COMPILATION ERROR CODES</a><br> +<P> +The following table lists the error codes than may be returned by +<b>pcre_compile2()</b>, along with the error messages that may be returned by +both compiling functions. +<pre> + 0 no error + 1 \ at end of pattern + 2 \c at end of pattern + 3 unrecognized character follows \ + 4 numbers out of order in {} quantifier + 5 number too big in {} quantifier + 6 missing terminating ] for character class + 7 invalid escape sequence in character class + 8 range out of order in character class + 9 nothing to repeat + 10 operand of unlimited repeat could match the empty string + 11 internal error: unexpected repeat + 12 unrecognized character after (? + 13 POSIX named classes are supported only within a class + 14 missing ) + 15 reference to non-existent subpattern + 16 erroffset passed as NULL + 17 unknown option bit(s) set + 18 missing ) after comment + 19 parentheses nested too deeply + 20 regular expression too large + 21 failed to get memory + 22 unmatched parentheses + 23 internal error: code overflow + 24 unrecognized character after (?< + 25 lookbehind assertion is not fixed length + 26 malformed number after (?( + 27 conditional group contains more than two branches + 28 assertion expected after (?( + 29 (?R or (?digits must be followed by ) + 30 unknown POSIX class name + 31 POSIX collating elements are not supported + 32 this version of PCRE is not compiled with PCRE_UTF8 support + 33 spare error + 34 character value in \x{...} sequence is too large + 35 invalid condition (?(0) + 36 \C not allowed in lookbehind assertion + 37 PCRE does not support \L, \l, \N, \U, or \u + 38 number after (?C is > 255 + 39 closing ) for (?C expected + 40 recursive call could loop indefinitely + 41 unrecognized character after (?P + 42 syntax error after (?P + 43 two named groups have the same name + 44 invalid UTF-8 string + 45 support for \P, \p, and \X has not been compiled + 46 malformed \P or \p sequence + 47 unknown property name after \P or \p +</PRE> +</P> +<br><a name="SEC8" href="#TOC1">STUDYING A PATTERN</a><br> +<P> +<b>pcre_extra *pcre_study(const pcre *<i>code</i>, int <i>options</i></b> <b>const char **<i>errptr</i>);</b> </P> <P> @@ -481,7 +603,7 @@ described in the section on matching a pattern. </P> <P> -If studying the pattern does not produce any additional information, +If studying the pattern does not produce any additional information <b>pcre_study()</b> returns NULL. In that circumstance, if the calling program wants to pass any of the other fields to <b>pcre_exec()</b>, it must set up its own <b>pcre_extra</b> block. @@ -510,14 +632,14 @@ At present, studying a pattern is useful only for non-anchored patterns that do not have a single fixed starting character. A bitmap of possible starting bytes is created. <a name="localesupport"></a></P> -<br><a name="SEC8" href="#TOC1">LOCALE SUPPORT</a><br> +<br><a name="SEC9" href="#TOC1">LOCALE SUPPORT</a><br> <P> -PCRE handles caseless matching, and determines whether characters are letters, +PCRE handles caseless matching, and determines whether characters are letters digits, or whatever, by reference to a set of tables, indexed by character -value. (When running in UTF-8 mode, this applies only to characters with codes +value. When running in UTF-8 mode, this applies only to characters with codes less than 128. Higher-valued codes never match escapes such as \w or \d, but can be tested with \p if PCRE is built with Unicode character property -support.) +support. </P> <P> An internal set of tables is created in the default C locale when PCRE is @@ -558,7 +680,7 @@ this facility could be used to match a pattern in a different locale from the one in which it was compiled. Passing table pointers at run time is discussed below in the section on matching a pattern. </P> -<br><a name="SEC9" href="#TOC1">INFORMATION ABOUT A PATTERN</a><br> +<br><a name="SEC10" href="#TOC1">INFORMATION ABOUT A PATTERN</a><br> <P> <b>int pcre_fullinfo(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> <b>int <i>what</i>, void *<i>where</i>);</b> @@ -607,7 +729,7 @@ no back references. Return the number of capturing subpatterns in the pattern. The fourth argument should point to an <b>int</b> variable. <pre> - PCRE_INFO_DEFAULTTABLES + PCRE_INFO_DEFAULT_TABLES </pre> Return a pointer to the internal default character tables within PCRE. The fourth argument should point to an <b>unsigned char *</b> variable. This @@ -729,7 +851,7 @@ a <b>pcre_extra</b> block. That is, it is the value that was passed to created by <b>pcre_study()</b>. The fourth argument should point to a <b>size_t</b> variable. </P> -<br><a name="SEC10" href="#TOC1">OBSOLETE INFO FUNCTION</a><br> +<br><a name="SEC11" href="#TOC1">OBSOLETE INFO FUNCTION</a><br> <P> <b>int pcre_info(const pcre *<i>code</i>, int *<i>optptr</i>, int</b> <b>*<i>firstcharptr</i>);</b> @@ -753,7 +875,31 @@ If the pattern is not anchored and the <i>firstcharptr</i> argument is not NULL, it is used to pass back information about the first character of any matched string (see PCRE_INFO_FIRSTBYTE above). </P> -<br><a name="SEC11" href="#TOC1">MATCHING A PATTERN</a><br> +<br><a name="SEC12" href="#TOC1">REFERENCE COUNTS</a><br> +<P> +<b>int pcre_refcount(pcre *<i>code</i>, int <i>adjust</i>);</b> +</P> +<P> +The <b>pcre_refcount()</b> function is used to maintain a reference count in the +data block that contains a compiled pattern. It is provided for the benefit of +applications that operate in an object-oriented manner, where different parts +of the application may be using the same compiled pattern, but you want to free +the block when they are all done. +</P> +<P> +When a pattern is compiled, the reference count field is initialized to zero. +It is changed only by calling this function, whose action is to add the +<i>adjust</i> value (which may be positive or negative) to it. The yield of the +function is the new value. However, the value of the count is constrained to +lie between 0 and 65535, inclusive. If the new value is outside these limits, +it is forced to the appropriate limit value. +</P> +<P> +Except when it is zero, the reference count is not correctly preserved if a +pattern is compiled on one host and then transferred to a host whose byte-order +is different. (This seems a highly unlikely scenario.) +</P> +<br><a name="SEC13" href="#TOC1">MATCHING A PATTERN: THE TRADITIONAL FUNCTION</a><br> <P> <b>int pcre_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> <b>const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> @@ -763,7 +909,11 @@ string (see PCRE_INFO_FIRSTBYTE above). The function <b>pcre_exec()</b> is called to match a subject string against a compiled pattern, which is passed in the <i>code</i> argument. If the pattern has been studied, the result of the study should be passed in the -<i>extra</i> argument. +<i>extra</i> argument. This function is the main matching facility of the +library, and it operates in a Perl-like manner. For specialist use there is +also an alternative matching function, which is described +<a href="#dfamatch">below</a> +in the section about the <b>pcre_dfa_exec()</b> function. </P> <P> In most applications, the pattern will have been compiled (and optionally @@ -787,7 +937,7 @@ Here is an example of a simple call to <b>pcre_exec()</b>: 0, /* start at offset 0 in the subject */ 0, /* default options */ ovector, /* vector of integers for substring information */ - 30); /* number of elements in the vector (NOT size in bytes) */ + 30); /* number of elements (NOT size in bytes) */ <a name="extradata"></a></PRE> </P> <br><b> @@ -1046,7 +1196,7 @@ Note that <b>pcre_info()</b> can be used to find out how many capturing subpatterns there are in a compiled pattern. The smallest size for <i>ovector</i> that will allow for <i>n</i> captured substrings, in addition to the offsets of the substring matched by the whole pattern, is (<i>n</i>+1)*3. -</P> +<a name="errorlist"></a></P> <br><b> Return values from <b>pcre_exec()</b> </b><br> @@ -1117,29 +1267,29 @@ A string that contains an invalid UTF-8 byte sequence was passed as a subject. The UTF-8 byte sequence that was passed as a subject was valid, but the value of <i>startoffset</i> did not point to the beginning of a UTF-8 character. <pre> - PCRE_ERROR_PARTIAL (-12) + PCRE_ERROR_PARTIAL (-12) </pre> The subject string did not match, but it did match partially. See the <a href="pcrepartial.html"><b>pcrepartial</b></a> documentation for details of partial matching. <pre> - PCRE_ERROR_BAD_PARTIAL (-13) + PCRE_ERROR_BADPARTIAL (-13) </pre> The PCRE_PARTIAL option was used with a compiled pattern containing items that are not supported for partial matching. See the <a href="pcrepartial.html"><b>pcrepartial</b></a> documentation for details of partial matching. <pre> - PCRE_ERROR_INTERNAL (-14) + PCRE_ERROR_INTERNAL (-14) </pre> An unexpected internal error has occurred. This error could be caused by a bug in PCRE or by overwriting of the compiled pattern. <pre> - PCRE_ERROR_BADCOUNT (-15) + PCRE_ERROR_BADCOUNT (-15) </pre> This error is given if the value of the <i>ovecsize</i> argument is negative. </P> -<br><a name="SEC12" href="#TOC1">EXTRACTING CAPTURED SUBSTRINGS BY NUMBER</a><br> +<br><a name="SEC14" href="#TOC1">EXTRACTING CAPTURED SUBSTRINGS BY NUMBER</a><br> <P> <b>int pcre_copy_substring(const char *<i>subject</i>, int *<i>ovector</i>,</b> <b>int <i>stringcount</i>, int <i>stringnumber</i>, char *<i>buffer</i>,</b> @@ -1227,7 +1377,7 @@ linked via a special interface to another programming language which cannot use <b>pcre_free</b> directly; it is for these cases that the functions are provided. </P> -<br><a name="SEC13" href="#TOC1">EXTRACTING CAPTURED SUBSTRINGS BY NAME</a><br> +<br><a name="SEC15" href="#TOC1">EXTRACTING CAPTURED SUBSTRINGS BY NAME</a><br> <P> <b>int pcre_get_stringnumber(const pcre *<i>code</i>,</b> <b>const char *<i>name</i>);</b> @@ -1278,10 +1428,191 @@ These functions call <b>pcre_get_stringnumber()</b>, and if it succeeds, they then call <i>pcre_copy_substring()</i> or <i>pcre_get_substring()</i>, as appropriate. </P> +<br><a name="SEC16" href="#TOC1">FINDING ALL POSSIBLE MATCHES</a><br> +<P> +The traditional matching function uses a similar algorithm to Perl, which stops +when it finds the first match, starting at a given point in the subject. If you +want to find all possible matches, or the longest possible match, consider +using the alternative matching function (see below) instead. If you cannot use +the alternative function, but still need to find all possible matches, you +can kludge it up by making use of the callout facility, which is described in +the +<a href="pcrecallout.html"><b>pcrecallout</b></a> +documentation. +</P> +<P> +What you have to do is to insert a callout right at the end of the pattern. +When your callout function is called, extract and save the current matched +substring. Then return 1, which forces <b>pcre_exec()</b> to backtrack and try +other alternatives. Ultimately, when it runs out of matches, <b>pcre_exec()</b> +will yield PCRE_ERROR_NOMATCH. +<a name="dfamatch"></a></P> +<br><a name="SEC17" href="#TOC1">MATCHING A PATTERN: THE ALTERNATIVE FUNCTION</a><br> +<P> +<b>int pcre_dfa_exec(const pcre *<i>code</i>, const pcre_extra *<i>extra</i>,</b> +<b>const char *<i>subject</i>, int <i>length</i>, int <i>startoffset</i>,</b> +<b>int <i>options</i>, int *<i>ovector</i>, int <i>ovecsize</i>,</b> +<b>int *<i>workspace</i>, int <i>wscount</i>);</b> +</P> +<P> +The function <b>pcre_dfa_exec()</b> is called to match a subject string against +a compiled pattern, using a "DFA" matching algorithm. This has different +characteristics to the normal algorithm, and is not compatible with Perl. Some +of the features of PCRE patterns are not supported. Nevertheless, there are +times when this kind of matching can be useful. For a discussion of the two +matching algorithms, see the +<a href="pcrematching.html"><b>pcrematching</b></a> +documentation. +</P> +<P> +The arguments for the <b>pcre_dfa_exec()</b> function are the same as for +<b>pcre_exec()</b>, plus two extras. The <i>ovector</i> argument is used in a +different way, and this is described below. The other common arguments are used +in the same way as for <b>pcre_exec()</b>, so their description is not repeated +here. +</P> +<P> +The two additional arguments provide workspace for the function. The workspace +vector should contain at least 20 elements. It is used for keeping track of +multiple paths through the pattern tree. More workspace will be needed for +patterns and subjects where there are a lot of possible matches. +</P> +<P> +Here is an example of a simple call to <b>pcre_exec()</b>: +<pre> + int rc; + int ovector[10]; + int wspace[20]; + rc = pcre_exec( + re, /* result of pcre_compile() */ + NULL, /* we didn't study the pattern */ + "some string", /* the subject string */ + 11, /* the length of the subject string */ + 0, /* start at offset 0 in the subject */ + 0, /* default options */ + ovector, /* vector of integers for substring information */ + 10, /* number of elements (NOT size in bytes) */ + wspace, /* working space vector */ + 20); /* number of elements (NOT size in bytes) */ +</PRE> +</P> +<br><b> +Option bits for <b>pcre_dfa_exec()</b> +</b><br> +<P> +The unused bits of the <i>options</i> argument for <b>pcre_dfa_exec()</b> must be +zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NOTBOL, +PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL, +PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last three of these are +the same as for <b>pcre_exec()</b>, so their description is not repeated here. +<pre> + PCRE_PARTIAL +</pre> +This has the same general effect as it does for <b>pcre_exec()</b>, but the +details are slightly different. When PCRE_PARTIAL is set for +<b>pcre_dfa_exec()</b>, the return code PCRE_ERROR_NOMATCH is converted into +PCRE_ERROR_PARTIAL if the end of the subject is reached, there have been no +complete matches, but there is still at least one matching possibility. The +portion of the string that provided the partial match is set as the first +matching string. +<pre> + PCRE_DFA_SHORTEST +</pre> +Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to stop as +soon as it has found one match. Because of the way the DFA algorithm works, +this is necessarily the shortest possible match at the first possible matching +point in the subject string. +<pre> + PCRE_DFA_RESTART +</pre> +When <b>pcre_dfa_exec()</b> is called with the PCRE_PARTIAL option, and returns +a partial match, it is possible to call it again, with additional subject +characters, and have it continue with the same match. The PCRE_DFA_RESTART +option requests this action; when it is set, the <i>workspace</i> and +<i>wscount</i> options must reference the same vector as before because data +about the match so far is left in them after a partial match. There is more +discussion of this facility in the +<a href="pcrepartial.html"><b>pcrepartial</b></a> +documentation. +</P> +<br><b> +Successful returns from <b>pcre_dfa_exec()</b> +</b><br> +<P> +When <b>pcre_dfa_exec()</b> succeeds, it may have matched more than one +substring in the subject. Note, however, that all the matches from one run of +the function start at the same point in the subject. The shorter matches are +all initial substrings of the longer matches. For example, if the pattern +<pre> + <.*> +</pre> +is matched against the string +<pre> + This is <something> <something else> <something further> no more +</pre> +the three matched strings are +<pre> + <something> + <something> <something else> + <something> <something else> <something further> +</pre> +On success, the yield of the function is a number greater than zero, which is +the number of matched substrings. The substrings themselves are returned in +<i>ovector</i>. Each string uses two elements; the first is the offset to the +start, and the second is the offset to the end. All the strings have the same +start offset. (Space could have been saved by giving this only once, but it was +decided to retain some compatibility with the way <b>pcre_exec()</b> returns +data, even though the meaning of the strings is different.) +</P> +<P> +The strings are returned in reverse order of length; that is, the longest +matching string is given first. If there were too many matches to fit into +<i>ovector</i>, the yield of the function is zero, and the vector is filled with +the longest matches. +</P> +<br><b> +Error returns from <b>pcre_dfa_exec()</b> +</b><br> +<P> +The <b>pcre_dfa_exec()</b> function returns a negative number when it fails. +Many of the errors are the same as for <b>pcre_exec()</b>, and these are +described +<a href="#errorlist">above.</a> +There are in addition the following errors that are specific to +<b>pcre_dfa_exec()</b>: +<pre> + PCRE_ERROR_DFA_UITEM (-16) +</pre> +This return is given if <b>pcre_dfa_exec()</b> encounters an item in the pattern +that it does not support, for instance, the use of \C or a back reference. +<pre> + PCRE_ERROR_DFA_UCOND (-17) +</pre> +This return is given if <b>pcre_dfa_exec()</b> encounters a condition item in a +pattern that uses a back reference for the condition. This is not supported. +<pre> + PCRE_ERROR_DFA_UMLIMIT (-18) +</pre> +This return is given if <b>pcre_dfa_exec()</b> is called with an <i>extra</i> +block that contains a setting of the <i>match_limit</i> field. This is not +supported (it is meaningless). +<pre> + PCRE_ERROR_DFA_WSSIZE (-19) +</pre> +This return is given if <b>pcre_dfa_exec()</b> runs out of space in the +<i>workspace</i> vector. +<pre> + PCRE_ERROR_DFA_RECURSE (-20) +</pre> +When a recursive subpattern is processed, the matching function calls itself +recursively, using private vectors for <i>ovector</i> and <i>workspace</i>. This +error is given if the output vector is not large enough. This should be +extremely rare, as a vector of size 1000 is used. +</P> <P> -Last updated: 09 September 2004 +Last updated: 16 May 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcrebuild.html b/doc/html/pcrebuild.html index 98c7d27..4f9ff8b 100644 --- a/doc/html/pcrebuild.html +++ b/doc/html/pcrebuild.html @@ -113,17 +113,19 @@ to the <b>configure</b> command. <br><a name="SEC7" href="#TOC1">LIMITING PCRE RESOURCE USAGE</a><br> <P> Internally, PCRE has a function called <b>match()</b>, which it calls repeatedly -(possibly recursively) when matching a pattern. By controlling the maximum -number of times this function may be called during a single matching operation, -a limit can be placed on the resources used by a single call to -<b>pcre_exec()</b>. The limit can be changed at run time, as described in the +(possibly recursively) when matching a pattern with the <b>pcre_exec()</b> +function. By controlling the maximum number of times this function may be +called during a single matching operation, a limit can be placed on the +resources used by a single call to <b>pcre_exec()</b>. The limit can be changed +at run time, as described in the <a href="pcreapi.html"><b>pcreapi</b></a> documentation. The default is 10 million, but this can be changed by adding a setting such as <pre> --with-match-limit=500000 </pre> -to the <b>configure</b> command. +to the <b>configure</b> command. This setting has no effect on the +<b>pcre_dfa_exec()</b> matching function. </P> <br><a name="SEC8" href="#TOC1">HANDLING VERY LARGE PATTERNS</a><br> <P> @@ -148,13 +150,14 @@ of the compiled pattern, and this changes with the link size. </P> <br><a name="SEC9" href="#TOC1">AVOIDING EXCESSIVE STACK USAGE</a><br> <P> -PCRE implements backtracking while matching by making recursive calls to an -internal function called <b>match()</b>. In environments where the size of the -stack is limited, this can severely limit PCRE's operation. (The Unix -environment does not usually suffer from this problem.) An alternative approach -that uses memory from the heap to remember data, instead of using recursive -function calls, has been implemented to work round this problem. If you want to -build a version of PCRE that works this way, add +When matching with the <b>pcre_exec()</b> function, PCRE implements backtracking +by making recursive calls to an internal function called <b>match()</b>. In +environments where the size of the stack is limited, this can severely limit +PCRE's operation. (The Unix environment does not usually suffer from this +problem.) An alternative approach that uses memory from the heap to remember +data, instead of using recursive function calls, has been implemented to work +round this problem. If you want to build a version of PCRE that works this way, +add <pre> --disable-stack-for-recursion </pre> @@ -165,7 +168,8 @@ predictable: the block sizes requested are always the same, and the blocks are always freed in reverse order. A calling program might be able to implement optimized functions that perform better than the standard <b>malloc()</b> and <b>free()</b> functions. PCRE runs noticeably more slowly when built in this -way. +way. This option affects only the <b>pcre_exec()</b> function; it is not +relevant for the the <b>pcre_dfa_exec()</b> function. </P> <br><a name="SEC10" href="#TOC1">USING EBCDIC CODE</a><br> <P> @@ -178,9 +182,9 @@ compiled to run in an EBCDIC environment by adding to the <b>configure</b> command. </P> <P> -Last updated: 09 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcrecallout.html b/doc/html/pcrecallout.html index dc2ef51..5ccfd5c 100644 --- a/doc/html/pcrecallout.html +++ b/doc/html/pcrecallout.html @@ -72,9 +72,10 @@ no match, the callout is obeyed. <br><a name="SEC3" href="#TOC1">THE CALLOUT INTERFACE</a><br> <P> During matching, when PCRE reaches a callout point, the external function -defined by <i>pcre_callout</i> is called (if it is set). The only argument is a -pointer to a <b>pcre_callout</b> block. This structure contains the following -fields: +defined by <i>pcre_callout</i> is called (if it is set). This applies to both +the <b>pcre_exec()</b> and the <b>pcre_dfa_exec()</b> matching functions. The +only argument to the callout function is a pointer to a <b>pcre_callout</b> +block. This structure contains the following fields: <pre> int <i>version</i>; int <i>callout_number</i>; @@ -101,9 +102,11 @@ automatically generated callouts). </P> <P> The <i>offset_vector</i> field is a pointer to the vector of offsets that was -passed by the caller to <b>pcre_exec()</b>. The contents can be inspected in -order to extract substrings that have been matched so far, in the same way as -for extracting substrings after a match has completed. +passed by the caller to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>. When +<b>pcre_exec()</b> is used, the contents can be inspected in order to extract +substrings that have been matched so far, in the same way as for extracting +substrings after a match has completed. For <b>pcre_dfa_exec()</b> this field is +not useful. </P> <P> The <i>subject</i> and <i>subject_length</i> fields contain copies of the values @@ -120,21 +123,24 @@ The <i>current_position</i> field contains the offset within the subject of the current match pointer. </P> <P> -The <i>capture_top</i> field contains one more than the number of the highest -numbered captured substring so far. If no substrings have been captured, -the value of <i>capture_top</i> is one. +When the <b>pcre_exec()</b> function is used, the <i>capture_top</i> field +contains one more than the number of the highest numbered captured substring so +far. If no substrings have been captured, the value of <i>capture_top</i> is +one. This is always the case when <b>pcre_dfa_exec()</b> is used, because it +does not support captured substrings. </P> <P> The <i>capture_last</i> field contains the number of the most recently captured -substring. If no substrings have been captured, its value is -1. +substring. If no substrings have been captured, its value is -1. This is always +the case when <b>pcre_dfa_exec()</b> is used. </P> <P> The <i>callout_data</i> field contains a value that is passed to -<b>pcre_exec()</b> by the caller specifically so that it can be passed back in -callouts. It is passed in the <i>pcre_callout</i> field of the <b>pcre_extra</b> -data structure. If no such data was passed, the value of <i>callout_data</i> in -a <b>pcre_callout</b> block is NULL. There is a description of the -<b>pcre_extra</b> structure in the +<b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> specifically so that it can be +passed back in callouts. It is passed in the <i>pcre_callout</i> field of the +<b>pcre_extra</b> data structure. If no such data was passed, the value of +<i>callout_data</i> in a <b>pcre_callout</b> block is NULL. There is a +description of the <b>pcre_extra</b> structure in the <a href="pcreapi.html"><b>pcreapi</b></a> documentation. </P> @@ -160,10 +166,10 @@ same callout number. However, they are set for all callouts. <P> The external callout function returns an integer to PCRE. If the value is zero, matching proceeds as normal. If the value is greater than zero, matching fails -at the current point, but backtracking to test other matching possibilities -goes ahead, just as if a lookahead assertion had failed. If the value is less -than zero, the match is abandoned, and <b>pcre_exec()</b> returns the negative -value. +at the current point, but the testing of other matching possibilities goes +ahead, just as if a lookahead assertion had failed. If the value is less than +zero, the match is abandoned, and <b>pcre_exec()</b> (or <b>pcre_dfa_exec()</b>) +returns the negative value. </P> <P> Negative values should normally be chosen from the set of PCRE_ERROR_xxx @@ -172,9 +178,9 @@ The error number PCRE_ERROR_CALLOUT is reserved for use by callout functions; it will never be used by PCRE itself. </P> <P> -Last updated: 09 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcrecompat.html b/doc/html/pcrecompat.html index 6529c09..d15b3b0 100644 --- a/doc/html/pcrecompat.html +++ b/doc/html/pcrecompat.html @@ -140,11 +140,15 @@ package. <br> (m) Patterns compiled by PCRE can be saved and re-used at a later time, even on different hosts that have the other endianness. +<br> +<br> +(n) The alternative matching function (<b>pcre_dfa_exec()</b>) matches in a +different way and is not Perl-compatible. </P> <P> -Last updated: 09 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcrecpp.html b/doc/html/pcrecpp.html new file mode 100644 index 0000000..3f597b1 --- /dev/null +++ b/doc/html/pcrecpp.html @@ -0,0 +1,241 @@ +<html> +<head> +<title>pcrecpp specification</title> +</head> +<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> +<h1>pcrecpp man page</h1> +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> +<p> +This page is part of the PCRE HTML documentation. It was generated automatically +from the original man page. If there is any nonsense in it, please consult the +man page, in case the conversion went wrong. +<br> +<ul> +<li><a name="TOC1" href="#SEC1">SYNOPSIS OF C++ WRAPPER</a> +<li><a name="TOC2" href="#SEC2">DESCRIPTION</a> +<li><a name="TOC3" href="#SEC3">MATCHING INTERFACE</a> +<li><a name="TOC4" href="#SEC4">PARTIAL MATCHES</a> +<li><a name="TOC5" href="#SEC5">UTF-8 AND THE MATCHING INTERFACE</a> +<li><a name="TOC6" href="#SEC6">SCANNING TEXT INCREMENTALLY</a> +<li><a name="TOC7" href="#SEC7">PARSING HEX/OCTAL/C-RADIX NUMBERS</a> +<li><a name="TOC8" href="#SEC8">REPLACING PARTS OF STRINGS</a> +<li><a name="TOC9" href="#SEC9">AUTHOR</a> +</ul> +<br><a name="SEC1" href="#TOC1">SYNOPSIS OF C++ WRAPPER</a><br> +<P> +<b>#include <pcrecpp.h></b> +</P> +<P> +</P> +<br><a name="SEC2" href="#TOC1">DESCRIPTION</a><br> +<P> +The C++ wrapper for PCRE was provided by Google Inc. This brief man page was +constructed from the notes in the <i>pcrecpp.h</i> file, which should be +consulted for further details. +</P> +<br><a name="SEC3" href="#TOC1">MATCHING INTERFACE</a><br> +<P> +The "FullMatch" operation checks that supplied text matches a supplied pattern +exactly. If pointer arguments are supplied, it copies matched sub-strings that +match sub-patterns into them. +<pre> + Example: successful match + pcrecpp::RE re("h.*o"); + re.FullMatch("hello"); + + Example: unsuccessful match (requires full match): + pcrecpp::RE re("e"); + !re.FullMatch("hello"); + + Example: creating a temporary RE object: + pcrecpp::RE("h.*o").FullMatch("hello"); +</pre> +You can pass in a "const char*" or a "string" for "text". The examples below +tend to use a const char*. You can, as in the different examples above, store +the RE object explicitly in a variable or use a temporary RE object. The +examples below use one mode or the other arbitrarily. Either could correctly be +used for any of these examples. +</P> +<P> +You must supply extra pointer arguments to extract matched subpieces. +<pre> + Example: extracts "ruby" into "s" and 1234 into "i" + int i; + string s; + pcrecpp::RE re("(\\w+):(\\d+)"); + re.FullMatch("ruby:1234", &s, &i); + + Example: does not try to extract any extra sub-patterns + re.FullMatch("ruby:1234", &s); + + Example: does not try to extract into NULL + re.FullMatch("ruby:1234", NULL, &i); + + Example: integer overflow causes failure + !re.FullMatch("ruby:1234567891234", NULL, &i); + + Example: fails because there aren't enough sub-patterns: + !pcrecpp::RE("\\w+:\\d+").FullMatch("ruby:1234", &s); + + Example: fails because string cannot be stored in integer + !pcrecpp::RE("(.*)").FullMatch("ruby", &i); +</pre> +The provided pointer arguments can be pointers to any scalar numeric +type, or one of: +<pre> + string (matched piece is copied to string) + StringPiece (StringPiece is mutated to point to matched piece) + T (where "bool T::ParseFrom(const char*, int)" exists) + NULL (the corresponding matched sub-pattern is not copied) +</pre> +The function returns true iff all of the following conditions are satisfied: +<pre> + a. "text" matches "pattern" exactly; + + b. The number of matched sub-patterns is >= number of supplied + pointers; + + c. The "i"th argument has a suitable type for holding the + string captured as the "i"th sub-pattern. If you pass in + NULL for the "i"th argument, or pass fewer arguments than + number of sub-patterns, "i"th captured sub-pattern is + ignored. +</pre> +The matching interface supports at most 16 arguments per call. +If you need more, consider using the more general interface +<b>pcrecpp::RE::DoMatch</b>. See <b>pcrecpp.h</b> for the signature for +<b>DoMatch</b>. +</P> +<br><a name="SEC4" href="#TOC1">PARTIAL MATCHES</a><br> +<P> +You can use the "PartialMatch" operation when you want the pattern +to match any substring of the text. +<pre> + Example: simple search for a string: + pcrecpp::RE("ell").PartialMatch("hello"); + + Example: find first number in a string: + int number; + pcrecpp::RE re("(\\d+)"); + re.PartialMatch("x*100 + 20", &number); + assert(number == 100); +</PRE> +</P> +<br><a name="SEC5" href="#TOC1">UTF-8 AND THE MATCHING INTERFACE</a><br> +<P> +By default, pattern and text are plain text, one byte per character. The UTF8 +flag, passed to the constructor, causes both pattern and string to be treated +as UTF-8 text, still a byte stream but potentially multiple bytes per +character. In practice, the text is likelier to be UTF-8 than the pattern, but +the match returned may depend on the UTF8 flag, so always use it when matching +UTF8 text. For example, "." will match one byte normally but with UTF8 set may +match up to three bytes of a multi-byte character. +<pre> + Example: + pcrecpp::RE_Options options; + options.set_utf8(); + pcrecpp::RE re(utf8_pattern, options); + re.FullMatch(utf8_string); + + Example: using the convenience function UTF8(): + pcrecpp::RE re(utf8_pattern, pcrecpp::UTF8()); + re.FullMatch(utf8_string); +</pre> +NOTE: The UTF8 flag is ignored if pcre was not configured with the +<pre> + --enable-utf8 flag. +</PRE> +</P> +<br><a name="SEC6" href="#TOC1">SCANNING TEXT INCREMENTALLY</a><br> +<P> +The "Consume" operation may be useful if you want to repeatedly +match regular expressions at the front of a string and skip over +them as they match. This requires use of the "StringPiece" type, +which represents a sub-range of a real string. Like RE, StringPiece +is defined in the pcrecpp namespace. +<pre> + Example: read lines of the form "var = value" from a string. + string contents = ...; // Fill string somehow + pcrecpp::StringPiece input(contents); // Wrap in a StringPiece +</PRE> +</P> +<P> +<pre> + string var; + int value; + pcrecpp::RE re("(\\w+) = (\\d+)\n"); + while (re.Consume(&input, &var, &value)) { + ...; + } +</pre> +Each successful call to "Consume" will set "var/value", and also +advance "input" so it points past the matched text. +</P> +<P> +The "FindAndConsume" operation is similar to "Consume" but does not +anchor your match at the beginning of the string. For example, you +could extract all words from a string by repeatedly calling +<pre> + pcrecpp::RE("(\\w+)").FindAndConsume(&input, &word) +</PRE> +</P> +<br><a name="SEC7" href="#TOC1">PARSING HEX/OCTAL/C-RADIX NUMBERS</a><br> +<P> +By default, if you pass a pointer to a numeric value, the +corresponding text is interpreted as a base-10 number. You can +instead wrap the pointer with a call to one of the operators Hex(), +Octal(), or CRadix() to interpret the text in another base. The +CRadix operator interprets C-style "0" (base-8) and "0x" (base-16) +prefixes, but defaults to base-10. +<pre> + Example: + int a, b, c, d; + pcrecpp::RE re("(.*) (.*) (.*) (.*)"); + re.FullMatch("100 40 0100 0x40", + pcrecpp::Octal(&a), pcrecpp::Hex(&b), + pcrecpp::CRadix(&c), pcrecpp::CRadix(&d)); +</pre> +will leave 64 in a, b, c, and d. +</P> +<br><a name="SEC8" href="#TOC1">REPLACING PARTS OF STRINGS</a><br> +<P> +You can replace the first match of "pattern" in "str" with "rewrite". +Within "rewrite", backslash-escaped digits (\1 to \9) can be +used to insert text matching corresponding parenthesized group +from the pattern. \0 in "rewrite" refers to the entire matching +text. For example: +<pre> + string s = "yabba dabba doo"; + pcrecpp::RE("b+").Replace("d", &s); +</pre> +will leave "s" containing "yada dabba doo". The result is true if the pattern +matches and a replacement occurs, false otherwise. +</P> +<P> +<b>GlobalReplace</b> is like <b>Replace</b> except that it replaces all +occurrences of the pattern in the string with the rewrite. Replacements are +not subject to re-matching. For example: +<pre> + string s = "yabba dabba doo"; + pcrecpp::RE("b+").GlobalReplace("d", &s); +</pre> +will leave "s" containing "yada dada doo". It returns the number of +replacements made. +</P> +<P> +<b>Extract</b> is like <b>Replace</b>, except that if the pattern matches, +"rewrite" is copied into "out" (an additional argument) with substitutions. +The non-matching portions of "text" are ignored. Returns true iff a match +occurred and the extraction happened successfully; if no match occurs, the +string is left unaffected. +</P> +<br><a name="SEC9" href="#TOC1">AUTHOR</a><br> +<P> +The C++ wrapper was contributed by Google Inc. +<br> +Copyright © 2005 Google Inc. +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> diff --git a/doc/html/pcregrep.html b/doc/html/pcregrep.html index 922487d..614034d 100644 --- a/doc/html/pcregrep.html +++ b/doc/html/pcregrep.html @@ -17,12 +17,13 @@ man page, in case the conversion went wrong. <li><a name="TOC2" href="#SEC2">DESCRIPTION</a> <li><a name="TOC3" href="#SEC3">OPTIONS</a> <li><a name="TOC4" href="#SEC4">LONG OPTIONS</a> -<li><a name="TOC5" href="#SEC5">DIAGNOSTICS</a> -<li><a name="TOC6" href="#SEC6">AUTHOR</a> +<li><a name="TOC5" href="#SEC5">OPTIONS WITH DATA</a> +<li><a name="TOC6" href="#SEC6">DIAGNOSTICS</a> +<li><a name="TOC7" href="#SEC7">AUTHOR</a> </ul> <br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> <P> -<b>pcregrep [-Vcfhilnrsuvx] [long options] [pattern] [file1 file2 ...]</b> +<b>pcregrep [options] [long options] [pattern] [file1 file2 ...]</b> </P> <br><a name="SEC2" href="#TOC1">DESCRIPTION</a><br> <P> @@ -38,21 +39,50 @@ A pattern must be specified on the command line unless the <b>-f</b> option is used (see below). </P> <P> -If no files are specified, <b>pcregrep</b> reads the standard input. By default, -each line that matches the pattern is copied to the standard output, and if -there is more than one file, the file name is printed before each line of -output. However, there are options that can change how <b>pcregrep</b> behaves. +If no files are specified, <b>pcregrep</b> reads the standard input. The +standard input can also be referenced by a name consisting of a single hyphen. +For example: +<pre> + pcregrep some-pattern /file1 - /file3 +</pre> +By default, each line that matches the pattern is copied to the standard +output, and if there is more than one file, the file name is printed before +each line of output. However, there are options that can change how +<b>pcregrep</b> behaves. In particular, the <b>-M</b> option makes it possible to +search for patterns that span line boundaries. </P> <P> -Lines are limited to BUFSIZ characters. BUFSIZ is defined in <b><stdio.h></b>. -The newline character is removed from the end of each line before it is matched -against the pattern. +Patterns are limited to 8K or BUFSIZ characters, whichever is the greater. +BUFSIZ is defined in <b><stdio.h></b>. </P> <br><a name="SEC3" href="#TOC1">OPTIONS</a><br> <P> -<b>-V</b> -Write the version number of the PCRE library being used to the standard error -stream. +<b>--</b> +This terminate the list of options. It is useful if the next item on the +command line starts with a hyphen, but is not an option. +</P> +<P> +<b>-A</b> <i>number</i> +Print <i>number</i> lines of context after each matching line. If file names +and/or line numbers are being printed, a hyphen separator is used instead of a +colon for the context lines. A line containing "--" is printed between each +group of lines, unless they are in fact contiguous in the input file. The value +of <i>number</i> is expected to be relatively small. However, <b>pcregrep</b> +guarantees to have up to 8K of following text available for context printing. +</P> +<P> +<b>-B</b> <i>number</i> +Print <i>number</i> lines of context before each matching line. If file names +and/or line numbers are being printed, a hyphen separator is used instead of a +colon for the context lines. A line containing "--" is printed between each +group of lines, unless they are in fact contiguous in the input file. The value +of <i>number</i> is expected to be relatively small. However, <b>pcregrep</b> +guarantees to have up to 8K of preceding text available for context printing. +</P> +<P> +<b>-C</b> <i>number</i> +Print <i>number</i> lines of context both before and after each matching line. +This is equivalent to setting both <b>-A</b> and <b>-B</b> to the same value. </P> <P> <b>-c</b> @@ -61,6 +91,14 @@ lines that would otherwise have been printed. If several files are given, a count is printed for each of them. </P> <P> +<b>--exclude</b>=<i>pattern</i> +When <b>pcregrep</b> is searching the files in a directory as a consequence of +the <b>-r</b> (recursive search) option, any files whose names match the pattern +are excluded. The pattern is a PCRE regular expression. If a file name matches +both <b>--include</b> and <b>--exclude</b>, it is excluded. There is no short +form for this option. +</P> +<P> <b>-f</b><i>filename</i> Read a number of patterns from the file, one per line, and match all of them against each line of input. A line is output if any of the patterns match it. @@ -78,24 +116,64 @@ Suppress printing of filenames when searching multiple files. Ignore upper/lower case distinctions during comparisons. </P> <P> +<b>--include</b>=<i>pattern</i> +When <b>pcregrep</b> is searching the files in a directory as a consequence of +the <b>-r</b> (recursive search) option, only files whose names match the +pattern are included. The pattern is a PCRE regular expression. If a file name +matches both <b>--include</b> and <b>--exclude</b>, it is excluded. There is no +short form for this option. +</P> +<P> +<b>-L</b> +Instead of printing lines from the files, just print the names of the files +that do not contain any lines that would have been printed. Each file name is +printed once, on a separate line. +</P> +<P> <b>-l</b> Instead of printing lines from the files, just print the names of the files containing lines that would have been printed. Each file name is printed once, on a separate line. </P> <P> +<b>--label</b>=<i>name</i> +This option supplies a name to be used for the standard input when file names +are being printed. If not supplied, "(standard input)" is used. There is no +short form for this option. +</P> +<P> +<b>-M</b> +Allow patterns to match more than one line. When this option is given, patterns +may usefully contain literal newline characters and internal occurrences of ^ +and $ characters. The output for any one match may consist of more than one +line. When this option is set, the PCRE library is called in "multiline" mode. +There is a limit to the number of lines that can be matched, imposed by the way +that <b>pcregrep</b> buffers the input file as it scans it. However, +<b>pcregrep</b> ensures that at least 8K characters or the rest of the document +(whichever is the shorter) are available for forward matching, and similarly +the previous 8K characters (or all the previous characters, if fewer than 8K) +are guaranteed to be available for lookbehind assertions. +</P> +<P> <b>-n</b> Precede each line by its line number in the file. </P> <P> +<b>-q</b> +Work quietly, that is, display nothing except error messages. +The exit status indicates whether or not any matches were found. +</P> +<P> <b>-r</b> -If any file is a directory, recursively scan the files it contains. Without +If any given path is a directory, recursively scan the files it contains, +taking note of any <b>--include</b> and <b>--exclude</b> settings. Without <b>-r</b> a directory is scanned as a normal file. </P> <P> <b>-s</b> -Work silently, that is, display nothing except error messages. -The exit status indicates whether any matches were found. +Suppress error messages about non-existent or unreadable files. Such files are +quietly skipped. However, the return code is still 2, even if matches were +found in other files. </P> <P> <b>-u</b> @@ -104,9 +182,19 @@ with UTF-8 support. Both the pattern and each subject line must be valid strings of UTF-8 characters. </P> <P> +<b>-V</b> +Write the version numbers of <b>pcregrep</b> and the PCRE library that is being +used to the standard error stream. +</P> +<P> <b>-v</b> Invert the sense of the match, so that lines which do <i>not</i> match the -pattern are now the ones that are found. +pattern are the ones that are found. +</P> +<P> +<b>-w</b> +Force the pattern to match only whole words. This is equivalent to having \b +at the start and end of the pattern. </P> <P> <b>-x</b> @@ -120,39 +208,67 @@ alternative branch in the regular expression. Long forms of all the options are available, as in GNU grep. They are shown in the following table: <pre> + -A --after-context + -B --before-context + -C --context -c --count + --exclude (no short form) + -f --file -h --no-filename + --help (no short form) -i --ignore-case + --include (no short form) + -L --files-without-match -l --files-with-matches + --label (no short form) -n --line-number -r --recursive + -q --quiet -s --no-messages -u --utf-8 -V --version -v --invert-match -x --line-regex -x --line-regexp +</PRE> +</P> +<br><a name="SEC5" href="#TOC1">OPTIONS WITH DATA</a><br> +<P> +There are four different ways in which an option with data can be specified. +If a short form option is used, the data may follow immediately, or in the next +command line item. For example: +<pre> + -f/some/file + -f /some/file </pre> -In addition, --file=<i>filename</i> is equivalent to -f<i>filename</i>, and ---help shows the list of options and then exits. +If a long form option is used, the data may appear in the same command line +item, separated by an = character, or it may appear in the next command line +item. For example: +<pre> + --file=/some/file + --file /some/file + +</PRE> </P> -<br><a name="SEC5" href="#TOC1">DIAGNOSTICS</a><br> +<br><a name="SEC6" href="#TOC1">DIAGNOSTICS</a><br> <P> Exit status is 0 if any matches were found, 1 if no matches were found, and 2 -for syntax errors or inacessible files (even if matches were found). +for syntax errors and non-existent or inacessible files (even if matches were +found in other files). Using the <b>-s</b> option to suppress error messages +about inaccessble files does not affect the return code. </P> -<br><a name="SEC6" href="#TOC1">AUTHOR</a><br> +<br><a name="SEC7" href="#TOC1">AUTHOR</a><br> <P> -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel <br> University Computing Service <br> Cambridge CB2 3QG, England. </P> <P> -Last updated: 09 September 2004 +Last updated: 16 May 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcrematching.html b/doc/html/pcrematching.html new file mode 100644 index 0000000..eb381b7 --- /dev/null +++ b/doc/html/pcrematching.html @@ -0,0 +1,192 @@ +<html> +<head> +<title>pcrematching specification</title> +</head> +<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> +<h1>pcrematching man page</h1> +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> +<p> +This page is part of the PCRE HTML documentation. It was generated automatically +from the original man page. If there is any nonsense in it, please consult the +man page, in case the conversion went wrong. +<br> +<ul> +<li><a name="TOC1" href="#SEC1">PCRE MATCHING ALGORITHMS</a> +<li><a name="TOC2" href="#SEC2">REGULAR EXPRESSIONS AS TREES</a> +<li><a name="TOC3" href="#SEC3">THE STANDARD MATCHING ALGORITHM</a> +<li><a name="TOC4" href="#SEC4">THE DFA MATCHING ALGORITHM</a> +<li><a name="TOC5" href="#SEC5">ADVANTAGES OF THE DFA ALGORITHM</a> +<li><a name="TOC6" href="#SEC6">DISADVANTAGES OF THE DFA ALGORITHM</a> +</ul> +<br><a name="SEC1" href="#TOC1">PCRE MATCHING ALGORITHMS</a><br> +<P> +This document describes the two different algorithms that are available in PCRE +for matching a compiled regular expression against a given subject string. The +"standard" algorithm is the one provided by the <b>pcre_exec()</b> function. +This works in the same was as Perl's matching function, and provides a +Perl-compatible matching operation. +</P> +<P> +An alternative algorithm is provided by the <b>pcre_dfa_exec()</b> function; +this operates in a different way, and is not Perl-compatible. It has advantages +and disadvantages compared with the standard algorithm, and these are described +below. +</P> +<P> +When there is only one possible way in which a given subject string can match a +pattern, the two algorithms give the same answer. A difference arises, however, +when there are multiple possibilities. For example, if the pattern +<pre> + ^<.*> +</pre> +is matched against the string +<pre> + <something> <something else> <something further> +</pre> +there are three possible answers. The standard algorithm finds only one of +them, whereas the DFA algorithm finds all three. +</P> +<br><a name="SEC2" href="#TOC1">REGULAR EXPRESSIONS AS TREES</a><br> +<P> +The set of strings that are matched by a regular expression can be represented +as a tree structure. An unlimited repetition in the pattern makes the tree of +infinite size, but it is still a tree. Matching the pattern to a given subject +string (from a given starting point) can be thought of as a search of the tree. +There are two standard ways to search a tree: depth-first and breadth-first, +and these correspond to the two matching algorithms provided by PCRE. +</P> +<br><a name="SEC3" href="#TOC1">THE STANDARD MATCHING ALGORITHM</a><br> +<P> +In the terminology of Jeffrey Friedl's book \fIMastering Regular +Expressions\fP, the standard algorithm is an "NFA algorithm". It conducts a +depth-first search of the pattern tree. That is, it proceeds along a single +path through the tree, checking that the subject matches what is required. When +there is a mismatch, the algorithm tries any alternatives at the current point, +and if they all fail, it backs up to the previous branch point in the tree, and +tries the next alternative branch at that level. This often involves backing up +(moving to the left) in the subject string as well. The order in which +repetition branches are tried is controlled by the greedy or ungreedy nature of +the quantifier. +</P> +<P> +If a leaf node is reached, a matching string has been found, and at that point +the algorithm stops. Thus, if there is more than one possible match, this +algorithm returns the first one that it finds. Whether this is the shortest, +the longest, or some intermediate length depends on the way the greedy and +ungreedy repetition quantifiers are specified in the pattern. +</P> +<P> +Because it ends up with a single path through the tree, it is relatively +straightforward for this algorithm to keep track of the substrings that are +matched by portions of the pattern in parentheses. This provides support for +capturing parentheses and back references. +</P> +<br><a name="SEC4" href="#TOC1">THE DFA MATCHING ALGORITHM</a><br> +<P> +DFA stands for "deterministic finite automaton", but you do not need to +understand the origins of that name. This algorithm conducts a breadth-first +search of the tree. Starting from the first matching point in the subject, it +scans the subject string from left to right, once, character by character, and +as it does this, it remembers all the paths through the tree that represent +valid matches. +</P> +<P> +The scan continues until either the end of the subject is reached, or there are +no more unterminated paths. At this point, terminated paths represent the +different matching possibilities (if there are none, the match has failed). +Thus, if there is more than one possible match, this algorithm finds all of +them, and in particular, it finds the longest. In PCRE, there is an option to +stop the algorithm after the first match (which is necessarily the shortest) +has been found. +</P> +<P> +Note that all the matches that are found start at the same point in the +subject. If the pattern +<pre> + cat(er(pillar)?) +</pre> +is matched against the string "the caterpillar catchment", the result will be +the three strings "cat", "cater", and "caterpillar" that start at the fourth +character of the subject. The algorithm does not automatically move on to find +matches that start at later positions. +</P> +<P> +There are a number of features of PCRE regular expressions that are not +supported by the DFA matching algorithm. They are as follows: +</P> +<P> +1. Because the algorithm finds all possible matches, the greedy or ungreedy +nature of repetition quantifiers is not relevant. Greedy and ungreedy +quantifiers are treated in exactly the same way. +</P> +<P> +2. When dealing with multiple paths through the tree simultaneously, it is not +straightforward to keep track of captured substrings for the different matching +possibilities, and PCRE's implementation of this algorithm does not attempt to +do this. This means that no captured substrings are available. +</P> +<P> +3. Because no substrings are captured, back references within the pattern are +not supported, and cause errors if encountered. +</P> +<P> +4. For the same reason, conditional expressions that use a backreference as the +condition are not supported. +</P> +<P> +5. Callouts are supported, but the value of the <i>capture_top</i> field is +always 1, and the value of the <i>capture_last</i> field is always -1. +</P> +<P> +6. +The \C escape sequence, which (in the standard algorithm) matches a single +byte, even in UTF-8 mode, is not supported because the DFA algorithm moves +through the subject string one character at a time, for all active paths +through the tree. +</P> +<br><a name="SEC5" href="#TOC1">ADVANTAGES OF THE DFA ALGORITHM</a><br> +<P> +Using the DFA matching algorithm provides the following advantages: +</P> +<P> +1. All possible matches (at a single point in the subject) are automatically +found, and in particular, the longest match is found. To find more than one +match using the standard algorithm, you have to do kludgy things with +callouts. +</P> +<P> +2. There is much better support for partial matching. The restrictions on the +content of the pattern that apply when using the standard algorithm for partial +matching do not apply to the DFA algorithm. For non-anchored patterns, the +starting position of a partial match is available. +</P> +<P> +3. Because the DFA algorithm scans the subject string just once, and never +needs to backtrack, it is possible to pass very long subject strings to the +matching function in several pieces, checking for partial matching each time. +</P> +<br><a name="SEC6" href="#TOC1">DISADVANTAGES OF THE DFA ALGORITHM</a><br> +<P> +The DFA algorithm suffers from a number of disadvantages: +</P> +<P> +1. It is substantially slower than the standard algorithm. This is partly +because it has to search for all possible matches, but is also because it is +less susceptible to optimization. +</P> +<P> +2. Capturing parentheses and back references are not supported. +</P> +<P> +3. The "atomic group" feature of PCRE regular expressions is supported, but +does not provide the advantage that it does for the standard algorithm. +</P> +<P> +Last updated: 28 February 2005 +<br> +Copyright © 1997-2005 University of Cambridge. +<p> +Return to the <a href="index.html">PCRE index page</a>. +</p> diff --git a/doc/html/pcrepartial.html b/doc/html/pcrepartial.html index c4dd886..0dbfc02 100644 --- a/doc/html/pcrepartial.html +++ b/doc/html/pcrepartial.html @@ -16,14 +16,15 @@ man page, in case the conversion went wrong. <li><a name="TOC1" href="#SEC1">PARTIAL MATCHING IN PCRE</a> <li><a name="TOC2" href="#SEC2">RESTRICTED PATTERNS FOR PCRE_PARTIAL</a> <li><a name="TOC3" href="#SEC3">EXAMPLE OF PARTIAL MATCHING USING PCRETEST</a> +<li><a name="TOC4" href="#SEC4">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec()</a> </ul> <br><a name="SEC1" href="#TOC1">PARTIAL MATCHING IN PCRE</a><br> <P> In normal use of PCRE, if the subject string that is passed to -<b>pcre_exec()</b> matches as far as it goes, but is too short to match the -entire pattern, PCRE_ERROR_NOMATCH is returned. There are circumstances where -it might be helpful to distinguish this case from other cases in which there is -no match. +<b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> matches as far as it goes, but is +too short to match the entire pattern, PCRE_ERROR_NOMATCH is returned. There +are circumstances where it might be helpful to distinguish this case from other +cases in which there is no match. </P> <P> Consider, for example, an application where a human is required to type in data @@ -41,10 +42,20 @@ entered. </P> <P> PCRE supports the concept of partial matching by means of the PCRE_PARTIAL -option, which can be set when calling <b>pcre_exec()</b>. When this is done, the -return code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if at any -time during the matching process the entire subject string matched part of the -pattern. No captured data is set when this occurs. +option, which can be set when calling <b>pcre_exec()</b> or +<b>pcre_dfa_exec()</b>. When this flag is set for <b>pcre_exec()</b>, the return +code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if at any time +during the matching process the last part of the subject string matched part of +the pattern. Unfortunately, for non-anchored matching, it is not possible to +obtain the position of the start of the partial match. No captured data is set +when PCRE_ERROR_PARTIAL is returned. +</P> +<P> +When PCRE_PARTIAL is set for <b>pcre_dfa_exec()</b>, the return code +PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end of the +subject is reached, there have been no complete matches, but there is still at +least one matching possibility. The portion of the string that provided the +partial match is set as the first matching string. </P> <P> Using PCRE_PARTIAL disables one of PCRE's optimizations. PCRE remembers the @@ -54,9 +65,10 @@ for a subject string that might match only partially. </P> <br><a name="SEC2" href="#TOC1">RESTRICTED PATTERNS FOR PCRE_PARTIAL</a><br> <P> -Because of the way certain internal optimizations are implemented in PCRE, the -PCRE_PARTIAL option cannot be used with all patterns. Repeated single -characters such as +Because of the way certain internal optimizations are implemented in the +<b>pcre_exec()</b> function, the PCRE_PARTIAL option cannot be used with all +patterns. These restrictions do not apply when <b>pcre_dfa_exec()</b> is used. +For <b>pcre_exec()</b>, repeated single characters such as <pre> a{2,4} </pre> @@ -100,12 +112,94 @@ uses the date example quoted above: </pre> The first data string is matched completely, so <b>pcretest</b> shows the matched substrings. The remaining four strings do not match the complete -pattern, but the first two are partial matches. +pattern, but the first two are partial matches. The same test, using DFA +matching (by means of the \D escape sequence), produces the following output: +<pre> + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 25jun04\P\D + 0: 25jun04 + data> 23dec3\P\D + Partial match: 23dec3 + data> 3ju\P\D + Partial match: 3ju + data> 3juj\P\D + No match + data> j\P\D + No match +</pre> +Notice that in this case the portion of the string that was matched is made +available. +</P> +<br><a name="SEC4" href="#TOC1">MULTI-SEGMENT MATCHING WITH pcre_dfa_exec()</a><br> +<P> +When a partial match has been found using <b>pcre_dfa_exec()</b>, it is possible +to continue the match by providing additional subject data and calling +<b>pcre_dfa_exec()</b> again with the PCRE_DFA_RESTART option and the same +working space (where details of the previous partial match are stored). Here is +an example using <b>pcretest</b>, where the \R escape sequence sets the +PCRE_DFA_RESTART option and the \D escape sequence requests the use of +<b>pcre_dfa_exec()</b>: +<pre> + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 23ja\P\D + Partial match: 23ja + data> n05\R\D + 0: n05 +</pre> +The first call has "23ja" as the subject, and requests partial matching; the +second call has "n05" as the subject for the continued (restarted) match. +Notice that when the match is complete, only the last part is shown; PCRE does +not retain the previously partially-matched string. It is up to the calling +program to do that if it needs to. +</P> +<P> +This facility can be used to pass very long subject strings to +<b>pcre_dfa_exec()</b>. However, some care is needed for certain types of +pattern. +</P> +<P> +1. If the pattern contains tests for the beginning or end of a line, you need +to pass the PCRE_NOTBOL or PCRE_NOTEOL options, as appropriate, when the +subject string for any call does not contain the beginning or end of a line. +</P> +<P> +2. If the pattern contains backward assertions (including \b or \B), you need +to arrange for some overlap in the subject strings to allow for this. For +example, you could pass the subject in chunks that were 500 bytes long, but in +a buffer of 700 bytes, with the starting offset set to 200 and the previous 200 +bytes at the start of the buffer. +</P> +<P> +3. Matching a subject string that is split into multiple segments does not +always produce exactly the same result as matching over one single long string. +The difference arises when there are multiple matching possibilities, because a +partial match result is given only when there are no completed matches in a +call to fBpcre_dfa_exec()\fP. This means that as soon as the shortest match has +been found, continuation to a new subject segment is no longer possible. +Consider this <b>pcretest</b> example: +<pre> + re> /dog(sbody)?/ + data> do\P\D + Partial match: do + data> gsb\R\P\D + 0: g + data> dogsbody\D + 0: dogsbody + 1: dog +</pre> +The pattern matches the words "dog" or "dogsbody". When the subject is +presented in several parts ("do" and "gsb" being the first two) the match stops +when "dog" has been found, and it is not possible to continue. On the other +hand, if "dogsbody" is presented as a single string, both matches are found. +</P> +<P> +Because of this phenomenon, it does not usually make sense to end a pattern +that is going to be matched in this way with a variable repeat. </P> <P> -Last updated: 08 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcrepattern.html b/doc/html/pcrepattern.html index 1220eb7..0f77b32 100644 --- a/doc/html/pcrepattern.html +++ b/doc/html/pcrepattern.html @@ -55,15 +55,35 @@ in the main page. </P> <P> +The remainder of this document discusses the patterns that are supported by +PCRE when its main matching function, <b>pcre_exec()</b>, is used. +From release 6.0, PCRE offers a second matching function, +<b>pcre_dfa_exec()</b>, which matches using a different algorithm that is not +Perl-compatible. The advantages and disadvantages of the alternative function, +and how it differs from the normal function, are discussed in the +<a href="pcrematching.html"><b>pcrematching</b></a> +page. +</P> +<P> A regular expression is a pattern that is matched against a subject string from left to right. Most characters stand for themselves in a pattern, and match the corresponding characters in the subject. As a trivial example, the pattern <pre> The quick brown fox </pre> -matches a portion of a subject string that is identical to itself. The power of -regular expressions comes from the ability to include alternatives and -repetitions in the pattern. These are encoded in the pattern by the use of +matches a portion of a subject string that is identical to itself. When +caseless matching is specified (the PCRE_CASELESS option), letters are matched +independently of case. In UTF-8 mode, PCRE always understands the concept of +case for characters whose values are less than 128, so caseless matching is +always possible. For characters with higher values, the concept of case is +supported if PCRE is compiled with Unicode property support, but not otherwise. +If you want to use caseless matching for characters 128 and above, you must +ensure that PCRE is compiled with Unicode property support as well as with +UTF-8 support. +</P> +<P> +The power of regular expressions comes from the ability to include alternatives +and repetitions in the pattern. These are encoded in the pattern by the use of <i>metacharacters</i>, which do not stand for themselves but instead are interpreted in some special way. </P> @@ -536,9 +556,13 @@ class as a literal string of bytes, or by using the \x{ escaping mechanism. When caseless matching is set, any letters in a class represent both their upper case and lower case versions, so for example, a caseless [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a -caseful version would. When running in UTF-8 mode, PCRE supports the concept of -case for characters with values greater than 128 only when it is compiled with -Unicode property support. +caseful version would. In UTF-8 mode, PCRE always understands the concept of +case for characters whose values are less than 128, so caseless matching is +always possible. For characters with higher values, the concept of case is +supported if PCRE is compiled with Unicode property support, but not otherwise. +If you want to use caseless matching for characters 128 and above, you must +ensure that PCRE is compiled with Unicode property support as well as with +UTF-8 support. </P> <P> The newline character is never treated in any special way in character classes, @@ -1462,9 +1486,9 @@ description of the interface to the callout function is given in the documentation. </P> <P> -Last updated: 09 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcreperform.html b/doc/html/pcreperform.html index f0ffa68..e97e748 100644 --- a/doc/html/pcreperform.html +++ b/doc/html/pcreperform.html @@ -50,8 +50,8 @@ this, PCRE has to retry the match starting after every newline in the subject. <P> If you are using such a pattern with subject strings that do not contain newlines, the best performance is obtained by setting PCRE_DOTALL, or starting -the pattern with ^.* to indicate explicit anchoring. That saves PCRE from -having to scan along the subject looking for a newline to restart at. +the pattern with ^.* or ^.*? to indicate explicit anchoring. That saves PCRE +from having to scan along the subject looking for a newline to restart at. </P> <P> Beware of patterns that contain nested indefinite repeats. These can take a @@ -89,9 +89,9 @@ In many cases, the solution to this kind of performance issue is to use an atomic group or a possessive quantifier. </P> <P> -Last updated: 09 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcreposix.html b/doc/html/pcreposix.html index 2f7aaa9..53ea2aa 100644 --- a/doc/html/pcreposix.html +++ b/doc/html/pcreposix.html @@ -46,8 +46,8 @@ man page, in case the conversion went wrong. This set of functions provides a POSIX-style API to the PCRE regular expression package. See the <a href="pcreapi.html"><b>pcreapi</b></a> -documentation for a description of PCRE's native API, which contains additional -functionality. +documentation for a description of PCRE's native API, which contains much +additional functionality. </P> <P> The functions described here are just wrapper functions that ultimately call @@ -95,6 +95,11 @@ about the compiled expression. The argument <i>cflags</i> is either zero, or contains one or more of the bits defined by the following macros: <pre> + REG_DOTALL +</pre> +The PCRE_DOTALL option is set when the expression is passed for compilation to +the native function. Note that REG_DOTALL is not part of the POSIX standard. +<pre> REG_ICASE </pre> The PCRE_CASELESS option is set when the expression is passed for compilation @@ -203,16 +208,16 @@ memory, after which <i>preg</i> may no longer be used as a compiled expression. </P> <br><a name="SEC8" href="#TOC1">AUTHOR</a><br> <P> -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel <br> University Computing Service, <br> Cambridge CB2 3QG, England. </P> <P> -Last updated: 07 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcreprecompile.html b/doc/html/pcreprecompile.html index f1c109e..10be496 100644 --- a/doc/html/pcreprecompile.html +++ b/doc/html/pcreprecompile.html @@ -88,16 +88,17 @@ return a non-NULL value before trying to save the study data. <br><a name="SEC3" href="#TOC1">RE-USING A PRECOMPILED PATTERN</a><br> <P> Re-using a precompiled pattern is straightforward. Having reloaded it into main -memory, you pass its pointer to <b>pcre_exec()</b> in the usual way. This should -work even on another host, and even if that host has the opposite endianness to -the one where the pattern was compiled. +memory, you pass its pointer to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> in +the usual way. This should work even on another host, and even if that host has +the opposite endianness to the one where the pattern was compiled. </P> <P> However, if you passed a pointer to custom character tables when the pattern was compiled (the <i>tableptr</i> argument of <b>pcre_compile()</b>), you must -now pass a similar pointer to <b>pcre_exec()</b>, because the value saved with -the compiled pattern will obviously be nonsense. A field in a -<b>pcre_extra()</b> block is used to pass this data, as described in the +now pass a similar pointer to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b>, +because the value saved with the compiled pattern will obviously be nonsense. A +field in a <b>pcre_extra()</b> block is used to pass this data, as described in +the <a href="pcreapi.html#extradata">section on matching a pattern</a> in the <a href="pcreapi.html"><b>pcreapi</b></a> @@ -114,7 +115,8 @@ If you saved study data with the compiled pattern, you need to create your own <b>pcre_extra</b> data block and set the <i>study_data</i> field to point to the reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in the <i>flags</i> field to indicate that study data is present. Then pass the -<b>pcre_extra</b> block to <b>pcre_exec()</b> in the usual way. +<b>pcre_extra</b> block to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> in the +usual way. </P> <br><a name="SEC4" href="#TOC1">COMPATIBILITY WITH DIFFERENT PCRE RELEASES</a><br> <P> @@ -125,9 +127,9 @@ advertised), you will have to recompile them for release 5.0. However, from now on, it should be possible to make changes in a compabible manner. </P> <P> -Last updated: 10 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> diff --git a/doc/html/pcretest.html b/doc/html/pcretest.html index d82dfcc..c43c8cb 100644 --- a/doc/html/pcretest.html +++ b/doc/html/pcretest.html @@ -18,14 +18,17 @@ man page, in case the conversion went wrong. <li><a name="TOC3" href="#SEC3">DESCRIPTION</a> <li><a name="TOC4" href="#SEC4">PATTERN MODIFIERS</a> <li><a name="TOC5" href="#SEC5">DATA LINES</a> -<li><a name="TOC6" href="#SEC6">OUTPUT FROM PCRETEST</a> -<li><a name="TOC7" href="#SEC7">CALLOUTS</a> -<li><a name="TOC8" href="#SEC8">SAVING AND RELOADING COMPILED PATTERNS</a> -<li><a name="TOC9" href="#SEC9">AUTHOR</a> +<li><a name="TOC6" href="#SEC6">THE ALTERNATIVE MATCHING FUNCTION</a> +<li><a name="TOC7" href="#SEC7">DEFAULT OUTPUT FROM PCRETEST</a> +<li><a name="TOC8" href="#SEC8">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a> +<li><a name="TOC9" href="#SEC9">RESTARTING AFTER A PARTIAL MATCH</a> +<li><a name="TOC10" href="#SEC10">CALLOUTS</a> +<li><a name="TOC11" href="#SEC11">SAVING AND RELOADING COMPILED PATTERNS</a> +<li><a name="TOC12" href="#SEC12">AUTHOR</a> </ul> <br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br> <P> -<b>pcretest [-C] [-d] [-i] [-m] [-o osize] [-p] [-t] [source]</b> +<b>pcretest [-C] [-d] [-dfa] [-i] [-m] [-o osize] [-p] [-t] [source]</b> <b>[destination]</b> </P> <P> @@ -47,12 +50,18 @@ about the optional features that are included, and then exit. </P> <P> <b>-d</b> -Behave as if each regex had the <b>/D</b> (debug) modifier; the internal +Behave as if each regex has the <b>/D</b> (debug) modifier; the internal form is output after compilation. </P> <P> +<b>-dfa</b> +Behave as if each data line contains the \D escape sequence; this causes the +alternative matching function, <b>pcre_dfa_exec()</b>, to be used instead of the +standard <b>pcre_exec()</b> function (more detail is given below). +</P> +<P> <b>-i</b> -Behave as if each regex had the <b>/I</b> modifier; information about the +Behave as if each regex has the <b>/I</b> modifier; information about the compiled pattern is given after compilation. </P> <P> @@ -70,8 +79,9 @@ matching calls by including \O in the data line (see below). </P> <P> <b>-p</b> -Behave as if each regex has <b>/P</b> modifier; the POSIX wrapper API is used -to call PCRE. None of the other options has any effect when <b>-p</b> is set. +Behave as if each regex has the <b>/P</b> modifier; the POSIX wrapper API is +used to call PCRE. None of the other options has any effect when <b>-p</b> is +set. </P> <P> <b>-t</b> @@ -152,6 +162,7 @@ not correspond to anything in Perl: <b>/A</b> PCRE_ANCHORED <b>/C</b> PCRE_AUTO_CALLOUT <b>/E</b> PCRE_DOLLAR_ENDONLY + <b>/f</b> PCRE_FIRSTLINE <b>/N</b> PCRE_NO_AUTO_CAPTURE <b>/U</b> PCRE_UNGREEDY <b>/X</b> PCRE_EXTRA @@ -274,6 +285,8 @@ recognized: \C!n return 1 instead of 0 when callout number n is reached \C!n!m return 1 instead of 0 when callout number n is reached for the nth time \C*n pass the number n (may be negative) as callout data; this is used as the callout return value + \D use the <b>pcre_dfa_exec()</b> match function + \F only shortest match for <b>pcre_dfa_exec()</b> \Gdd call pcre_get_substring() for substring dd after a successful match (number less than 32) \Gname call pcre_get_named_substring() for substring "name" after a successful match (name termin- ated by next non-alphanumeric character) @@ -281,7 +294,8 @@ recognized: \M discover the minimum MATCH_LIMIT setting \N pass the PCRE_NOTEMPTY option to <b>pcre_exec()</b> \Odd set the size of the output vector passed to <b>pcre_exec()</b> to dd (any number of digits) - \P pass the PCRE_PARTIAL option to <b>pcre_exec()</b> + \P pass the PCRE_PARTIAL option to <b>pcre_exec()</b> or <b>pcre_dfa_exec()</b> + \R pass the PCRE_DFA_RESTART option to <b>pcre_dfa_exec()</b> \S output details of memory get/free calls during matching \Z pass the PCRE_NOTEOL option to <b>pcre_exec()</b> \? pass the PCRE_NO_UTF8_CHECK option to <b>pcre_exec()</b> @@ -318,14 +332,35 @@ of the <b>/8</b> modifier on the pattern. It is recognized always. There may be any number of hexadecimal digits inside the braces. The result is from one to six bytes, encoded according to the UTF-8 rules. </P> -<br><a name="SEC6" href="#TOC1">OUTPUT FROM PCRETEST</a><br> +<br><a name="SEC6" href="#TOC1">THE ALTERNATIVE MATCHING FUNCTION</a><br> +<P> +By default, <b>pcretest</b> uses the standard PCRE matching function, +<b>pcre_exec()</b> to match each data line. From release 6.0, PCRE supports an +alternative matching function, <b>pcre_dfa_test()</b>, which operates in a +different way, and has some restrictions. The differences between the two +functions are described in the +<a href="pcrematching.html"><b>pcrematching</b></a> +documentation. +</P> +<P> +If a data line contains the \D escape sequence, or if the command line +contains the <b>-dfa</b> option, the alternative matching function is called. +This function finds all possible matches at a given point. If, however, the \F +escape sequence is present in the data line, it stops after the first match is +found. This is always the shortest possible match. +</P> +<br><a name="SEC7" href="#TOC1">DEFAULT OUTPUT FROM PCRETEST</a><br> +<P> +This section describes the output when the normal matching function, +<b>pcre_exec()</b>, is being used. +</P> <P> When a match succeeds, pcretest outputs the list of captured substrings that <b>pcre_exec()</b> returns, starting with number 0 for the string that matched the whole pattern. Otherwise, it outputs "No match" or "Partial match" when <b>pcre_exec()</b> returns PCRE_ERROR_NOMATCH or PCRE_ERROR_PARTIAL, respectively, and otherwise the PCRE negative error number. Here is an example -of an interactive pcretest run. +of an interactive <b>pcretest</b> run. <pre> $ pcretest PCRE version 5.00 07-Sep-2004 @@ -375,12 +410,62 @@ Note that while patterns can be continued over several lines (a plain ">" prompt is used for continuations), data lines may not. However newlines can be included in data by means of the \n escape. </P> -<br><a name="SEC7" href="#TOC1">CALLOUTS</a><br> +<br><a name="SEC8" href="#TOC1">OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION</a><br> +<P> +When the alternative matching function, <b>pcre_dfa_exec()</b>, is used (by +means of the \D escape sequence or the <b>-dfa</b> command line option), the +output consists of a list of all the matches that start at the first point in +the subject where there is at least one match. For example: +<pre> + re> /(tang|tangerine|tan)/ + data> yellow tangerine\D + 0: tangerine + 1: tang + 2: tan +</pre> +(Using the normal matching function on this data finds only "tang".) The +longest matching string is always given first (and numbered zero). +</P> +<P> +If \fB/g\P is present on the pattern, the search for further matches resumes +at the end of the longest match. For example: +<pre> + re> /(tang|tangerine|tan)/g + data> yellow tangerine and tangy sultana\D + 0: tangerine + 1: tang + 2: tan + 0: tang + 1: tan + 0: tan +</pre> +Since the matching function does not support substring capture, the escape +sequences that are concerned with captured substrings are not relevant. +</P> +<br><a name="SEC9" href="#TOC1">RESTARTING AFTER A PARTIAL MATCH</a><br> +<P> +When the alternative matching function has given the PCRE_ERROR_PARTIAL return, +indicating that the subject partially matched the pattern, you can restart the +match with additional subject data by means of the \R escape sequence. For +example: +<pre> + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 23ja\P\D + Partial match: 23ja + data> n05\R\D + 0: n05 +</pre> +For further information about partial matching, see the +<a href="pcrepartial.html"><b>pcrepartial</b></a> +documentation. +</P> +<br><a name="SEC10" href="#TOC1">CALLOUTS</a><br> <P> If the pattern contains any callout requests, <b>pcretest</b>'s callout function -is called during matching. By default, it displays the callout number, the -start and current positions in the text at the callout time, and the next -pattern item to be tested. For example, the output +is called during matching. This works with both matching functions. By default, +the called function displays the callout number, the start and current +positions in the text at the callout time, and the next pattern item to be +tested. For example, the output <pre> --->pqrabcdef 0 ^ ^ \d @@ -406,7 +491,7 @@ example: 0: E* </pre> The callout function in <b>pcretest</b> returns zero (carry on matching) by -default, but you can use an \C item in a data line (as described above) to +default, but you can use a \C item in a data line (as described above) to change this. </P> <P> @@ -416,7 +501,7 @@ the <a href="pcrecallout.html"><b>pcrecallout</b></a> documentation. </P> -<br><a name="SEC8" href="#TOC1">SAVING AND RELOADING COMPILED PATTERNS</a><br> +<br><a name="SEC11" href="#TOC1">SAVING AND RELOADING COMPILED PATTERNS</a><br> <P> The facilities described in this section are not available when the POSIX inteface to PCRE is being used, that is, when the <b>/P</b> pattern modifier is @@ -478,18 +563,18 @@ string using a reloaded pattern is likely to cause <b>pcretest</b> to crash. Finally, if you attempt to load a file that is not in the correct format, the result is undefined. </P> -<br><a name="SEC9" href="#TOC1">AUTHOR</a><br> +<br><a name="SEC12" href="#TOC1">AUTHOR</a><br> <P> -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel <br> University Computing Service, <br> Cambridge CB2 3QG, England. </P> <P> -Last updated: 10 September 2004 +Last updated: 28 February 2005 <br> -Copyright © 1997-2004 University of Cambridge. +Copyright © 1997-2005 University of Cambridge. <p> Return to the <a href="index.html">PCRE index page</a>. </p> @@ -6,15 +6,29 @@ PCRE - Perl-compatible regular expressions .sp The PCRE library is a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl, with just a few -differences. The current implementation of PCRE (release 5.x) corresponds +differences. The current implementation of PCRE (release 6.x) corresponds approximately with Perl 5.8, including support for UTF-8 encoded strings and Unicode general category properties. However, this support has to be explicitly enabled; it is not the default. .P +In addition to the Perl-compatible matching function, PCRE also contains an +alternative matching function that matches the same compiled patterns in a +different way. In certain circumstances, the alternative function has some +advantages. For a discussion of the two matching algorithms, see the +.\" HREF +\fBpcrematching\fP +.\" +page. +.P PCRE is written in C and released as a C library. A number of people have -written wrappers and interfaces of various kinds. A C++ class is included in -these contributions, which can be found in the \fIContrib\fR directory at the -primary FTP site, which is: +written wrappers and interfaces of various kinds. In particular, Google Inc. +have provided a comprehensive C++ wrapper. This is now included as part of the +PCRE distribution. The +.\" HREF +\fBpcrecpp\fP +.\" +page has details of this interface. Other people's contributions can be found +in the \fIContrib\fR directory at the primary FTP site, which is: .sp .\" HTML <a href="ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre"> .\" </a> @@ -43,6 +57,11 @@ available. The features themselves are described in the .\" page. Documentation about building PCRE for various operating systems can be found in the \fBREADME\fP file in the source distribution. +.P +The library contains a number of undocumented internal functions and data +tables that are used by more than one of the exported external functions, but +which are not intended for use by external callers. Their names all begin with +"_pcre_", which hopefully will not provoke any name clashes. . . .SH "USER DOCUMENTATION" @@ -55,23 +74,25 @@ all the sections are concatenated, for ease of searching. The sections are as follows: .sp pcre this document - pcreapi details of PCRE's native API + pcreapi details of PCRE's native C API pcrebuild options for building PCRE pcrecallout details of the callout feature pcrecompat discussion of Perl compatibility + pcrecpp details of the C++ wrapper pcregrep description of the \fBpcregrep\fP command + pcrematching discussion of the two matching algorithms pcrepartial details of the partial matching facility .\" JOIN pcrepattern syntax and semantics of supported regular expressions pcreperform discussion of performance issues - pcreposix the POSIX-compatible API + pcreposix the POSIX-compatible C API pcreprecompile details of saving and re-using precompiled patterns pcresample discussion of the sample program pcretest description of the \fBpcretest\fP testing command .sp In addition, in the "man" and HTML formats, there is a short page for each -library function, listing its arguments and results. +C library function, listing its arguments and results. . . .SH LIMITATIONS @@ -99,9 +120,10 @@ depth of nesting of all kinds of parenthesized subpattern, including capturing subpatterns, assertions, and other types of subpattern, is 200. .P The maximum length of a subject string is the largest positive number that an -integer variable can hold. However, PCRE uses recursion to handle subpatterns -and indefinite repetition. This means that the available stack space may limit -the size of a subject string that can be processed by certain patterns. +integer variable can hold. However, when using the traditional matching +function, PCRE uses recursion to handle subpatterns and indefinite repetition. +This means that the available stack space may limit the size of a subject +string that can be processed by certain patterns. .sp .\" HTML <a name="utf8support"></a> . @@ -167,7 +189,8 @@ bytes, for example: \ex{100}{3}. 5. The dot metacharacter matches one UTF-8 character instead of a single byte. .P 6. The escape sequence \eC can be used to match a single byte in UTF-8 mode, -but its use can lead to some strange effects. +but its use can lead to some strange effects. This facility is not available in +the alternative matching function, \fBpcre_dfa_exec()\fP. .P 7. The character escapes \eb, \eB, \ed, \eD, \es, \eS, \ew, and \eW correctly test characters of any code value, but the characters that PCRE recognizes as @@ -190,15 +213,17 @@ values. .SH AUTHOR .rs .sp -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel .br University Computing Service, .br Cambridge CB2 3QG, England. -.br -Phone: +44 1223 334714 +.P +Putting an actual email address here seems to have been a spam magnet, so I've +taken it away. If you want to email me, use my initial and surname, separated +by a dot, at the domain ucs.cam.ac.uk. .sp .in 0 -Last updated: 09 September 2004 +Last updated: 07 March 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcre.txt b/doc/pcre.txt index fdf0d6f..9de0d62 100644 --- a/doc/pcre.txt +++ b/doc/pcre.txt @@ -6,26 +6,33 @@ synopses of each function in the library have not been included. There are separate text files for the pcregrep and pcretest commands. ----------------------------------------------------------------------------- -PCRE(3) PCRE(3) - NAME PCRE - Perl-compatible regular expressions + INTRODUCTION The PCRE library is a set of functions that implement regular expres- sion pattern matching using the same syntax and semantics as Perl, with just a few differences. The current implementation of PCRE (release - 5.x) corresponds approximately with Perl 5.8, including support for + 6.x) corresponds approximately with Perl 5.8, including support for UTF-8 encoded strings and Unicode general category properties. However, this support has to be explicitly enabled; it is not the default. + In addition to the Perl-compatible matching function, PCRE also con- + tains an alternative matching function that matches the same compiled + patterns in a different way. In certain circumstances, the alternative + function has some advantages. For a discussion of the two matching + algorithms, see the pcrematching page. + PCRE is written in C and released as a C library. A number of people - have written wrappers and interfaces of various kinds. A C++ class is - included in these contributions, which can be found in the Contrib - directory at the primary FTP site, which is: + have written wrappers and interfaces of various kinds. In particular, + Google Inc. have provided a comprehensive C++ wrapper. This is now + included as part of the PCRE distribution. The pcrecpp page has details + of this interface. Other people's contributions can be found in the + Contrib directory at the primary FTP site, which is: ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre @@ -40,6 +47,12 @@ INTRODUCTION ing PCRE for various operating systems can be found in the README file in the source distribution. + The library contains a number of undocumented internal functions and + data tables that are used by more than one of the exported external + functions, but which are not intended for use by external callers. + Their names all begin with "_pcre_", which hopefully will not provoke + any name clashes. + USER DOCUMENTATION @@ -50,22 +63,24 @@ USER DOCUMENTATION of searching. The sections are as follows: pcre this document - pcreapi details of PCRE's native API + pcreapi details of PCRE's native C API pcrebuild options for building PCRE pcrecallout details of the callout feature pcrecompat discussion of Perl compatibility + pcrecpp details of the C++ wrapper pcregrep description of the pcregrep command + pcrematching discussion of the two matching algorithms pcrepartial details of the partial matching facility pcrepattern syntax and semantics of supported regular expressions pcreperform discussion of performance issues - pcreposix the POSIX-compatible API + pcreposix the POSIX-compatible C API pcreprecompile details of saving and re-using precompiled patterns pcresample discussion of the sample program pcretest description of the pcretest testing command In addition, in the "man" and HTML formats, there is a short page for - each library function, listing its arguments and results. + each C library function, listing its arguments and results. LIMITATIONS @@ -90,70 +105,71 @@ LIMITATIONS tern, is 200. The maximum length of a subject string is the largest positive number - that an integer variable can hold. However, PCRE uses recursion to han- - dle subpatterns and indefinite repetition. This means that the avail- - able stack space may limit the size of a subject string that can be - processed by certain patterns. + that an integer variable can hold. However, when using the traditional + matching function, PCRE uses recursion to handle subpatterns and indef- + inite repetition. This means that the available stack space may limit + the size of a subject string that can be processed by certain patterns. UTF-8 AND UNICODE PROPERTY SUPPORT - From release 3.3, PCRE has had some support for character strings - encoded in the UTF-8 format. For release 4.0 this was greatly extended - to cover most common requirements, and in release 5.0 additional sup- + From release 3.3, PCRE has had some support for character strings + encoded in the UTF-8 format. For release 4.0 this was greatly extended + to cover most common requirements, and in release 5.0 additional sup- port for Unicode general category properties was added. - In order process UTF-8 strings, you must build PCRE to include UTF-8 - support in the code, and, in addition, you must call pcre_compile() - with the PCRE_UTF8 option flag. When you do this, both the pattern and - any subject strings that are matched against it are treated as UTF-8 + In order process UTF-8 strings, you must build PCRE to include UTF-8 + support in the code, and, in addition, you must call pcre_compile() + with the PCRE_UTF8 option flag. When you do this, both the pattern and + any subject strings that are matched against it are treated as UTF-8 strings instead of just strings of bytes. - If you compile PCRE with UTF-8 support, but do not use it at run time, - the library will be a bit bigger, but the additional run time overhead - is limited to testing the PCRE_UTF8 flag in several places, so should + If you compile PCRE with UTF-8 support, but do not use it at run time, + the library will be a bit bigger, but the additional run time overhead + is limited to testing the PCRE_UTF8 flag in several places, so should not be very large. If PCRE is built with Unicode character property support (which implies - UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup- + UTF-8 support), the escape sequences \p{..}, \P{..}, and \X are sup- ported. The available properties that can be tested are limited to the - general category properties such as Lu for an upper case letter or Nd - for a decimal number. A full list is given in the pcrepattern documen- + general category properties such as Lu for an upper case letter or Nd + for a decimal number. A full list is given in the pcrepattern documen- tation. The PCRE library is increased in size by about 90K when Unicode property support is included. The following comments apply when PCRE is running in UTF-8 mode: - 1. When you set the PCRE_UTF8 flag, the strings passed as patterns and - subjects are checked for validity on entry to the relevant functions. + 1. When you set the PCRE_UTF8 flag, the strings passed as patterns and + subjects are checked for validity on entry to the relevant functions. If an invalid UTF-8 string is passed, an error return is given. In some - situations, you may already know that your strings are valid, and + situations, you may already know that your strings are valid, and therefore want to skip these checks in order to improve performance. If - you set the PCRE_NO_UTF8_CHECK flag at compile time or at run time, - PCRE assumes that the pattern or subject it is given (respectively) - contains only valid UTF-8 codes. In this case, it does not diagnose an - invalid UTF-8 string. If you pass an invalid UTF-8 string to PCRE when - PCRE_NO_UTF8_CHECK is set, the results are undefined. Your program may + you set the PCRE_NO_UTF8_CHECK flag at compile time or at run time, + PCRE assumes that the pattern or subject it is given (respectively) + contains only valid UTF-8 codes. In this case, it does not diagnose an + invalid UTF-8 string. If you pass an invalid UTF-8 string to PCRE when + PCRE_NO_UTF8_CHECK is set, the results are undefined. Your program may crash. 2. In a pattern, the escape sequence \x{...}, where the contents of the - braces is a string of hexadecimal digits, is interpreted as a UTF-8 - character whose code number is the given hexadecimal number, for exam- - ple: \x{1234}. If a non-hexadecimal digit appears between the braces, + braces is a string of hexadecimal digits, is interpreted as a UTF-8 + character whose code number is the given hexadecimal number, for exam- + ple: \x{1234}. If a non-hexadecimal digit appears between the braces, the item is not recognized. This escape sequence can be used either as a literal, or within a character class. - 3. The original hexadecimal escape sequence, \xhh, matches a two-byte + 3. The original hexadecimal escape sequence, \xhh, matches a two-byte UTF-8 character if the value is greater than 127. - 4. Repeat quantifiers apply to complete UTF-8 characters, not to indi- + 4. Repeat quantifiers apply to complete UTF-8 characters, not to indi- vidual bytes, for example: \x{100}{3}. - 5. The dot metacharacter matches one UTF-8 character instead of a sin- + 5. The dot metacharacter matches one UTF-8 character instead of a sin- gle byte. - 6. The escape sequence \C can be used to match a single byte in UTF-8 - mode, but its use can lead to some strange effects. + 6. The escape sequence \C can be used to match a single byte in UTF-8 + mode, but its use can lead to some strange effects. This facility is + not available in the alternative matching function, pcre_dfa_exec(). 7. The character escapes \b, \B, \d, \D, \s, \S, \w, and \W correctly test characters of any code value, but the characters that PCRE recog- @@ -177,22 +193,24 @@ UTF-8 AND UNICODE PROPERTY SUPPORT AUTHOR - Philip Hazel <ph10@cam.ac.uk> + Philip Hazel University Computing Service, Cambridge CB2 3QG, England. - Phone: +44 1223 334714 -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. ------------------------------------------------------------------------------ + Putting an actual email address here seems to have been a spam magnet, + so I've taken it away. If you want to email me, use my initial and sur- + name, separated by a dot, at the domain ucs.cam.ac.uk. -PCRE(3) PCRE(3) +Last updated: 07 March 2005 +Copyright (c) 1997-2005 University of Cambridge. +----------------------------------------------------------------------------- NAME PCRE - Perl-compatible regular expressions + PCRE BUILD-TIME OPTIONS This document describes the optional features of PCRE that can be @@ -287,16 +305,18 @@ POSIX MALLOC USAGE LIMITING PCRE RESOURCE USAGE Internally, PCRE has a function called match(), which it calls repeat- - edly (possibly recursively) when matching a pattern. By controlling the - maximum number of times this function may be called during a single - matching operation, a limit can be placed on the resources used by a - single call to pcre_exec(). The limit can be changed at run time, as - described in the pcreapi documentation. The default is 10 million, but - this can be changed by adding a setting such as + edly (possibly recursively) when matching a pattern with the + pcre_exec() function. By controlling the maximum number of times this + function may be called during a single matching operation, a limit can + be placed on the resources used by a single call to pcre_exec(). The + limit can be changed at run time, as described in the pcreapi documen- + tation. The default is 10 million, but this can be changed by adding a + setting such as --with-match-limit=500000 - to the configure command. + to the configure command. This setting has no effect on the + pcre_dfa_exec() matching function. HANDLING VERY LARGE PATTERNS @@ -324,14 +344,14 @@ HANDLING VERY LARGE PATTERNS AVOIDING EXCESSIVE STACK USAGE - PCRE implements backtracking while matching by making recursive calls - to an internal function called match(). In environments where the size - of the stack is limited, this can severely limit PCRE's operation. (The - Unix environment does not usually suffer from this problem.) An alter- - native approach that uses memory from the heap to remember data, - instead of using recursive function calls, has been implemented to work - round this problem. If you want to build a version of PCRE that works - this way, add + When matching with the pcre_exec() function, PCRE implements backtrack- + ing by making recursive calls to an internal function called match(). + In environments where the size of the stack is limited, this can se- + verely limit PCRE's operation. (The Unix environment does not usually + suffer from this problem.) An alternative approach that uses memory + from the heap to remember data, instead of using recursive function + calls, has been implemented to work round this problem. If you want to + build a version of PCRE that works this way, add --disable-stack-for-recursion @@ -342,31 +362,197 @@ AVOIDING EXCESSIVE STACK USAGE the blocks are always freed in reverse order. A calling program might be able to implement optimized functions that perform better than the standard malloc() and free() functions. PCRE runs noticeably more - slowly when built in this way. + slowly when built in this way. This option affects only the pcre_exec() + function; it is not relevant for the the pcre_dfa_exec() function. USING EBCDIC CODE - PCRE assumes by default that it will run in an environment where the - character code is ASCII (or Unicode, which is a superset of ASCII). - PCRE can, however, be compiled to run in an EBCDIC environment by + PCRE assumes by default that it will run in an environment where the + character code is ASCII (or Unicode, which is a superset of ASCII). + PCRE can, however, be compiled to run in an EBCDIC environment by adding --enable-ebcdic to the configure command. -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. ----------------------------------------------------------------------------- -PCRE(3) PCRE(3) + + +NAME + PCRE - Perl-compatible regular expressions + + +PCRE MATCHING ALGORITHMS + + This document describes the two different algorithms that are available + in PCRE for matching a compiled regular expression against a given sub- + ject string. The "standard" algorithm is the one provided by the + pcre_exec() function. This works in the same was as Perl's matching + function, and provides a Perl-compatible matching operation. + + An alternative algorithm is provided by the pcre_dfa_exec() function; + this operates in a different way, and is not Perl-compatible. It has + advantages and disadvantages compared with the standard algorithm, and + these are described below. + + When there is only one possible way in which a given subject string can + match a pattern, the two algorithms give the same answer. A difference + arises, however, when there are multiple possibilities. For example, if + the pattern + + ^<.*> + + is matched against the string + + <something> <something else> <something further> + + there are three possible answers. The standard algorithm finds only one + of them, whereas the DFA algorithm finds all three. + + +REGULAR EXPRESSIONS AS TREES + + The set of strings that are matched by a regular expression can be rep- + resented as a tree structure. An unlimited repetition in the pattern + makes the tree of infinite size, but it is still a tree. Matching the + pattern to a given subject string (from a given starting point) can be + thought of as a search of the tree. There are two standard ways to + search a tree: depth-first and breadth-first, and these correspond to + the two matching algorithms provided by PCRE. + + +THE STANDARD MATCHING ALGORITHM + + In the terminology of Jeffrey Friedl's book Mastering Regular Expres- + sions, the standard algorithm is an "NFA algorithm". It conducts a + depth-first search of the pattern tree. That is, it proceeds along a + single path through the tree, checking that the subject matches what is + required. When there is a mismatch, the algorithm tries any alterna- + tives at the current point, and if they all fail, it backs up to the + previous branch point in the tree, and tries the next alternative + branch at that level. This often involves backing up (moving to the + left) in the subject string as well. The order in which repetition + branches are tried is controlled by the greedy or ungreedy nature of + the quantifier. + + If a leaf node is reached, a matching string has been found, and at + that point the algorithm stops. Thus, if there is more than one possi- + ble match, this algorithm returns the first one that it finds. Whether + this is the shortest, the longest, or some intermediate length depends + on the way the greedy and ungreedy repetition quantifiers are specified + in the pattern. + + Because it ends up with a single path through the tree, it is rela- + tively straightforward for this algorithm to keep track of the sub- + strings that are matched by portions of the pattern in parentheses. + This provides support for capturing parentheses and back references. + + +THE DFA MATCHING ALGORITHM + + DFA stands for "deterministic finite automaton", but you do not need to + understand the origins of that name. This algorithm conducts a breadth- + first search of the tree. Starting from the first matching point in the + subject, it scans the subject string from left to right, once, charac- + ter by character, and as it does this, it remembers all the paths + through the tree that represent valid matches. + + The scan continues until either the end of the subject is reached, or + there are no more unterminated paths. At this point, terminated paths + represent the different matching possibilities (if there are none, the + match has failed). Thus, if there is more than one possible match, + this algorithm finds all of them, and in particular, it finds the long- + est. In PCRE, there is an option to stop the algorithm after the first + match (which is necessarily the shortest) has been found. + + Note that all the matches that are found start at the same point in the + subject. If the pattern + + cat(er(pillar)?) + + is matched against the string "the caterpillar catchment", the result + will be the three strings "cat", "cater", and "caterpillar" that start + at the fourth character of the subject. The algorithm does not automat- + ically move on to find matches that start at later positions. + + There are a number of features of PCRE regular expressions that are not + supported by the DFA matching algorithm. They are as follows: + + 1. Because the algorithm finds all possible matches, the greedy or + ungreedy nature of repetition quantifiers is not relevant. Greedy and + ungreedy quantifiers are treated in exactly the same way. + + 2. When dealing with multiple paths through the tree simultaneously, it + is not straightforward to keep track of captured substrings for the + different matching possibilities, and PCRE's implementation of this + algorithm does not attempt to do this. This means that no captured sub- + strings are available. + + 3. Because no substrings are captured, back references within the pat- + tern are not supported, and cause errors if encountered. + + 4. For the same reason, conditional expressions that use a backrefer- + ence as the condition are not supported. + + 5. Callouts are supported, but the value of the capture_top field is + always 1, and the value of the capture_last field is always -1. + + 6. The \C escape sequence, which (in the standard algorithm) matches a + single byte, even in UTF-8 mode, is not supported because the DFA algo- + rithm moves through the subject string one character at a time, for all + active paths through the tree. + + +ADVANTAGES OF THE DFA ALGORITHM + + Using the DFA matching algorithm provides the following advantages: + + 1. All possible matches (at a single point in the subject) are automat- + ically found, and in particular, the longest match is found. To find + more than one match using the standard algorithm, you have to do kludgy + things with callouts. + + 2. There is much better support for partial matching. The restrictions + on the content of the pattern that apply when using the standard algo- + rithm for partial matching do not apply to the DFA algorithm. For non- + anchored patterns, the starting position of a partial match is avail- + able. + + 3. Because the DFA algorithm scans the subject string just once, and + never needs to backtrack, it is possible to pass very long subject + strings to the matching function in several pieces, checking for par- + tial matching each time. + + +DISADVANTAGES OF THE DFA ALGORITHM + + The DFA algorithm suffers from a number of disadvantages: + + 1. It is substantially slower than the standard algorithm. This is + partly because it has to search for all possible matches, but is also + because it is less susceptible to optimization. + + 2. Capturing parentheses and back references are not supported. + + 3. The "atomic group" feature of PCRE regular expressions is supported, + but does not provide the advantage that it does for the standard algo- + rithm. + +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. +----------------------------------------------------------------------------- NAME PCRE - Perl-compatible regular expressions + PCRE NATIVE API #include <pcre.h> @@ -375,6 +561,11 @@ PCRE NATIVE API const char **errptr, int *erroffset, const unsigned char *tableptr); + pcre *pcre_compile2(const char *pattern, int options, + int *errorcodeptr, + const char **errptr, int *erroffset, + const unsigned char *tableptr); + pcre_extra *pcre_study(const pcre *code, int options, const char **errptr); @@ -382,6 +573,11 @@ PCRE NATIVE API const char *subject, int length, int startoffset, int options, int *ovector, int ovecsize); + int pcre_dfa_exec(const pcre *code, const pcre_extra *extra, + const char *subject, int length, int startoffset, + int options, int *ovector, int ovecsize, + int *workspace, int wscount); + int pcre_copy_named_substring(const pcre *code, const char *subject, int *ovector, int stringcount, const char *stringname, @@ -417,6 +613,8 @@ PCRE NATIVE API int pcre_info(const pcre *code, int *optptr, int *firstcharptr); + int pcre_refcount(pcre *code, int adjust); + int pcre_config(int what, void *where); char *pcre_version(void); @@ -436,25 +634,36 @@ PCRE API OVERVIEW PCRE has its own native API, which is described in this document. There is also a set of wrapper functions that correspond to the POSIX regular - expression API. These are described in the pcreposix documentation. + expression API. These are described in the pcreposix documentation. + Both of these APIs define a set of C function calls. A C++ wrapper is + distributed with PCRE. It is documented in the pcrecpp page. - The native API function prototypes are defined in the header file - pcre.h, and on Unix systems the library itself is called libpcre. It + The native API C function prototypes are defined in the header file + pcre.h, and on Unix systems the library itself is called libpcre. It can normally be accessed by adding -lpcre to the command for linking an application that uses PCRE. The header file defines the macros PCRE_MAJOR and PCRE_MINOR to contain the major and minor release num- bers for the library. Applications can use these to include support for different releases of PCRE. - The functions pcre_compile(), pcre_study(), and pcre_exec() are used - for compiling and matching regular expressions. A sample program that - demonstrates the simplest way of using them is provided in the file - called pcredemo.c in the source distribution. The pcresample documenta- - tion describes how to run it. - - In addition to the main compiling and matching functions, there are - convenience functions for extracting captured substrings from a matched - subject string. They are: + The functions pcre_compile(), pcre_compile2(), pcre_study(), and + pcre_exec() are used for compiling and matching regular expressions in + a Perl-compatible manner. A sample program that demonstrates the sim- + plest way of using them is provided in the file called pcredemo.c in + the source distribution. The pcresample documentation describes how to + run it. + + A second matching function, pcre_dfa_exec(), which is not Perl-compati- + ble, is also provided. This uses a different algorithm for the match- + ing. This allows it to find all possible matches (at a given point in + the subject), not just one. However, this algorithm does not return + captured substrings. A description of the two matching algorithms and + their advantages and disadvantages is given in the pcrematching docu- + mentation. + + In addition to the main compiling and matching functions, there are + convenience functions for extracting captured substrings from a subject + string that is matched by pcre_exec(). They are: pcre_copy_substring() pcre_copy_named_substring() @@ -466,11 +675,12 @@ PCRE API OVERVIEW pcre_free_substring() and pcre_free_substring_list() are also provided, to free the memory used for extracted strings. - The function pcre_maketables() is used to build a set of character - tables in the current locale for passing to pcre_compile() or - pcre_exec(). This is an optional facility that is provided for spe- - cialist use. Most commonly, no special tables are passed, in which case - internal tables that are generated when PCRE is built are used. + The function pcre_maketables() is used to build a set of character + tables in the current locale for passing to pcre_compile(), + pcre_exec(), or pcre_dfa_exec(). This is an optional facility that is + provided for specialist use. Most commonly, no special tables are + passed, in which case internal tables that are generated when PCRE is + built are used. The function pcre_fullinfo() is used to find out information about a compiled pattern; pcre_info() is an obsolete version that returns only @@ -478,6 +688,10 @@ PCRE API OVERVIEW patibility. The function pcre_version() returns a pointer to a string containing the version of PCRE and its date of release. + The function pcre_refcount() maintains a reference count in a data + block containing a compiled pattern. This is provided for the benefit + of object-oriented applications. + The global variables pcre_malloc and pcre_free initially contain the entry points of the standard malloc() and free() functions, respec- tively. PCRE calls the memory management functions via these variables, @@ -487,28 +701,28 @@ PCRE API OVERVIEW The global variables pcre_stack_malloc and pcre_stack_free are also indirections to memory management functions. These special functions are used only when PCRE is compiled to use the heap for remembering - data, instead of recursive function calls. This is a non-standard way - of building PCRE, for use in environments that have limited stacks. - Because of the greater use of memory management, it runs more slowly. - Separate functions are provided so that special-purpose external code - can be used for this case. When used, these functions are always called - in a stack-like manner (last obtained, first freed), and always for - memory blocks of the same size. + data, instead of recursive function calls, when running the pcre_exec() + function. This is a non-standard way of building PCRE, for use in envi- + ronments that have limited stacks. Because of the greater use of memory + management, it runs more slowly. Separate functions are provided so + that special-purpose external code can be used for this case. When + used, these functions are always called in a stack-like manner (last + obtained, first freed), and always for memory blocks of the same size. The global variable pcre_callout initially contains NULL. It can be set - by the caller to a "callout" function, which PCRE will then call at - specified points during a matching operation. Details are given in the + by the caller to a "callout" function, which PCRE will then call at + specified points during a matching operation. Details are given in the pcrecallout documentation. MULTITHREADING - The PCRE functions can be used in multi-threading applications, with + The PCRE functions can be used in multi-threading applications, with the proviso that the memory management functions pointed to by pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the callout function pointed to by pcre_callout, are shared by all threads. - The compiled form of a regular expression is not altered during match- + The compiled form of a regular expression is not altered during match- ing, so the same compiled pattern can safely be used by several threads at once. @@ -516,8 +730,8 @@ MULTITHREADING SAVING PRECOMPILED PATTERNS FOR LATER USE The compiled form of a regular expression can be saved and re-used at a - later time, possibly by a different program, and even on a host other - than the one on which it was compiled. Details are given in the + later time, possibly by a different program, and even on a host other + than the one on which it was compiled. Details are given in the pcreprecompile documentation. @@ -525,63 +739,63 @@ CHECKING BUILD-TIME OPTIONS int pcre_config(int what, void *where); - The function pcre_config() makes it possible for a PCRE client to dis- + The function pcre_config() makes it possible for a PCRE client to dis- cover which optional features have been compiled into the PCRE library. - The pcrebuild documentation has more details about these optional fea- + The pcrebuild documentation has more details about these optional fea- tures. - The first argument for pcre_config() is an integer, specifying which + The first argument for pcre_config() is an integer, specifying which information is required; the second argument is a pointer to a variable - into which the information is placed. The following information is + into which the information is placed. The following information is available: PCRE_CONFIG_UTF8 - The output is an integer that is set to one if UTF-8 support is avail- + The output is an integer that is set to one if UTF-8 support is avail- able; otherwise it is set to zero. PCRE_CONFIG_UNICODE_PROPERTIES - The output is an integer that is set to one if support for Unicode + The output is an integer that is set to one if support for Unicode character properties is available; otherwise it is set to zero. PCRE_CONFIG_NEWLINE - The output is an integer that is set to the value of the code that is - used for the newline character. It is either linefeed (10) or carriage - return (13), and should normally be the standard character for your + The output is an integer that is set to the value of the code that is + used for the newline character. It is either linefeed (10) or carriage + return (13), and should normally be the standard character for your operating system. PCRE_CONFIG_LINK_SIZE - The output is an integer that contains the number of bytes used for + The output is an integer that contains the number of bytes used for internal linkage in compiled regular expressions. The value is 2, 3, or - 4. Larger values allow larger regular expressions to be compiled, at - the expense of slower matching. The default value of 2 is sufficient - for all but the most massive patterns, since it allows the compiled + 4. Larger values allow larger regular expressions to be compiled, at + the expense of slower matching. The default value of 2 is sufficient + for all but the most massive patterns, since it allows the compiled pattern to be up to 64K in size. PCRE_CONFIG_POSIX_MALLOC_THRESHOLD - The output is an integer that contains the threshold above which the - POSIX interface uses malloc() for output vectors. Further details are + The output is an integer that contains the threshold above which the + POSIX interface uses malloc() for output vectors. Further details are given in the pcreposix documentation. PCRE_CONFIG_MATCH_LIMIT The output is an integer that gives the default limit for the number of - internal matching function calls in a pcre_exec() execution. Further + internal matching function calls in a pcre_exec() execution. Further details are given with pcre_exec() below. PCRE_CONFIG_STACKRECURSE - The output is an integer that is set to one if internal recursion is - implemented by recursive function calls that use the stack to remember - their state. This is the usual way that PCRE is compiled. The output is - zero if PCRE was compiled to use blocks of data on the heap instead of - recursive function calls. In this case, pcre_stack_malloc and - pcre_stack_free are called to manage memory blocks on the heap, thus - avoiding the use of the stack. + The output is an integer that is set to one if internal recursion when + running pcre_exec() is implemented by recursive function calls that use + the stack to remember their state. This is the usual way that PCRE is + compiled. The output is zero if PCRE was compiled to use blocks of data + on the heap instead of recursive function calls. In this case, + pcre_stack_malloc and pcre_stack_free are called to manage memory + blocks on the heap, thus avoiding the use of the stack. COMPILING A PATTERN @@ -590,38 +804,52 @@ COMPILING A PATTERN const char **errptr, int *erroffset, const unsigned char *tableptr); - The function pcre_compile() is called to compile a pattern into an - internal form. The pattern is a C string terminated by a binary zero, - and is passed in the pattern argument. A pointer to a single block of - memory that is obtained via pcre_malloc is returned. This contains the - compiled code and related data. The pcre type is defined for the - returned block; this is a typedef for a structure whose contents are - not externally defined. It is up to the caller to free the memory when - it is no longer required. + pcre *pcre_compile2(const char *pattern, int options, + int *errorcodeptr, + const char **errptr, int *erroffset, + const unsigned char *tableptr); + + Either of the functions pcre_compile() or pcre_compile2() can be called + to compile a pattern into an internal form. The only difference between + the two interfaces is that pcre_compile2() has an additional argument, + errorcodeptr, via which a numerical error code can be returned. + + The pattern is a C string terminated by a binary zero, and is passed in + the pattern argument. A pointer to a single block of memory that is + obtained via pcre_malloc is returned. This contains the compiled code + and related data. The pcre type is defined for the returned block; this + is a typedef for a structure whose contents are not externally defined. + It is up to the caller to free the memory when it is no longer + required. - Although the compiled code of a PCRE regex is relocatable, that is, it + Although the compiled code of a PCRE regex is relocatable, that is, it does not depend on memory location, the complete pcre data block is not - fully relocatable, because it may contain a copy of the tableptr argu- + fully relocatable, because it may contain a copy of the tableptr argu- ment, which is an address (see below). The options argument contains independent bits that affect the compila- - tion. It should be zero if no options are required. The available - options are described below. Some of them, in particular, those that - are compatible with Perl, can also be set and unset from within the - pattern (see the detailed description in the pcrepattern documenta- - tion). For these options, the contents of the options argument speci- - fies their initial settings at the start of compilation and execution. - The PCRE_ANCHORED option can be set at the time of matching as well as + tion. It should be zero if no options are required. The available + options are described below. Some of them, in particular, those that + are compatible with Perl, can also be set and unset from within the + pattern (see the detailed description in the pcrepattern documenta- + tion). For these options, the contents of the options argument speci- + fies their initial settings at the start of compilation and execution. + The PCRE_ANCHORED option can be set at the time of matching as well as at compile time. If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise, - if compilation of a pattern fails, pcre_compile() returns NULL, and + if compilation of a pattern fails, pcre_compile() returns NULL, and sets the variable pointed to by errptr to point to a textual error mes- - sage. The offset from the start of the pattern to the character where - the error was discovered is placed in the variable pointed to by - erroffset, which must not be NULL. If it is, an immediate error is + sage. The offset from the start of the pattern to the character where + the error was discovered is placed in the variable pointed to by + erroffset, which must not be NULL. If it is, an immediate error is given. + If pcre_compile2() is used instead of pcre_compile(), and the error- + codeptr argument is not NULL, a non-zero error code number is returned + via this argument in the event of an error. This is in addition to the + textual error message. Error codes and messages are listed below. + If the final argument, tableptr, is NULL, PCRE uses a default set of character tables that are built when PCRE is compiled, using the default C locale. Otherwise, tableptr must be an address that is the @@ -664,139 +892,206 @@ COMPILING A PATTERN If this bit is set, letters in the pattern match both upper and lower case letters. It is equivalent to Perl's /i option, and it can be - changed within a pattern by a (?i) option setting. When running in - UTF-8 mode, case support for high-valued characters is available only - when PCRE is built with Unicode character property support. + changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE + always understands the concept of case for characters whose values are + less than 128, so caseless matching is always possible. For characters + with higher values, the concept of case is supported if PCRE is com- + piled with Unicode property support, but not otherwise. If you want to + use caseless matching for characters 128 and above, you must ensure + that PCRE is compiled with Unicode property support as well as with + UTF-8 support. PCRE_DOLLAR_ENDONLY - If this bit is set, a dollar metacharacter in the pattern matches only - at the end of the subject string. Without this option, a dollar also - matches immediately before the final character if it is a newline (but - not before any other newlines). The PCRE_DOLLAR_ENDONLY option is + If this bit is set, a dollar metacharacter in the pattern matches only + at the end of the subject string. Without this option, a dollar also + matches immediately before the final character if it is a newline (but + not before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. There is no equivalent to this option in Perl, and no way to set it within a pattern. PCRE_DOTALL If this bit is set, a dot metacharater in the pattern matches all char- - acters, including newlines. Without it, newlines are excluded. This - option is equivalent to Perl's /s option, and it can be changed within - a pattern by a (?s) option setting. A negative class such as [^a] - always matches a newline character, independent of the setting of this + acters, including newlines. Without it, newlines are excluded. This + option is equivalent to Perl's /s option, and it can be changed within + a pattern by a (?s) option setting. A negative class such as [^a] + always matches a newline character, independent of the setting of this option. PCRE_EXTENDED - If this bit is set, whitespace data characters in the pattern are - totally ignored except when escaped or inside a character class. - Whitespace does not include the VT character (code 11). In addition, - characters between an unescaped # outside a character class and the - next newline character, inclusive, are also ignored. This is equivalent - to Perl's /x option, and it can be changed within a pattern by a (?x) + If this bit is set, whitespace data characters in the pattern are + totally ignored except when escaped or inside a character class. White- + space does not include the VT character (code 11). In addition, charac- + ters between an unescaped # outside a character class and the next new- + line character, inclusive, are also ignored. This is equivalent to + Perl's /x option, and it can be changed within a pattern by a (?x) option setting. - This option makes it possible to include comments inside complicated - patterns. Note, however, that this applies only to data characters. - Whitespace characters may never appear within special character - sequences in a pattern, for example within the sequence (?( which + This option makes it possible to include comments inside complicated + patterns. Note, however, that this applies only to data characters. + Whitespace characters may never appear within special character + sequences in a pattern, for example within the sequence (?( which introduces a conditional subpattern. PCRE_EXTRA - This option was invented in order to turn on additional functionality - of PCRE that is incompatible with Perl, but it is currently of very - little use. When set, any backslash in a pattern that is followed by a - letter that has no special meaning causes an error, thus reserving - these combinations for future expansion. By default, as in Perl, a - backslash followed by a letter with no special meaning is treated as a - literal. There are at present no other features controlled by this + This option was invented in order to turn on additional functionality + of PCRE that is incompatible with Perl, but it is currently of very + little use. When set, any backslash in a pattern that is followed by a + letter that has no special meaning causes an error, thus reserving + these combinations for future expansion. By default, as in Perl, a + backslash followed by a letter with no special meaning is treated as a + literal. There are at present no other features controlled by this option. It can also be set by a (?X) option setting within a pattern. + PCRE_FIRSTLINE + + If this option is set, an unanchored pattern is required to match + before or at the first newline character in the subject string, though + the matched text may continue over the newline. + PCRE_MULTILINE - By default, PCRE treats the subject string as consisting of a single - line of characters (even if it actually contains newlines). The "start - of line" metacharacter (^) matches only at the start of the string, - while the "end of line" metacharacter ($) matches only at the end of + By default, PCRE treats the subject string as consisting of a single + line of characters (even if it actually contains newlines). The "start + of line" metacharacter (^) matches only at the start of the string, + while the "end of line" metacharacter ($) matches only at the end of the string, or before a terminating newline (unless PCRE_DOLLAR_ENDONLY is set). This is the same as Perl. - When PCRE_MULTILINE it is set, the "start of line" and "end of line" - constructs match immediately following or immediately before any new- - line in the subject string, respectively, as well as at the very start - and end. This is equivalent to Perl's /m option, and it can be changed + When PCRE_MULTILINE it is set, the "start of line" and "end of line" + constructs match immediately following or immediately before any new- + line in the subject string, respectively, as well as at the very start + and end. This is equivalent to Perl's /m option, and it can be changed within a pattern by a (?m) option setting. If there are no "\n" charac- - ters in a subject string, or no occurrences of ^ or $ in a pattern, + ters in a subject string, or no occurrences of ^ or $ in a pattern, setting PCRE_MULTILINE has no effect. PCRE_NO_AUTO_CAPTURE If this option is set, it disables the use of numbered capturing paren- - theses in the pattern. Any opening parenthesis that is not followed by - ? behaves as if it were followed by ?: but named parentheses can still - be used for capturing (and they acquire numbers in the usual way). + theses in the pattern. Any opening parenthesis that is not followed by + ? behaves as if it were followed by ?: but named parentheses can still + be used for capturing (and they acquire numbers in the usual way). There is no equivalent of this option in Perl. PCRE_UNGREEDY - This option inverts the "greediness" of the quantifiers so that they - are not greedy by default, but become greedy if followed by "?". It is - not compatible with Perl. It can also be set by a (?U) option setting + This option inverts the "greediness" of the quantifiers so that they + are not greedy by default, but become greedy if followed by "?". It is + not compatible with Perl. It can also be set by a (?U) option setting within the pattern. PCRE_UTF8 - This option causes PCRE to regard both the pattern and the subject as - strings of UTF-8 characters instead of single-byte character strings. - However, it is available only when PCRE is built to include UTF-8 sup- - port. If not, the use of this option provokes an error. Details of how - this option changes the behaviour of PCRE are given in the section on + This option causes PCRE to regard both the pattern and the subject as + strings of UTF-8 characters instead of single-byte character strings. + However, it is available only when PCRE is built to include UTF-8 sup- + port. If not, the use of this option provokes an error. Details of how + this option changes the behaviour of PCRE are given in the section on UTF-8 support in the main pcre page. PCRE_NO_UTF8_CHECK When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is - automatically checked. If an invalid UTF-8 sequence of bytes is found, - pcre_compile() returns an error. If you already know that your pattern - is valid, and you want to skip this check for performance reasons, you - can set the PCRE_NO_UTF8_CHECK option. When it is set, the effect of + automatically checked. If an invalid UTF-8 sequence of bytes is found, + pcre_compile() returns an error. If you already know that your pattern + is valid, and you want to skip this check for performance reasons, you + can set the PCRE_NO_UTF8_CHECK option. When it is set, the effect of passing an invalid UTF-8 string as a pattern is undefined. It may cause - your program to crash. Note that this option can also be passed to - pcre_exec(), to suppress the UTF-8 validity checking of subject - strings. + your program to crash. Note that this option can also be passed to + pcre_exec() and pcre_dfa_exec(), to suppress the UTF-8 validity check- + ing of subject strings. + + +COMPILATION ERROR CODES + + The following table lists the error codes than may be returned by + pcre_compile2(), along with the error messages that may be returned by + both compiling functions. + + 0 no error + 1 \ at end of pattern + 2 \c at end of pattern + 3 unrecognized character follows \ + 4 numbers out of order in {} quantifier + 5 number too big in {} quantifier + 6 missing terminating ] for character class + 7 invalid escape sequence in character class + 8 range out of order in character class + 9 nothing to repeat + 10 operand of unlimited repeat could match the empty string + 11 internal error: unexpected repeat + 12 unrecognized character after (? + 13 POSIX named classes are supported only within a class + 14 missing ) + 15 reference to non-existent subpattern + 16 erroffset passed as NULL + 17 unknown option bit(s) set + 18 missing ) after comment + 19 parentheses nested too deeply + 20 regular expression too large + 21 failed to get memory + 22 unmatched parentheses + 23 internal error: code overflow + 24 unrecognized character after (?< + 25 lookbehind assertion is not fixed length + 26 malformed number after (?( + 27 conditional group contains more than two branches + 28 assertion expected after (?( + 29 (?R or (?digits must be followed by ) + 30 unknown POSIX class name + 31 POSIX collating elements are not supported + 32 this version of PCRE is not compiled with PCRE_UTF8 support + 33 spare error + 34 character value in \x{...} sequence is too large + 35 invalid condition (?(0) + 36 \C not allowed in lookbehind assertion + 37 PCRE does not support \L, \l, \N, \U, or \u + 38 number after (?C is > 255 + 39 closing ) for (?C expected + 40 recursive call could loop indefinitely + 41 unrecognized character after (?P + 42 syntax error after (?P + 43 two named groups have the same name + 44 invalid UTF-8 string + 45 support for \P, \p, and \X has not been compiled + 46 malformed \P or \p sequence + 47 unknown property name after \P or \p STUDYING A PATTERN - pcre_extra *pcre_study(const pcre *code, int options, + pcre_extra *pcre_study(const pcre *code, int options const char **errptr); - If a compiled pattern is going to be used several times, it is worth + If a compiled pattern is going to be used several times, it is worth spending more time analyzing it in order to speed up the time taken for - matching. The function pcre_study() takes a pointer to a compiled pat- + matching. The function pcre_study() takes a pointer to a compiled pat- tern as its first argument. If studying the pattern produces additional - information that will help speed up matching, pcre_study() returns a - pointer to a pcre_extra block, in which the study_data field points to + information that will help speed up matching, pcre_study() returns a + pointer to a pcre_extra block, in which the study_data field points to the results of the study. The returned value from pcre_study() can be passed directly to - pcre_exec(). However, a pcre_extra block also contains other fields - that can be set by the caller before the block is passed; these are + pcre_exec(). However, a pcre_extra block also contains other fields + that can be set by the caller before the block is passed; these are described below in the section on matching a pattern. - If studying the pattern does not produce any additional information, + If studying the pattern does not produce any additional information pcre_study() returns NULL. In that circumstance, if the calling program - wants to pass any of the other fields to pcre_exec(), it must set up + wants to pass any of the other fields to pcre_exec(), it must set up its own pcre_extra block. - The second argument of pcre_study() contains option bits. At present, + The second argument of pcre_study() contains option bits. At present, no options are defined, and this argument should always be zero. - The third argument for pcre_study() is a pointer for an error message. - If studying succeeds (even if no data is returned), the variable it - points to is set to NULL. Otherwise it points to a textual error mes- - sage. You should therefore test the error pointer for NULL after call- + The third argument for pcre_study() is a pointer for an error message. + If studying succeeds (even if no data is returned), the variable it + points to is set to NULL. Otherwise it points to a textual error mes- + sage. You should therefore test the error pointer for NULL after call- ing pcre_study(), to be sure that it has run successfully. This is a typical call to pcre_study(): @@ -808,51 +1103,51 @@ STUDYING A PATTERN &error); /* set to NULL or points to a message */ At present, studying a pattern is useful only for non-anchored patterns - that do not have a single fixed starting character. A bitmap of possi- + that do not have a single fixed starting character. A bitmap of possi- ble starting bytes is created. LOCALE SUPPORT - PCRE handles caseless matching, and determines whether characters are - letters, digits, or whatever, by reference to a set of tables, indexed - by character value. (When running in UTF-8 mode, this applies only to - characters with codes less than 128. Higher-valued codes never match - escapes such as \w or \d, but can be tested with \p if PCRE is built - with Unicode character property support.) - - An internal set of tables is created in the default C locale when PCRE - is built. This is used when the final argument of pcre_compile() is - NULL, and is sufficient for many applications. An alternative set of - tables can, however, be supplied. These may be created in a different - locale from the default. As more and more applications change to using + PCRE handles caseless matching, and determines whether characters are + letters digits, or whatever, by reference to a set of tables, indexed + by character value. When running in UTF-8 mode, this applies only to + characters with codes less than 128. Higher-valued codes never match + escapes such as \w or \d, but can be tested with \p if PCRE is built + with Unicode character property support. + + An internal set of tables is created in the default C locale when PCRE + is built. This is used when the final argument of pcre_compile() is + NULL, and is sufficient for many applications. An alternative set of + tables can, however, be supplied. These may be created in a different + locale from the default. As more and more applications change to using Unicode, the need for this locale support is expected to die away. - External tables are built by calling the pcre_maketables() function, - which has no arguments, in the relevant locale. The result can then be - passed to pcre_compile() or pcre_exec() as often as necessary. For - example, to build and use tables that are appropriate for the French - locale (where accented characters with values greater than 128 are + External tables are built by calling the pcre_maketables() function, + which has no arguments, in the relevant locale. The result can then be + passed to pcre_compile() or pcre_exec() as often as necessary. For + example, to build and use tables that are appropriate for the French + locale (where accented characters with values greater than 128 are treated as letters), the following code could be used: setlocale(LC_CTYPE, "fr_FR"); tables = pcre_maketables(); re = pcre_compile(..., tables); - When pcre_maketables() runs, the tables are built in memory that is - obtained via pcre_malloc. It is the caller's responsibility to ensure - that the memory containing the tables remains available for as long as + When pcre_maketables() runs, the tables are built in memory that is + obtained via pcre_malloc. It is the caller's responsibility to ensure + that the memory containing the tables remains available for as long as it is needed. The pointer that is passed to pcre_compile() is saved with the compiled - pattern, and the same tables are used via this pointer by pcre_study() + pattern, and the same tables are used via this pointer by pcre_study() and normally also by pcre_exec(). Thus, by default, for any single pat- tern, compilation, studying and matching all happen in the same locale, but different patterns can be compiled in different locales. - It is possible to pass a table pointer or NULL (indicating the use of - the internal tables) to pcre_exec(). Although not intended for this - purpose, this facility could be used to match a pattern in a different + It is possible to pass a table pointer or NULL (indicating the use of + the internal tables) to pcre_exec(). Although not intended for this + purpose, this facility could be used to match a pattern in a different locale from the one in which it was compiled. Passing table pointers at run time is discussed below in the section on matching a pattern. @@ -862,15 +1157,15 @@ INFORMATION ABOUT A PATTERN int pcre_fullinfo(const pcre *code, const pcre_extra *extra, int what, void *where); - The pcre_fullinfo() function returns information about a compiled pat- + The pcre_fullinfo() function returns information about a compiled pat- tern. It replaces the obsolete pcre_info() function, which is neverthe- less retained for backwards compability (and is documented below). - The first argument for pcre_fullinfo() is a pointer to the compiled - pattern. The second argument is the result of pcre_study(), or NULL if - the pattern was not studied. The third argument specifies which piece - of information is required, and the fourth argument is a pointer to a - variable to receive the data. The yield of the function is zero for + The first argument for pcre_fullinfo() is a pointer to the compiled + pattern. The second argument is the result of pcre_study(), or NULL if + the pattern was not studied. The third argument specifies which piece + of information is required, and the fourth argument is a pointer to a + variable to receive the data. The yield of the function is zero for success, or one of the following negative numbers: PCRE_ERROR_NULL the argument code was NULL @@ -878,9 +1173,9 @@ INFORMATION ABOUT A PATTERN PCRE_ERROR_BADMAGIC the "magic number" was not found PCRE_ERROR_BADOPTION the value of what was invalid - The "magic number" is placed at the start of each compiled pattern as - an simple check against passing an arbitrary memory pointer. Here is a - typical call of pcre_fullinfo(), to obtain the length of the compiled + The "magic number" is placed at the start of each compiled pattern as + an simple check against passing an arbitrary memory pointer. Here is a + typical call of pcre_fullinfo(), to obtain the length of the compiled pattern: int rc; @@ -891,64 +1186,64 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_SIZE, /* what is required */ &length); /* where to put the data */ - The possible values for the third argument are defined in pcre.h, and + The possible values for the third argument are defined in pcre.h, and are as follows: PCRE_INFO_BACKREFMAX - Return the number of the highest back reference in the pattern. The - fourth argument should point to an int variable. Zero is returned if + Return the number of the highest back reference in the pattern. The + fourth argument should point to an int variable. Zero is returned if there are no back references. PCRE_INFO_CAPTURECOUNT - Return the number of capturing subpatterns in the pattern. The fourth + Return the number of capturing subpatterns in the pattern. The fourth argument should point to an int variable. - PCRE_INFO_DEFAULTTABLES + PCRE_INFO_DEFAULT_TABLES - Return a pointer to the internal default character tables within PCRE. - The fourth argument should point to an unsigned char * variable. This + Return a pointer to the internal default character tables within PCRE. + The fourth argument should point to an unsigned char * variable. This information call is provided for internal use by the pcre_study() func- - tion. External callers can cause PCRE to use its internal tables by + tion. External callers can cause PCRE to use its internal tables by passing a NULL table pointer. PCRE_INFO_FIRSTBYTE - Return information about the first byte of any matched string, for a - non-anchored pattern. (This option used to be called - PCRE_INFO_FIRSTCHAR; the old name is still recognized for backwards + Return information about the first byte of any matched string, for a + non-anchored pattern. (This option used to be called + PCRE_INFO_FIRSTCHAR; the old name is still recognized for backwards compatibility.) - If there is a fixed first byte, for example, from a pattern such as - (cat|cow|coyote), it is returned in the integer pointed to by where. + If there is a fixed first byte, for example, from a pattern such as + (cat|cow|coyote), it is returned in the integer pointed to by where. Otherwise, if either - (a) the pattern was compiled with the PCRE_MULTILINE option, and every + (a) the pattern was compiled with the PCRE_MULTILINE option, and every branch starts with "^", or (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set (if it were set, the pattern would be anchored), - -1 is returned, indicating that the pattern matches only at the start - of a subject string or after any newline within the string. Otherwise + -1 is returned, indicating that the pattern matches only at the start + of a subject string or after any newline within the string. Otherwise -2 is returned. For anchored patterns, -2 is returned. PCRE_INFO_FIRSTTABLE - If the pattern was studied, and this resulted in the construction of a + If the pattern was studied, and this resulted in the construction of a 256-bit table indicating a fixed set of bytes for the first byte in any - matching string, a pointer to the table is returned. Otherwise NULL is - returned. The fourth argument should point to an unsigned char * vari- + matching string, a pointer to the table is returned. Otherwise NULL is + returned. The fourth argument should point to an unsigned char * vari- able. PCRE_INFO_LASTLITERAL - Return the value of the rightmost literal byte that must exist in any - matched string, other than at its start, if such a byte has been + Return the value of the rightmost literal byte that must exist in any + matched string, other than at its start, if such a byte has been recorded. The fourth argument should point to an int variable. If there - is no such byte, -1 is returned. For anchored patterns, a last literal - byte is recorded only if it follows something of variable length. For + is no such byte, -1 is returned. For anchored patterns, a last literal + byte is recorded only if it follows something of variable length. For example, for the pattern /^a\d+z\d+/ the returned value is "z", but for /^a\dz\d/ the returned value is -1. @@ -956,32 +1251,32 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_NAMEENTRYSIZE PCRE_INFO_NAMETABLE - PCRE supports the use of named as well as numbered capturing parenthe- - ses. The names are just an additional way of identifying the parenthe- + PCRE supports the use of named as well as numbered capturing parenthe- + ses. The names are just an additional way of identifying the parenthe- ses, which still acquire numbers. A convenience function called - pcre_get_named_substring() is provided for extracting an individual - captured substring by name. It is also possible to extract the data - directly, by first converting the name to a number in order to access - the correct pointers in the output vector (described with pcre_exec() - below). To do the conversion, you need to use the name-to-number map, + pcre_get_named_substring() is provided for extracting an individual + captured substring by name. It is also possible to extract the data + directly, by first converting the name to a number in order to access + the correct pointers in the output vector (described with pcre_exec() + below). To do the conversion, you need to use the name-to-number map, which is described by these three values. The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size - of each entry; both of these return an int value. The entry size - depends on the length of the longest name. PCRE_INFO_NAMETABLE returns - a pointer to the first entry of the table (a pointer to char). The + of each entry; both of these return an int value. The entry size + depends on the length of the longest name. PCRE_INFO_NAMETABLE returns + a pointer to the first entry of the table (a pointer to char). The first two bytes of each entry are the number of the capturing parenthe- - sis, most significant byte first. The rest of the entry is the corre- - sponding name, zero terminated. The names are in alphabetical order. - For example, consider the following pattern (assume PCRE_EXTENDED is + sis, most significant byte first. The rest of the entry is the corre- + sponding name, zero terminated. The names are in alphabetical order. + For example, consider the following pattern (assume PCRE_EXTENDED is set, so white space - including newlines - is ignored): (?P<date> (?P<year>(\d\d)?\d\d) - (?P<month>\d\d) - (?P<day>\d\d) ) - There are four named subpatterns, so the table has four entries, and - each entry in the table is eight bytes long. The table is as follows, + There are four named subpatterns, so the table has four entries, and + each entry in the table is eight bytes long. The table is as follows, with non-printing bytes shows in hexadecimal, and undefined bytes shown as ??: @@ -990,18 +1285,18 @@ INFORMATION ABOUT A PATTERN 00 04 m o n t h 00 00 02 y e a r 00 ?? - When writing code to extract data from named subpatterns using the + When writing code to extract data from named subpatterns using the name-to-number map, remember that the length of each entry is likely to be different for each compiled pattern. PCRE_INFO_OPTIONS - Return a copy of the options with which the pattern was compiled. The - fourth argument should point to an unsigned long int variable. These + Return a copy of the options with which the pattern was compiled. The + fourth argument should point to an unsigned long int variable. These option bits are those specified in the call to pcre_compile(), modified by any top-level option settings within the pattern itself. - A pattern is automatically anchored by PCRE if all of its top-level + A pattern is automatically anchored by PCRE if all of its top-level alternatives begin with one of the following: ^ unless PCRE_MULTILINE is set @@ -1015,7 +1310,7 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_SIZE - Return the size of the compiled pattern, that is, the value that was + Return the size of the compiled pattern, that is, the value that was passed as the argument to pcre_malloc() when PCRE was getting memory in which to place the compiled data. The fourth argument should point to a size_t variable. @@ -1023,9 +1318,9 @@ INFORMATION ABOUT A PATTERN PCRE_INFO_STUDYSIZE Return the size of the data block pointed to by the study_data field in - a pcre_extra block. That is, it is the value that was passed to + a pcre_extra block. That is, it is the value that was passed to pcre_malloc() when PCRE was getting memory into which to place the data - created by pcre_study(). The fourth argument should point to a size_t + created by pcre_study(). The fourth argument should point to a size_t variable. @@ -1033,34 +1328,59 @@ OBSOLETE INFO FUNCTION int pcre_info(const pcre *code, int *optptr, int *firstcharptr); - The pcre_info() function is now obsolete because its interface is too - restrictive to return all the available data about a compiled pattern. - New programs should use pcre_fullinfo() instead. The yield of - pcre_info() is the number of capturing subpatterns, or one of the fol- + The pcre_info() function is now obsolete because its interface is too + restrictive to return all the available data about a compiled pattern. + New programs should use pcre_fullinfo() instead. The yield of + pcre_info() is the number of capturing subpatterns, or one of the fol- lowing negative numbers: PCRE_ERROR_NULL the argument code was NULL PCRE_ERROR_BADMAGIC the "magic number" was not found - If the optptr argument is not NULL, a copy of the options with which - the pattern was compiled is placed in the integer it points to (see + If the optptr argument is not NULL, a copy of the options with which + the pattern was compiled is placed in the integer it points to (see PCRE_INFO_OPTIONS above). - If the pattern is not anchored and the firstcharptr argument is not - NULL, it is used to pass back information about the first character of + If the pattern is not anchored and the firstcharptr argument is not + NULL, it is used to pass back information about the first character of any matched string (see PCRE_INFO_FIRSTBYTE above). -MATCHING A PATTERN +REFERENCE COUNTS + + int pcre_refcount(pcre *code, int adjust); + + The pcre_refcount() function is used to maintain a reference count in + the data block that contains a compiled pattern. It is provided for the + benefit of applications that operate in an object-oriented manner, + where different parts of the application may be using the same compiled + pattern, but you want to free the block when they are all done. + + When a pattern is compiled, the reference count field is initialized to + zero. It is changed only by calling this function, whose action is to + add the adjust value (which may be positive or negative) to it. The + yield of the function is the new value. However, the value of the count + is constrained to lie between 0 and 65535, inclusive. If the new value + is outside these limits, it is forced to the appropriate limit value. + + Except when it is zero, the reference count is not correctly preserved + if a pattern is compiled on one host and then transferred to a host + whose byte-order is different. (This seems a highly unlikely scenario.) + + +MATCHING A PATTERN: THE TRADITIONAL FUNCTION int pcre_exec(const pcre *code, const pcre_extra *extra, const char *subject, int length, int startoffset, int options, int *ovector, int ovecsize); - The function pcre_exec() is called to match a subject string against a - compiled pattern, which is passed in the code argument. If the pattern + The function pcre_exec() is called to match a subject string against a + compiled pattern, which is passed in the code argument. If the pattern has been studied, the result of the study should be passed in the extra - argument. + argument. This function is the main matching facility of the library, + and it operates in a Perl-like manner. For specialist use there is also + an alternative matching function, which is described below in the sec- + tion about the pcre_dfa_exec() function. In most applications, the pattern will have been compiled (and option- ally studied) in the same process that calls pcre_exec(). However, it @@ -1080,15 +1400,14 @@ MATCHING A PATTERN 0, /* start at offset 0 in the subject */ 0, /* default options */ ovector, /* vector of integers for substring information */ - 30); /* number of elements in the vector (NOT size in - bytes) */ + 30); /* number of elements (NOT size in bytes) */ Extra data for pcre_exec() - If the extra argument is not NULL, it must point to a pcre_extra data - block. The pcre_study() function returns such a block (when it doesn't - return NULL), but you can also create one for yourself, and pass addi- - tional information in it. The fields in a pcre_extra block are as fol- + If the extra argument is not NULL, it must point to a pcre_extra data + block. The pcre_study() function returns such a block (when it doesn't + return NULL), but you can also create one for yourself, and pass addi- + tional information in it. The fields in a pcre_extra block are as fol- lows: unsigned long int flags; @@ -1097,7 +1416,7 @@ MATCHING A PATTERN void *callout_data; const unsigned char *tables; - The flags field is a bitmap that specifies which of the other fields + The flags field is a bitmap that specifies which of the other fields are set. The flag bits are: PCRE_EXTRA_STUDY_DATA @@ -1105,229 +1424,229 @@ MATCHING A PATTERN PCRE_EXTRA_CALLOUT_DATA PCRE_EXTRA_TABLES - Other flag bits should be set to zero. The study_data field is set in - the pcre_extra block that is returned by pcre_study(), together with + Other flag bits should be set to zero. The study_data field is set in + the pcre_extra block that is returned by pcre_study(), together with the appropriate flag bit. You should not set this yourself, but you may - add to the block by setting the other fields and their corresponding + add to the block by setting the other fields and their corresponding flag bits. The match_limit field provides a means of preventing PCRE from using up - a vast amount of resources when running patterns that are not going to - match, but which have a very large number of possibilities in their - search trees. The classic example is the use of nested unlimited + a vast amount of resources when running patterns that are not going to + match, but which have a very large number of possibilities in their + search trees. The classic example is the use of nested unlimited repeats. - Internally, PCRE uses a function called match() which it calls repeat- - edly (sometimes recursively). The limit is imposed on the number of - times this function is called during a match, which has the effect of - limiting the amount of recursion and backtracking that can take place. + Internally, PCRE uses a function called match() which it calls repeat- + edly (sometimes recursively). The limit is imposed on the number of + times this function is called during a match, which has the effect of + limiting the amount of recursion and backtracking that can take place. For patterns that are not anchored, the count starts from zero for each position in the subject string. - The default limit for the library can be set when PCRE is built; the - default default is 10 million, which handles all but the most extreme - cases. You can reduce the default by suppling pcre_exec() with a - pcre_extra block in which match_limit is set to a smaller value, and - PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is + The default limit for the library can be set when PCRE is built; the + default default is 10 million, which handles all but the most extreme + cases. You can reduce the default by suppling pcre_exec() with a + pcre_extra block in which match_limit is set to a smaller value, and + PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT. - The pcre_callout field is used in conjunction with the "callout" fea- + The pcre_callout field is used in conjunction with the "callout" fea- ture, which is described in the pcrecallout documentation. - The tables field is used to pass a character tables pointer to - pcre_exec(); this overrides the value that is stored with the compiled - pattern. A non-NULL value is stored with the compiled pattern only if - custom tables were supplied to pcre_compile() via its tableptr argu- + The tables field is used to pass a character tables pointer to + pcre_exec(); this overrides the value that is stored with the compiled + pattern. A non-NULL value is stored with the compiled pattern only if + custom tables were supplied to pcre_compile() via its tableptr argu- ment. If NULL is passed to pcre_exec() using this mechanism, it forces - PCRE's internal tables to be used. This facility is helpful when re- - using patterns that have been saved after compiling with an external - set of tables, because the external tables might be at a different - address when pcre_exec() is called. See the pcreprecompile documenta- + PCRE's internal tables to be used. This facility is helpful when re- + using patterns that have been saved after compiling with an external + set of tables, because the external tables might be at a different + address when pcre_exec() is called. See the pcreprecompile documenta- tion for a discussion of saving compiled patterns for later use. Option bits for pcre_exec() - The unused bits of the options argument for pcre_exec() must be zero. - The only bits that may be set are PCRE_ANCHORED, PCRE_NOTBOL, + The unused bits of the options argument for pcre_exec() must be zero. + The only bits that may be set are PCRE_ANCHORED, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK and PCRE_PARTIAL. PCRE_ANCHORED - The PCRE_ANCHORED option limits pcre_exec() to matching at the first - matching position. If a pattern was compiled with PCRE_ANCHORED, or - turned out to be anchored by virtue of its contents, it cannot be made + The PCRE_ANCHORED option limits pcre_exec() to matching at the first + matching position. If a pattern was compiled with PCRE_ANCHORED, or + turned out to be anchored by virtue of its contents, it cannot be made unachored at matching time. PCRE_NOTBOL This option specifies that first character of the subject string is not - the beginning of a line, so the circumflex metacharacter should not - match before it. Setting this without PCRE_MULTILINE (at compile time) - causes circumflex never to match. This option affects only the - behaviour of the circumflex metacharacter. It does not affect \A. + the beginning of a line, so the circumflex metacharacter should not + match before it. Setting this without PCRE_MULTILINE (at compile time) + causes circumflex never to match. This option affects only the behav- + iour of the circumflex metacharacter. It does not affect \A. PCRE_NOTEOL This option specifies that the end of the subject string is not the end - of a line, so the dollar metacharacter should not match it nor (except - in multiline mode) a newline immediately before it. Setting this with- + of a line, so the dollar metacharacter should not match it nor (except + in multiline mode) a newline immediately before it. Setting this with- out PCRE_MULTILINE (at compile time) causes dollar never to match. This - option affects only the behaviour of the dollar metacharacter. It does + option affects only the behaviour of the dollar metacharacter. It does not affect \Z or \z. PCRE_NOTEMPTY An empty string is not considered to be a valid match if this option is - set. If there are alternatives in the pattern, they are tried. If all - the alternatives match the empty string, the entire match fails. For + set. If there are alternatives in the pattern, they are tried. If all + the alternatives match the empty string, the entire match fails. For example, if the pattern a?b? - is applied to a string not beginning with "a" or "b", it matches the - empty string at the start of the subject. With PCRE_NOTEMPTY set, this + is applied to a string not beginning with "a" or "b", it matches the + empty string at the start of the subject. With PCRE_NOTEMPTY set, this match is not valid, so PCRE searches further into the string for occur- rences of "a" or "b". Perl has no direct equivalent of PCRE_NOTEMPTY, but it does make a spe- - cial case of a pattern match of the empty string within its split() - function, and when using the /g modifier. It is possible to emulate + cial case of a pattern match of the empty string within its split() + function, and when using the /g modifier. It is possible to emulate Perl's behaviour after matching a null string by first trying the match again at the same offset with PCRE_NOTEMPTY and PCRE_ANCHORED, and then - if that fails by advancing the starting offset (see below) and trying + if that fails by advancing the starting offset (see below) and trying an ordinary match again. There is some code that demonstrates how to do this in the pcredemo.c sample program. PCRE_NO_UTF8_CHECK When PCRE_UTF8 is set at compile time, the validity of the subject as a - UTF-8 string is automatically checked when pcre_exec() is subsequently - called. The value of startoffset is also checked to ensure that it - points to the start of a UTF-8 character. If an invalid UTF-8 sequence + UTF-8 string is automatically checked when pcre_exec() is subsequently + called. The value of startoffset is also checked to ensure that it + points to the start of a UTF-8 character. If an invalid UTF-8 sequence of bytes is found, pcre_exec() returns the error PCRE_ERROR_BADUTF8. If - startoffset contains an invalid value, PCRE_ERROR_BADUTF8_OFFSET is + startoffset contains an invalid value, PCRE_ERROR_BADUTF8_OFFSET is returned. - If you already know that your subject is valid, and you want to skip - these checks for performance reasons, you can set the - PCRE_NO_UTF8_CHECK option when calling pcre_exec(). You might want to - do this for the second and subsequent calls to pcre_exec() if you are - making repeated calls to find all the matches in a single subject - string. However, you should be sure that the value of startoffset - points to the start of a UTF-8 character. When PCRE_NO_UTF8_CHECK is - set, the effect of passing an invalid UTF-8 string as a subject, or a - value of startoffset that does not point to the start of a UTF-8 char- + If you already know that your subject is valid, and you want to skip + these checks for performance reasons, you can set the + PCRE_NO_UTF8_CHECK option when calling pcre_exec(). You might want to + do this for the second and subsequent calls to pcre_exec() if you are + making repeated calls to find all the matches in a single subject + string. However, you should be sure that the value of startoffset + points to the start of a UTF-8 character. When PCRE_NO_UTF8_CHECK is + set, the effect of passing an invalid UTF-8 string as a subject, or a + value of startoffset that does not point to the start of a UTF-8 char- acter, is undefined. Your program may crash. PCRE_PARTIAL - This option turns on the partial matching feature. If the subject - string fails to match the pattern, but at some point during the match- - ing process the end of the subject was reached (that is, the subject - partially matches the pattern and the failure to match occurred only - because there were not enough subject characters), pcre_exec() returns - PCRE_ERROR_PARTIAL instead of PCRE_ERROR_NOMATCH. When PCRE_PARTIAL is - used, there are restrictions on what may appear in the pattern. These + This option turns on the partial matching feature. If the subject + string fails to match the pattern, but at some point during the match- + ing process the end of the subject was reached (that is, the subject + partially matches the pattern and the failure to match occurred only + because there were not enough subject characters), pcre_exec() returns + PCRE_ERROR_PARTIAL instead of PCRE_ERROR_NOMATCH. When PCRE_PARTIAL is + used, there are restrictions on what may appear in the pattern. These are discussed in the pcrepartial documentation. The string to be matched by pcre_exec() - The subject string is passed to pcre_exec() as a pointer in subject, a - length in length, and a starting byte offset in startoffset. In UTF-8 - mode, the byte offset must point to the start of a UTF-8 character. - Unlike the pattern string, the subject may contain binary zero bytes. - When the starting offset is zero, the search for a match starts at the + The subject string is passed to pcre_exec() as a pointer in subject, a + length in length, and a starting byte offset in startoffset. In UTF-8 + mode, the byte offset must point to the start of a UTF-8 character. + Unlike the pattern string, the subject may contain binary zero bytes. + When the starting offset is zero, the search for a match starts at the beginning of the subject, and this is by far the most common case. - A non-zero starting offset is useful when searching for another match - in the same subject by calling pcre_exec() again after a previous suc- - cess. Setting startoffset differs from just passing over a shortened - string and setting PCRE_NOTBOL in the case of a pattern that begins + A non-zero starting offset is useful when searching for another match + in the same subject by calling pcre_exec() again after a previous suc- + cess. Setting startoffset differs from just passing over a shortened + string and setting PCRE_NOTBOL in the case of a pattern that begins with any kind of lookbehind. For example, consider the pattern \Biss\B - which finds occurrences of "iss" in the middle of words. (\B matches - only if the current position in the subject is not a word boundary.) - When applied to the string "Mississipi" the first call to pcre_exec() - finds the first occurrence. If pcre_exec() is called again with just - the remainder of the subject, namely "issipi", it does not match, + which finds occurrences of "iss" in the middle of words. (\B matches + only if the current position in the subject is not a word boundary.) + When applied to the string "Mississipi" the first call to pcre_exec() + finds the first occurrence. If pcre_exec() is called again with just + the remainder of the subject, namely "issipi", it does not match, because \B is always false at the start of the subject, which is deemed - to be a word boundary. However, if pcre_exec() is passed the entire + to be a word boundary. However, if pcre_exec() is passed the entire string again, but with startoffset set to 4, it finds the second occur- - rence of "iss" because it is able to look behind the starting point to + rence of "iss" because it is able to look behind the starting point to discover that it is preceded by a letter. - If a non-zero starting offset is passed when the pattern is anchored, + If a non-zero starting offset is passed when the pattern is anchored, one attempt to match at the given offset is made. This can only succeed - if the pattern does not require the match to be at the start of the + if the pattern does not require the match to be at the start of the subject. How pcre_exec() returns captured substrings - In general, a pattern matches a certain portion of the subject, and in - addition, further substrings from the subject may be picked out by - parts of the pattern. Following the usage in Jeffrey Friedl's book, - this is called "capturing" in what follows, and the phrase "capturing - subpattern" is used for a fragment of a pattern that picks out a sub- - string. PCRE supports several other kinds of parenthesized subpattern + In general, a pattern matches a certain portion of the subject, and in + addition, further substrings from the subject may be picked out by + parts of the pattern. Following the usage in Jeffrey Friedl's book, + this is called "capturing" in what follows, and the phrase "capturing + subpattern" is used for a fragment of a pattern that picks out a sub- + string. PCRE supports several other kinds of parenthesized subpattern that do not cause substrings to be captured. - Captured substrings are returned to the caller via a vector of integer - offsets whose address is passed in ovector. The number of elements in - the vector is passed in ovecsize, which must be a non-negative number. + Captured substrings are returned to the caller via a vector of integer + offsets whose address is passed in ovector. The number of elements in + the vector is passed in ovecsize, which must be a non-negative number. Note: this argument is NOT the size of ovector in bytes. - The first two-thirds of the vector is used to pass back captured sub- - strings, each substring using a pair of integers. The remaining third - of the vector is used as workspace by pcre_exec() while matching cap- - turing subpatterns, and is not available for passing back information. - The length passed in ovecsize should always be a multiple of three. If + The first two-thirds of the vector is used to pass back captured sub- + strings, each substring using a pair of integers. The remaining third + of the vector is used as workspace by pcre_exec() while matching cap- + turing subpatterns, and is not available for passing back information. + The length passed in ovecsize should always be a multiple of three. If it is not, it is rounded down. - When a match is successful, information about captured substrings is - returned in pairs of integers, starting at the beginning of ovector, - and continuing up to two-thirds of its length at the most. The first + When a match is successful, information about captured substrings is + returned in pairs of integers, starting at the beginning of ovector, + and continuing up to two-thirds of its length at the most. The first element of a pair is set to the offset of the first character in a sub- - string, and the second is set to the offset of the first character - after the end of a substring. The first pair, ovector[0] and ovec- - tor[1], identify the portion of the subject string matched by the - entire pattern. The next pair is used for the first capturing subpat- - tern, and so on. The value returned by pcre_exec() is the number of - pairs that have been set. If there are no capturing subpatterns, the - return value from a successful match is 1, indicating that just the + string, and the second is set to the offset of the first character + after the end of a substring. The first pair, ovector[0] and ovec- + tor[1], identify the portion of the subject string matched by the + entire pattern. The next pair is used for the first capturing subpat- + tern, and so on. The value returned by pcre_exec() is the number of + pairs that have been set. If there are no capturing subpatterns, the + return value from a successful match is 1, indicating that just the first pair of offsets has been set. - Some convenience functions are provided for extracting the captured - substrings as separate strings. These are described in the following + Some convenience functions are provided for extracting the captured + substrings as separate strings. These are described in the following section. - It is possible for an capturing subpattern number n+1 to match some - part of the subject when subpattern n has not been used at all. For + It is possible for an capturing subpattern number n+1 to match some + part of the subject when subpattern n has not been used at all. For example, if the string "abc" is matched against the pattern (a|(z))(bc) - subpatterns 1 and 3 are matched, but 2 is not. When this happens, both + subpatterns 1 and 3 are matched, but 2 is not. When this happens, both offset values corresponding to the unused subpattern are set to -1. If a capturing subpattern is matched repeatedly, it is the last portion of the string that it matched that is returned. - If the vector is too small to hold all the captured substring offsets, + If the vector is too small to hold all the captured substring offsets, it is used as far as possible (up to two-thirds of its length), and the - function returns a value of zero. In particular, if the substring off- + function returns a value of zero. In particular, if the substring off- sets are not of interest, pcre_exec() may be called with ovector passed - as NULL and ovecsize as zero. However, if the pattern contains back - references and the ovector is not big enough to remember the related - substrings, PCRE has to get additional memory for use during matching. + as NULL and ovecsize as zero. However, if the pattern contains back + references and the ovector is not big enough to remember the related + substrings, PCRE has to get additional memory for use during matching. Thus it is usually advisable to supply an ovector. - Note that pcre_info() can be used to find out how many capturing sub- + Note that pcre_info() can be used to find out how many capturing sub- patterns there are in a compiled pattern. The smallest size for ovector - that will allow for n captured substrings, in addition to the offsets + that will allow for n captured substrings, in addition to the offsets of the substring matched by the whole pattern, is (n+1)*3. Return values from pcre_exec() - If pcre_exec() fails, it returns a negative number. The following are + If pcre_exec() fails, it returns a negative number. The following are defined in the header file: PCRE_ERROR_NOMATCH (-1) @@ -1336,7 +1655,7 @@ MATCHING A PATTERN PCRE_ERROR_NULL (-2) - Either code or subject was passed as NULL, or ovector was NULL and + Either code or subject was passed as NULL, or ovector was NULL and ovecsize was not zero. PCRE_ERROR_BADOPTION (-3) @@ -1345,74 +1664,74 @@ MATCHING A PATTERN PCRE_ERROR_BADMAGIC (-4) - PCRE stores a 4-byte "magic number" at the start of the compiled code, + PCRE stores a 4-byte "magic number" at the start of the compiled code, to catch the case when it is passed a junk pointer and to detect when a pattern that was compiled in an environment of one endianness is run in - an environment with the other endianness. This is the error that PCRE + an environment with the other endianness. This is the error that PCRE gives when the magic number is not present. PCRE_ERROR_UNKNOWN_NODE (-5) While running the pattern match, an unknown item was encountered in the - compiled pattern. This error could be caused by a bug in PCRE or by + compiled pattern. This error could be caused by a bug in PCRE or by overwriting of the compiled pattern. PCRE_ERROR_NOMEMORY (-6) - If a pattern contains back references, but the ovector that is passed + If a pattern contains back references, but the ovector that is passed to pcre_exec() is not big enough to remember the referenced substrings, - PCRE gets a block of memory at the start of matching to use for this - purpose. If the call via pcre_malloc() fails, this error is given. The + PCRE gets a block of memory at the start of matching to use for this + purpose. If the call via pcre_malloc() fails, this error is given. The memory is automatically freed at the end of matching. PCRE_ERROR_NOSUBSTRING (-7) - This error is used by the pcre_copy_substring(), pcre_get_substring(), + This error is used by the pcre_copy_substring(), pcre_get_substring(), and pcre_get_substring_list() functions (see below). It is never returned by pcre_exec(). PCRE_ERROR_MATCHLIMIT (-8) - The recursion and backtracking limit, as specified by the match_limit - field in a pcre_extra structure (or defaulted) was reached. See the + The recursion and backtracking limit, as specified by the match_limit + field in a pcre_extra structure (or defaulted) was reached. See the description above. PCRE_ERROR_CALLOUT (-9) This error is never generated by pcre_exec() itself. It is provided for - use by callout functions that want to yield a distinctive error code. + use by callout functions that want to yield a distinctive error code. See the pcrecallout documentation for details. PCRE_ERROR_BADUTF8 (-10) - A string that contains an invalid UTF-8 byte sequence was passed as a + A string that contains an invalid UTF-8 byte sequence was passed as a subject. PCRE_ERROR_BADUTF8_OFFSET (-11) The UTF-8 byte sequence that was passed as a subject was valid, but the - value of startoffset did not point to the beginning of a UTF-8 charac- + value of startoffset did not point to the beginning of a UTF-8 charac- ter. - PCRE_ERROR_PARTIAL (-12) + PCRE_ERROR_PARTIAL (-12) - The subject string did not match, but it did match partially. See the + The subject string did not match, but it did match partially. See the pcrepartial documentation for details of partial matching. - PCRE_ERROR_BAD_PARTIAL (-13) + PCRE_ERROR_BADPARTIAL (-13) - The PCRE_PARTIAL option was used with a compiled pattern containing - items that are not supported for partial matching. See the pcrepartial + The PCRE_PARTIAL option was used with a compiled pattern containing + items that are not supported for partial matching. See the pcrepartial documentation for details of partial matching. - PCRE_ERROR_INTERNAL (-14) + PCRE_ERROR_INTERNAL (-14) - An unexpected internal error has occurred. This error could be caused + An unexpected internal error has occurred. This error could be caused by a bug in PCRE or by overwriting of the compiled pattern. - PCRE_ERROR_BADCOUNT (-15) + PCRE_ERROR_BADCOUNT (-15) - This error is given if the value of the ovecsize argument is negative. + This error is given if the value of the ovecsize argument is negative. EXTRACTING CAPTURED SUBSTRINGS BY NUMBER @@ -1428,72 +1747,72 @@ EXTRACTING CAPTURED SUBSTRINGS BY NUMBER int pcre_get_substring_list(const char *subject, int *ovector, int stringcount, const char ***listptr); - Captured substrings can be accessed directly by using the offsets - returned by pcre_exec() in ovector. For convenience, the functions + Captured substrings can be accessed directly by using the offsets + returned by pcre_exec() in ovector. For convenience, the functions pcre_copy_substring(), pcre_get_substring(), and pcre_get_sub- - string_list() are provided for extracting captured substrings as new, - separate, zero-terminated strings. These functions identify substrings - by number. The next section describes functions for extracting named - substrings. A substring that contains a binary zero is correctly - extracted and has a further zero added on the end, but the result is + string_list() are provided for extracting captured substrings as new, + separate, zero-terminated strings. These functions identify substrings + by number. The next section describes functions for extracting named + substrings. A substring that contains a binary zero is correctly + extracted and has a further zero added on the end, but the result is not, of course, a C string. - The first three arguments are the same for all three of these func- - tions: subject is the subject string that has just been successfully + The first three arguments are the same for all three of these func- + tions: subject is the subject string that has just been successfully matched, ovector is a pointer to the vector of integer offsets that was passed to pcre_exec(), and stringcount is the number of substrings that - were captured by the match, including the substring that matched the + were captured by the match, including the substring that matched the entire regular expression. This is the value returned by pcre_exec() if - it is greater than zero. If pcre_exec() returned zero, indicating that - it ran out of space in ovector, the value passed as stringcount should + it is greater than zero. If pcre_exec() returned zero, indicating that + it ran out of space in ovector, the value passed as stringcount should be the number of elements in the vector divided by three. - The functions pcre_copy_substring() and pcre_get_substring() extract a - single substring, whose number is given as stringnumber. A value of - zero extracts the substring that matched the entire pattern, whereas - higher values extract the captured substrings. For pcre_copy_sub- - string(), the string is placed in buffer, whose length is given by - buffersize, while for pcre_get_substring() a new block of memory is - obtained via pcre_malloc, and its address is returned via stringptr. - The yield of the function is the length of the string, not including + The functions pcre_copy_substring() and pcre_get_substring() extract a + single substring, whose number is given as stringnumber. A value of + zero extracts the substring that matched the entire pattern, whereas + higher values extract the captured substrings. For pcre_copy_sub- + string(), the string is placed in buffer, whose length is given by + buffersize, while for pcre_get_substring() a new block of memory is + obtained via pcre_malloc, and its address is returned via stringptr. + The yield of the function is the length of the string, not including the terminating zero, or one of PCRE_ERROR_NOMEMORY (-6) - The buffer was too small for pcre_copy_substring(), or the attempt to + The buffer was too small for pcre_copy_substring(), or the attempt to get memory failed for pcre_get_substring(). PCRE_ERROR_NOSUBSTRING (-7) There is no substring whose number is stringnumber. - The pcre_get_substring_list() function extracts all available sub- - strings and builds a list of pointers to them. All this is done in a + The pcre_get_substring_list() function extracts all available sub- + strings and builds a list of pointers to them. All this is done in a single block of memory that is obtained via pcre_malloc. The address of - the memory block is returned via listptr, which is also the start of - the list of string pointers. The end of the list is marked by a NULL + the memory block is returned via listptr, which is also the start of + the list of string pointers. The end of the list is marked by a NULL pointer. The yield of the function is zero if all went well, or PCRE_ERROR_NOMEMORY (-6) if the attempt to get the memory block failed. - When any of these functions encounter a substring that is unset, which - can happen when capturing subpattern number n+1 matches some part of - the subject, but subpattern n has not been used at all, they return an + When any of these functions encounter a substring that is unset, which + can happen when capturing subpattern number n+1 matches some part of + the subject, but subpattern n has not been used at all, they return an empty string. This can be distinguished from a genuine zero-length sub- - string by inspecting the appropriate offset in ovector, which is nega- + string by inspecting the appropriate offset in ovector, which is nega- tive for unset substrings. - The two convenience functions pcre_free_substring() and pcre_free_sub- - string_list() can be used to free the memory returned by a previous + The two convenience functions pcre_free_substring() and pcre_free_sub- + string_list() can be used to free the memory returned by a previous call of pcre_get_substring() or pcre_get_substring_list(), respec- - tively. They do nothing more than call the function pointed to by - pcre_free, which of course could be called directly from a C program. - However, PCRE is used in some situations where it is linked via a spe- + tively. They do nothing more than call the function pointed to by + pcre_free, which of course could be called directly from a C program. + However, PCRE is used in some situations where it is linked via a spe- cial interface to another programming language which cannot use - pcre_free directly; it is for these cases that the functions are - provided. + pcre_free directly; it is for these cases that the functions are pro- + vided. EXTRACTING CAPTURED SUBSTRINGS BY NAME @@ -1511,47 +1830,217 @@ EXTRACTING CAPTURED SUBSTRINGS BY NAME int stringcount, const char *stringname, const char **stringptr); - To extract a substring by name, you first have to find associated num- + To extract a substring by name, you first have to find associated num- ber. For example, for this pattern (a+)b(?<xxx>\d+)... the number of the subpattern called "xxx" is 2. You can find the number from the name by calling pcre_get_stringnumber(). The first argument is - the compiled pattern, and the second is the name. The yield of the - function is the subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if + the compiled pattern, and the second is the name. The yield of the + function is the subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no subpattern of that name. Given the number, you can extract the substring directly, or use one of the functions described in the previous section. For convenience, there are also two functions that do the whole job. - Most of the arguments of pcre_copy_named_substring() and - pcre_get_named_substring() are the same as those for the similarly - named functions that extract by number. As these are described in the - previous section, they are not re-described here. There are just two + Most of the arguments of pcre_copy_named_substring() and + pcre_get_named_substring() are the same as those for the similarly + named functions that extract by number. As these are described in the + previous section, they are not re-described here. There are just two differences: - First, instead of a substring number, a substring name is given. Sec- + First, instead of a substring number, a substring name is given. Sec- ond, there is an extra argument, given at the start, which is a pointer - to the compiled pattern. This is needed in order to gain access to the + to the compiled pattern. This is needed in order to gain access to the name-to-number translation table. - These functions call pcre_get_stringnumber(), and if it succeeds, they - then call pcre_copy_substring() or pcre_get_substring(), as appropri- + These functions call pcre_get_stringnumber(), and if it succeeds, they + then call pcre_copy_substring() or pcre_get_substring(), as appropri- ate. -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. ------------------------------------------------------------------------------ -PCRE(3) PCRE(3) +FINDING ALL POSSIBLE MATCHES + + The traditional matching function uses a similar algorithm to Perl, + which stops when it finds the first match, starting at a given point in + the subject. If you want to find all possible matches, or the longest + possible match, consider using the alternative matching function (see + below) instead. If you cannot use the alternative function, but still + need to find all possible matches, you can kludge it up by making use + of the callout facility, which is described in the pcrecallout documen- + tation. + + What you have to do is to insert a callout right at the end of the pat- + tern. When your callout function is called, extract and save the cur- + rent matched substring. Then return 1, which forces pcre_exec() to + backtrack and try other alternatives. Ultimately, when it runs out of + matches, pcre_exec() will yield PCRE_ERROR_NOMATCH. + + +MATCHING A PATTERN: THE ALTERNATIVE FUNCTION + + int pcre_dfa_exec(const pcre *code, const pcre_extra *extra, + const char *subject, int length, int startoffset, + int options, int *ovector, int ovecsize, + int *workspace, int wscount); + + The function pcre_dfa_exec() is called to match a subject string + against a compiled pattern, using a "DFA" matching algorithm. This has + different characteristics to the normal algorithm, and is not compati- + ble with Perl. Some of the features of PCRE patterns are not supported. + Nevertheless, there are times when this kind of matching can be useful. + For a discussion of the two matching algorithms, see the pcrematching + documentation. + + The arguments for the pcre_dfa_exec() function are the same as for + pcre_exec(), plus two extras. The ovector argument is used in a differ- + ent way, and this is described below. The other common arguments are + used in the same way as for pcre_exec(), so their description is not + repeated here. + + The two additional arguments provide workspace for the function. The + workspace vector should contain at least 20 elements. It is used for + keeping track of multiple paths through the pattern tree. More + workspace will be needed for patterns and subjects where there are a + lot of possible matches. + + Here is an example of a simple call to pcre_exec(): + + int rc; + int ovector[10]; + int wspace[20]; + rc = pcre_exec( + re, /* result of pcre_compile() */ + NULL, /* we didn't study the pattern */ + "some string", /* the subject string */ + 11, /* the length of the subject string */ + 0, /* start at offset 0 in the subject */ + 0, /* default options */ + ovector, /* vector of integers for substring information */ + 10, /* number of elements (NOT size in bytes) */ + wspace, /* working space vector */ + 20); /* number of elements (NOT size in bytes) */ + + Option bits for pcre_dfa_exec() + + The unused bits of the options argument for pcre_dfa_exec() must be + zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NOTBOL, + PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL, + PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last three of + these are the same as for pcre_exec(), so their description is not + repeated here. + + PCRE_PARTIAL + + This has the same general effect as it does for pcre_exec(), but the + details are slightly different. When PCRE_PARTIAL is set for + pcre_dfa_exec(), the return code PCRE_ERROR_NOMATCH is converted into + PCRE_ERROR_PARTIAL if the end of the subject is reached, there have + been no complete matches, but there is still at least one matching pos- + sibility. The portion of the string that provided the partial match is + set as the first matching string. + + PCRE_DFA_SHORTEST + + Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to + stop as soon as it has found one match. Because of the way the DFA + algorithm works, this is necessarily the shortest possible match at the + first possible matching point in the subject string. + + PCRE_DFA_RESTART + + When pcre_dfa_exec() is called with the PCRE_PARTIAL option, and + returns a partial match, it is possible to call it again, with addi- + tional subject characters, and have it continue with the same match. + The PCRE_DFA_RESTART option requests this action; when it is set, the + workspace and wscount options must reference the same vector as before + because data about the match so far is left in them after a partial + match. There is more discussion of this facility in the pcrepartial + documentation. + + Successful returns from pcre_dfa_exec() + + When pcre_dfa_exec() succeeds, it may have matched more than one sub- + string in the subject. Note, however, that all the matches from one run + of the function start at the same point in the subject. The shorter + matches are all initial substrings of the longer matches. For example, + if the pattern + + <.*> + + is matched against the string + + This is <something> <something else> <something further> no more + + the three matched strings are + + <something> + <something> <something else> + <something> <something else> <something further> + + On success, the yield of the function is a number greater than zero, + which is the number of matched substrings. The substrings themselves + are returned in ovector. Each string uses two elements; the first is + the offset to the start, and the second is the offset to the end. All + the strings have the same start offset. (Space could have been saved by + giving this only once, but it was decided to retain some compatibility + with the way pcre_exec() returns data, even though the meaning of the + strings is different.) + + The strings are returned in reverse order of length; that is, the long- + est matching string is given first. If there were too many matches to + fit into ovector, the yield of the function is zero, and the vector is + filled with the longest matches. + + Error returns from pcre_dfa_exec() + + The pcre_dfa_exec() function returns a negative number when it fails. + Many of the errors are the same as for pcre_exec(), and these are + described above. There are in addition the following errors that are + specific to pcre_dfa_exec(): + + PCRE_ERROR_DFA_UITEM (-16) + + This return is given if pcre_dfa_exec() encounters an item in the pat- + tern that it does not support, for instance, the use of \C or a back + reference. + + PCRE_ERROR_DFA_UCOND (-17) + + This return is given if pcre_dfa_exec() encounters a condition item in + a pattern that uses a back reference for the condition. This is not + supported. + + PCRE_ERROR_DFA_UMLIMIT (-18) + + This return is given if pcre_dfa_exec() is called with an extra block + that contains a setting of the match_limit field. This is not supported + (it is meaningless). + + PCRE_ERROR_DFA_WSSIZE (-19) + + This return is given if pcre_dfa_exec() runs out of space in the + workspace vector. + + PCRE_ERROR_DFA_RECURSE (-20) + + When a recursive subpattern is processed, the matching function calls + itself recursively, using private vectors for ovector and workspace. + This error is given if the output vector is not large enough. This + should be extremely rare, as a vector of size 1000 is used. + +Last updated: 16 May 2005 +Copyright (c) 1997-2005 University of Cambridge. +----------------------------------------------------------------------------- NAME PCRE - Perl-compatible regular expressions + PCRE CALLOUTS int (*pcre_callout)(pcre_callout_block *); @@ -1606,9 +2095,10 @@ MISSING CALLOUTS THE CALLOUT INTERFACE During matching, when PCRE reaches a callout point, the external func- - tion defined by pcre_callout is called (if it is set). The only argu- - ment is a pointer to a pcre_callout block. This structure contains the - following fields: + tion defined by pcre_callout is called (if it is set). This applies to + both the pcre_exec() and the pcre_dfa_exec() matching functions. The + only argument to the callout function is a pointer to a pcre_callout + block. This structure contains the following fields: int version; int callout_number; @@ -1623,87 +2113,91 @@ THE CALLOUT INTERFACE int pattern_position; int next_item_length; - The version field is an integer containing the version number of the - block format. The initial version was 0; the current version is 1. The - version number will change again in future if additional fields are + The version field is an integer containing the version number of the + block format. The initial version was 0; the current version is 1. The + version number will change again in future if additional fields are added, but the intention is never to remove any of the existing fields. - The callout_number field contains the number of the callout, as com- - piled into the pattern (that is, the number after ?C for manual call- + The callout_number field contains the number of the callout, as com- + piled into the pattern (that is, the number after ?C for manual call- outs, and 255 for automatically generated callouts). - The offset_vector field is a pointer to the vector of offsets that was - passed by the caller to pcre_exec(). The contents can be inspected in - order to extract substrings that have been matched so far, in the same - way as for extracting substrings after a match has completed. + The offset_vector field is a pointer to the vector of offsets that was + passed by the caller to pcre_exec() or pcre_dfa_exec(). When + pcre_exec() is used, the contents can be inspected in order to extract + substrings that have been matched so far, in the same way as for + extracting substrings after a match has completed. For pcre_dfa_exec() + this field is not useful. The subject and subject_length fields contain copies of the values that were passed to pcre_exec(). - The start_match field contains the offset within the subject at which - the current match attempt started. If the pattern is not anchored, the + The start_match field contains the offset within the subject at which + the current match attempt started. If the pattern is not anchored, the callout function may be called several times from the same point in the pattern for different starting points in the subject. - The current_position field contains the offset within the subject of + The current_position field contains the offset within the subject of the current match pointer. - The capture_top field contains one more than the number of the highest - numbered captured substring so far. If no substrings have been cap- - tured, the value of capture_top is one. - - The capture_last field contains the number of the most recently cap- - tured substring. If no substrings have been captured, its value is -1. - - The callout_data field contains a value that is passed to pcre_exec() - by the caller specifically so that it can be passed back in callouts. - It is passed in the pcre_callout field of the pcre_extra data struc- - ture. If no such data was passed, the value of callout_data in a - pcre_callout block is NULL. There is a description of the pcre_extra + When the pcre_exec() function is used, the capture_top field contains + one more than the number of the highest numbered captured substring so + far. If no substrings have been captured, the value of capture_top is + one. This is always the case when pcre_dfa_exec() is used, because it + does not support captured substrings. + + The capture_last field contains the number of the most recently cap- + tured substring. If no substrings have been captured, its value is -1. + This is always the case when pcre_dfa_exec() is used. + + The callout_data field contains a value that is passed to pcre_exec() + or pcre_dfa_exec() specifically so that it can be passed back in call- + outs. It is passed in the pcre_callout field of the pcre_extra data + structure. If no such data was passed, the value of callout_data in a + pcre_callout block is NULL. There is a description of the pcre_extra structure in the pcreapi documentation. - The pattern_position field is present from version 1 of the pcre_call- + The pattern_position field is present from version 1 of the pcre_call- out structure. It contains the offset to the next item to be matched in the pattern string. - The next_item_length field is present from version 1 of the pcre_call- + The next_item_length field is present from version 1 of the pcre_call- out structure. It contains the length of the next item to be matched in - the pattern string. When the callout immediately precedes an alterna- - tion bar, a closing parenthesis, or the end of the pattern, the length - is zero. When the callout precedes an opening parenthesis, the length + the pattern string. When the callout immediately precedes an alterna- + tion bar, a closing parenthesis, or the end of the pattern, the length + is zero. When the callout precedes an opening parenthesis, the length is that of the entire subpattern. - The pattern_position and next_item_length fields are intended to help - in distinguishing between different automatic callouts, which all have + The pattern_position and next_item_length fields are intended to help + in distinguishing between different automatic callouts, which all have the same callout number. However, they are set for all callouts. RETURN VALUES - The external callout function returns an integer to PCRE. If the value - is zero, matching proceeds as normal. If the value is greater than - zero, matching fails at the current point, but backtracking to test - other matching possibilities goes ahead, just as if a lookahead asser- - tion had failed. If the value is less than zero, the match is aban- - doned, and pcre_exec() returns the negative value. + The external callout function returns an integer to PCRE. If the value + is zero, matching proceeds as normal. If the value is greater than + zero, matching fails at the current point, but the testing of other + matching possibilities goes ahead, just as if a lookahead assertion had + failed. If the value is less than zero, the match is abandoned, and + pcre_exec() (or pcre_dfa_exec()) returns the negative value. - Negative values should normally be chosen from the set of + Negative values should normally be chosen from the set of PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a stan- - dard "no match" failure. The error number PCRE_ERROR_CALLOUT is - reserved for use by callout functions; it will never be used by PCRE + dard "no match" failure. The error number PCRE_ERROR_CALLOUT is + reserved for use by callout functions; it will never be used by PCRE itself. -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. ----------------------------------------------------------------------------- -PCRE(3) PCRE(3) - NAME PCRE - Perl-compatible regular expressions + DIFFERENCES BETWEEN PCRE AND PERL This document describes the differences in the ways that PCRE and Perl @@ -1808,17 +2302,19 @@ DIFFERENCES BETWEEN PCRE AND PERL (m) Patterns compiled by PCRE can be saved and re-used at a later time, even on different hosts that have the other endianness. -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. ------------------------------------------------------------------------------ + (n) The alternative matching function (pcre_dfa_exec()) matches in a + different way and is not Perl-compatible. -PCRE(3) PCRE(3) +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. +----------------------------------------------------------------------------- NAME PCRE - Perl-compatible regular expressions + PCRE REGULAR EXPRESSION DETAILS The syntax and semantics of the regular expressions supported by PCRE @@ -1836,6 +2332,14 @@ PCRE REGULAR EXPRESSION DETAILS of UTF-8 features in the section on UTF-8 support in the main pcre page. + The remainder of this document discusses the patterns that are sup- + ported by PCRE when its main matching function, pcre_exec(), is used. + From release 6.0, PCRE offers a second matching function, + pcre_dfa_exec(), which matches using a different algorithm that is not + Perl-compatible. The advantages and disadvantages of the alternative + function, and how it differs from the normal function, are discussed in + the pcrematching page. + A regular expression is a pattern that is matched against a subject string from left to right. Most characters stand for themselves in a pattern, and match the corresponding characters in the subject. As a @@ -1843,15 +2347,24 @@ PCRE REGULAR EXPRESSION DETAILS The quick brown fox - matches a portion of a subject string that is identical to itself. The - power of regular expressions comes from the ability to include alterna- - tives and repetitions in the pattern. These are encoded in the pattern - by the use of metacharacters, which do not stand for themselves but - instead are interpreted in some special way. - - There are two different sets of metacharacters: those that are recog- - nized anywhere in the pattern except within square brackets, and those - that are recognized in square brackets. Outside square brackets, the + matches a portion of a subject string that is identical to itself. When + caseless matching is specified (the PCRE_CASELESS option), letters are + matched independently of case. In UTF-8 mode, PCRE always understands + the concept of case for characters whose values are less than 128, so + caseless matching is always possible. For characters with higher val- + ues, the concept of case is supported if PCRE is compiled with Unicode + property support, but not otherwise. If you want to use caseless + matching for characters 128 and above, you must ensure that PCRE is + compiled with Unicode property support as well as with UTF-8 support. + + The power of regular expressions comes from the ability to include + alternatives and repetitions in the pattern. These are encoded in the + pattern by the use of metacharacters, which do not stand for themselves + but instead are interpreted in some special way. + + There are two different sets of metacharacters: those that are recog- + nized anywhere in the pattern except within square brackets, and those + that are recognized in square brackets. Outside square brackets, the metacharacters are as follows: \ general escape character with several uses @@ -1870,7 +2383,7 @@ PCRE REGULAR EXPRESSION DETAILS also "possessive quantifier" { start min/max quantifier - Part of a pattern that is in square brackets is called a "character + Part of a pattern that is in square brackets is called a "character class". In a character class the only metacharacters are: \ general escape character @@ -1880,33 +2393,33 @@ PCRE REGULAR EXPRESSION DETAILS syntax) ] terminates the character class - The following sections describe the use of each of the metacharacters. + The following sections describe the use of each of the metacharacters. BACKSLASH The backslash character has several uses. Firstly, if it is followed by - a non-alphanumeric character, it takes away any special meaning that - character may have. This use of backslash as an escape character + a non-alphanumeric character, it takes away any special meaning that + character may have. This use of backslash as an escape character applies both inside and outside character classes. - For example, if you want to match a * character, you write \* in the - pattern. This escaping action applies whether or not the following - character would otherwise be interpreted as a metacharacter, so it is - always safe to precede a non-alphanumeric with backslash to specify - that it stands for itself. In particular, if you want to match a back- + For example, if you want to match a * character, you write \* in the + pattern. This escaping action applies whether or not the following + character would otherwise be interpreted as a metacharacter, so it is + always safe to precede a non-alphanumeric with backslash to specify + that it stands for itself. In particular, if you want to match a back- slash, you write \\. - If a pattern is compiled with the PCRE_EXTENDED option, whitespace in - the pattern (other than in a character class) and characters between a + If a pattern is compiled with the PCRE_EXTENDED option, whitespace in + the pattern (other than in a character class) and characters between a # outside a character class and the next newline character are ignored. - An escaping backslash can be used to include a whitespace or # charac- + An escaping backslash can be used to include a whitespace or # charac- ter as part of the pattern. - If you want to remove the special meaning from a sequence of charac- - ters, you can do so by putting them between \Q and \E. This is differ- - ent from Perl in that $ and @ are handled as literals in \Q...\E - sequences in PCRE, whereas in Perl, $ and @ cause variable interpola- + If you want to remove the special meaning from a sequence of charac- + ters, you can do so by putting them between \Q and \E. This is differ- + ent from Perl in that $ and @ are handled as literals in \Q...\E + sequences in PCRE, whereas in Perl, $ and @ cause variable interpola- tion. Note the following examples: Pattern PCRE matches Perl matches @@ -1916,16 +2429,16 @@ BACKSLASH \Qabc\$xyz\E abc\$xyz abc\$xyz \Qabc\E\$\Qxyz\E abc$xyz abc$xyz - The \Q...\E sequence is recognized both inside and outside character + The \Q...\E sequence is recognized both inside and outside character classes. Non-printing characters A second use of backslash provides a way of encoding non-printing char- - acters in patterns in a visible manner. There is no restriction on the - appearance of non-printing characters, apart from the binary zero that - terminates a pattern, but when a pattern is being prepared by text - editing, it is usually easier to use one of the following escape + acters in patterns in a visible manner. There is no restriction on the + appearance of non-printing characters, apart from the binary zero that + terminates a pattern, but when a pattern is being prepared by text + editing, it is usually easier to use one of the following escape sequences than the binary character it represents: \a alarm, that is, the BEL character (hex 07) @@ -1939,44 +2452,44 @@ BACKSLASH \xhh character with hex code hh \x{hhh..} character with hex code hhh... (UTF-8 mode only) - The precise effect of \cx is as follows: if x is a lower case letter, - it is converted to upper case. Then bit 6 of the character (hex 40) is - inverted. Thus \cz becomes hex 1A, but \c{ becomes hex 3B, while \c; + The precise effect of \cx is as follows: if x is a lower case letter, + it is converted to upper case. Then bit 6 of the character (hex 40) is + inverted. Thus \cz becomes hex 1A, but \c{ becomes hex 3B, while \c; becomes hex 7B. - After \x, from zero to two hexadecimal digits are read (letters can be - in upper or lower case). In UTF-8 mode, any number of hexadecimal dig- - its may appear between \x{ and }, but the value of the character code - must be less than 2**31 (that is, the maximum hexadecimal value is - 7FFFFFFF). If characters other than hexadecimal digits appear between - \x{ and }, or if there is no terminating }, this form of escape is not - recognized. Instead, the initial \x will be interpreted as a basic hex- - adecimal escape, with no following digits, giving a character whose + After \x, from zero to two hexadecimal digits are read (letters can be + in upper or lower case). In UTF-8 mode, any number of hexadecimal dig- + its may appear between \x{ and }, but the value of the character code + must be less than 2**31 (that is, the maximum hexadecimal value is + 7FFFFFFF). If characters other than hexadecimal digits appear between + \x{ and }, or if there is no terminating }, this form of escape is not + recognized. Instead, the initial \x will be interpreted as a basic + hexadecimal escape, with no following digits, giving a character whose value is zero. Characters whose value is less than 256 can be defined by either of the - two syntaxes for \x when PCRE is in UTF-8 mode. There is no difference - in the way they are handled. For example, \xdc is exactly the same as + two syntaxes for \x when PCRE is in UTF-8 mode. There is no difference + in the way they are handled. For example, \xdc is exactly the same as \x{dc}. - After \0 up to two further octal digits are read. In both cases, if - there are fewer than two digits, just those that are present are used. - Thus the sequence \0\x\07 specifies two binary zeros followed by a BEL - character (code value 7). Make sure you supply two digits after the - initial zero if the pattern character that follows is itself an octal + After \0 up to two further octal digits are read. In both cases, if + there are fewer than two digits, just those that are present are used. + Thus the sequence \0\x\07 specifies two binary zeros followed by a BEL + character (code value 7). Make sure you supply two digits after the + initial zero if the pattern character that follows is itself an octal digit. The handling of a backslash followed by a digit other than 0 is compli- cated. Outside a character class, PCRE reads it and any following dig- - its as a decimal number. If the number is less than 10, or if there + its as a decimal number. If the number is less than 10, or if there have been at least that many previous capturing left parentheses in the - expression, the entire sequence is taken as a back reference. A - description of how this works is given later, following the discussion + expression, the entire sequence is taken as a back reference. A + description of how this works is given later, following the discussion of parenthesized subpatterns. - Inside a character class, or if the decimal number is greater than 9 - and there have not been that many capturing subpatterns, PCRE re-reads - up to three octal digits following the backslash, and generates a sin- + Inside a character class, or if the decimal number is greater than 9 + and there have not been that many capturing subpatterns, PCRE re-reads + up to three octal digits following the backslash, and generates a sin- gle byte from the least significant 8 bits of the value. Any subsequent digits stand for themselves. For example: @@ -1995,19 +2508,19 @@ BACKSLASH \81 is either a back reference, or a binary zero followed by the two characters "8" and "1" - Note that octal values of 100 or greater must not be introduced by a + Note that octal values of 100 or greater must not be introduced by a leading zero, because no more than three octal digits are ever read. - All the sequences that define a single byte value or a single UTF-8 + All the sequences that define a single byte value or a single UTF-8 character (in UTF-8 mode) can be used both inside and outside character - classes. In addition, inside a character class, the sequence \b is + classes. In addition, inside a character class, the sequence \b is interpreted as the backspace character (hex 08), and the sequence \X is - interpreted as the character "X". Outside a character class, these + interpreted as the character "X". Outside a character class, these sequences have different meanings (see below). Generic character types - The third use of backslash is for specifying generic character types. + The third use of backslash is for specifying generic character types. The following are always recognized: \d any decimal digit @@ -2018,48 +2531,48 @@ BACKSLASH \W any "non-word" character Each pair of escape sequences partitions the complete set of characters - into two disjoint sets. Any given character matches one, and only one, + into two disjoint sets. Any given character matches one, and only one, of each pair. These character type sequences can appear both inside and outside char- - acter classes. They each match one character of the appropriate type. - If the current matching point is at the end of the subject string, all + acter classes. They each match one character of the appropriate type. + If the current matching point is at the end of the subject string, all of them fail, since there is no character to match. - For compatibility with Perl, \s does not match the VT character (code - 11). This makes it different from the the POSIX "space" class. The \s + For compatibility with Perl, \s does not match the VT character (code + 11). This makes it different from the the POSIX "space" class. The \s characters are HT (9), LF (10), FF (12), CR (13), and space (32). A "word" character is an underscore or any character less than 256 that - is a letter or digit. The definition of letters and digits is con- - trolled by PCRE's low-valued character tables, and may vary if locale- - specific matching is taking place (see "Locale support" in the pcreapi - page). For example, in the "fr_FR" (French) locale, some character - codes greater than 128 are used for accented letters, and these are + is a letter or digit. The definition of letters and digits is con- + trolled by PCRE's low-valued character tables, and may vary if locale- + specific matching is taking place (see "Locale support" in the pcreapi + page). For example, in the "fr_FR" (French) locale, some character + codes greater than 128 are used for accented letters, and these are matched by \w. - In UTF-8 mode, characters with values greater than 128 never match \d, + In UTF-8 mode, characters with values greater than 128 never match \d, \s, or \w, and always match \D, \S, and \W. This is true even when Uni- code character property support is available. Unicode character properties When PCRE is built with Unicode character property support, three addi- - tional escape sequences to match generic character types are available + tional escape sequences to match generic character types are available when UTF-8 mode is selected. They are: \p{xx} a character with the xx property \P{xx} a character without the xx property \X an extended Unicode sequence - The property names represented by xx above are limited to the Unicode - general category properties. Each character has exactly one such prop- - erty, specified by a two-letter abbreviation. For compatibility with - Perl, negation can be specified by including a circumflex between the - opening brace and the property name. For example, \p{^Lu} is the same + The property names represented by xx above are limited to the Unicode + general category properties. Each character has exactly one such prop- + erty, specified by a two-letter abbreviation. For compatibility with + Perl, negation can be specified by including a circumflex between the + opening brace and the property name. For example, \p{^Lu} is the same as \P{Lu}. - If only one letter is specified with \p or \P, it includes all the + If only one letter is specified with \p or \P, it includes all the properties that start with that letter. In this case, in the absence of negation, the curly brackets in the escape sequence are optional; these two examples have the same effect: @@ -2113,33 +2626,33 @@ BACKSLASH Zp Paragraph separator Zs Space separator - Extended properties such as "Greek" or "InMusicalSymbols" are not sup- + Extended properties such as "Greek" or "InMusicalSymbols" are not sup- ported by PCRE. - Specifying caseless matching does not affect these escape sequences. + Specifying caseless matching does not affect these escape sequences. For example, \p{Lu} always matches only upper case letters. - The \X escape matches any number of Unicode characters that form an + The \X escape matches any number of Unicode characters that form an extended Unicode sequence. \X is equivalent to (?>\PM\pM*) - That is, it matches a character without the "mark" property, followed - by zero or more characters with the "mark" property, and treats the - sequence as an atomic group (see below). Characters with the "mark" + That is, it matches a character without the "mark" property, followed + by zero or more characters with the "mark" property, and treats the + sequence as an atomic group (see below). Characters with the "mark" property are typically accents that affect the preceding character. - Matching characters by Unicode property is not fast, because PCRE has - to search a structure that contains data for over fifteen thousand + Matching characters by Unicode property is not fast, because PCRE has + to search a structure that contains data for over fifteen thousand characters. That is why the traditional escape sequences such as \d and \w do not use Unicode properties in PCRE. Simple assertions The fourth use of backslash is for certain simple assertions. An asser- - tion specifies a condition that has to be met at a particular point in - a match, without consuming any characters from the subject string. The - use of subpatterns for more complicated assertions is described below. + tion specifies a condition that has to be met at a particular point in + a match, without consuming any characters from the subject string. The + use of subpatterns for more complicated assertions is described below. The backslashed assertions are: \b matches at a word boundary @@ -2149,42 +2662,42 @@ BACKSLASH \z matches at end of subject \G matches at first matching position in subject - These assertions may not appear in character classes (but note that \b + These assertions may not appear in character classes (but note that \b has a different meaning, namely the backspace character, inside a char- acter class). - A word boundary is a position in the subject string where the current - character and the previous character do not both match \w or \W (i.e. - one matches \w and the other matches \W), or the start or end of the + A word boundary is a position in the subject string where the current + character and the previous character do not both match \w or \W (i.e. + one matches \w and the other matches \W), or the start or end of the string if the first or last character matches \w, respectively. - The \A, \Z, and \z assertions differ from the traditional circumflex + The \A, \Z, and \z assertions differ from the traditional circumflex and dollar (described in the next section) in that they only ever match - at the very start and end of the subject string, whatever options are - set. Thus, they are independent of multiline mode. These three asser- + at the very start and end of the subject string, whatever options are + set. Thus, they are independent of multiline mode. These three asser- tions are not affected by the PCRE_NOTBOL or PCRE_NOTEOL options, which - affect only the behaviour of the circumflex and dollar metacharacters. - However, if the startoffset argument of pcre_exec() is non-zero, indi- + affect only the behaviour of the circumflex and dollar metacharacters. + However, if the startoffset argument of pcre_exec() is non-zero, indi- cating that matching is to start at a point other than the beginning of - the subject, \A can never match. The difference between \Z and \z is - that \Z matches before a newline that is the last character of the - string as well as at the end of the string, whereas \z matches only at + the subject, \A can never match. The difference between \Z and \z is + that \Z matches before a newline that is the last character of the + string as well as at the end of the string, whereas \z matches only at the end. - The \G assertion is true only when the current matching position is at - the start point of the match, as specified by the startoffset argument - of pcre_exec(). It differs from \A when the value of startoffset is - non-zero. By calling pcre_exec() multiple times with appropriate argu- + The \G assertion is true only when the current matching position is at + the start point of the match, as specified by the startoffset argument + of pcre_exec(). It differs from \A when the value of startoffset is + non-zero. By calling pcre_exec() multiple times with appropriate argu- ments, you can mimic Perl's /g option, and it is in this kind of imple- mentation where \G can be useful. - Note, however, that PCRE's interpretation of \G, as the start of the + Note, however, that PCRE's interpretation of \G, as the start of the current match, is subtly different from Perl's, which defines it as the - end of the previous match. In Perl, these can be different when the - previously matched string was empty. Because PCRE does just one match + end of the previous match. In Perl, these can be different when the + previously matched string was empty. Because PCRE does just one match at a time, it cannot reproduce this behaviour. - If all the alternatives of a pattern begin with \G, the expression is + If all the alternatives of a pattern begin with \G, the expression is anchored to the starting match position, and the "anchored" flag is set in the compiled regular expression. @@ -2192,73 +2705,73 @@ BACKSLASH CIRCUMFLEX AND DOLLAR Outside a character class, in the default matching mode, the circumflex - character is an assertion that is true only if the current matching - point is at the start of the subject string. If the startoffset argu- - ment of pcre_exec() is non-zero, circumflex can never match if the - PCRE_MULTILINE option is unset. Inside a character class, circumflex + character is an assertion that is true only if the current matching + point is at the start of the subject string. If the startoffset argu- + ment of pcre_exec() is non-zero, circumflex can never match if the + PCRE_MULTILINE option is unset. Inside a character class, circumflex has an entirely different meaning (see below). - Circumflex need not be the first character of the pattern if a number - of alternatives are involved, but it should be the first thing in each - alternative in which it appears if the pattern is ever to match that - branch. If all possible alternatives start with a circumflex, that is, - if the pattern is constrained to match only at the start of the sub- - ject, it is said to be an "anchored" pattern. (There are also other + Circumflex need not be the first character of the pattern if a number + of alternatives are involved, but it should be the first thing in each + alternative in which it appears if the pattern is ever to match that + branch. If all possible alternatives start with a circumflex, that is, + if the pattern is constrained to match only at the start of the sub- + ject, it is said to be an "anchored" pattern. (There are also other constructs that can cause a pattern to be anchored.) - A dollar character is an assertion that is true only if the current - matching point is at the end of the subject string, or immediately + A dollar character is an assertion that is true only if the current + matching point is at the end of the subject string, or immediately before a newline character that is the last character in the string (by - default). Dollar need not be the last character of the pattern if a - number of alternatives are involved, but it should be the last item in - any branch in which it appears. Dollar has no special meaning in a + default). Dollar need not be the last character of the pattern if a + number of alternatives are involved, but it should be the last item in + any branch in which it appears. Dollar has no special meaning in a character class. - The meaning of dollar can be changed so that it matches only at the - very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at + The meaning of dollar can be changed so that it matches only at the + very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This does not affect the \Z assertion. The meanings of the circumflex and dollar characters are changed if the PCRE_MULTILINE option is set. When this is the case, they match immedi- - ately after and immediately before an internal newline character, - respectively, in addition to matching at the start and end of the sub- - ject string. For example, the pattern /^abc$/ matches the subject - string "def\nabc" (where \n represents a newline character) in multi- + ately after and immediately before an internal newline character, + respectively, in addition to matching at the start and end of the sub- + ject string. For example, the pattern /^abc$/ matches the subject + string "def\nabc" (where \n represents a newline character) in multi- line mode, but not otherwise. Consequently, patterns that are anchored - in single line mode because all branches start with ^ are not anchored - in multiline mode, and a match for circumflex is possible when the - startoffset argument of pcre_exec() is non-zero. The PCRE_DOL- + in single line mode because all branches start with ^ are not anchored + in multiline mode, and a match for circumflex is possible when the + startoffset argument of pcre_exec() is non-zero. The PCRE_DOL- LAR_ENDONLY option is ignored if PCRE_MULTILINE is set. - Note that the sequences \A, \Z, and \z can be used to match the start - and end of the subject in both modes, and if all branches of a pattern - start with \A it is always anchored, whether PCRE_MULTILINE is set or + Note that the sequences \A, \Z, and \z can be used to match the start + and end of the subject in both modes, and if all branches of a pattern + start with \A it is always anchored, whether PCRE_MULTILINE is set or not. FULL STOP (PERIOD, DOT) Outside a character class, a dot in the pattern matches any one charac- - ter in the subject, including a non-printing character, but not (by - default) newline. In UTF-8 mode, a dot matches any UTF-8 character, + ter in the subject, including a non-printing character, but not (by + default) newline. In UTF-8 mode, a dot matches any UTF-8 character, which might be more than one byte long, except (by default) newline. If - the PCRE_DOTALL option is set, dots match newlines as well. The han- - dling of dot is entirely independent of the handling of circumflex and - dollar, the only relationship being that they both involve newline + the PCRE_DOTALL option is set, dots match newlines as well. The han- + dling of dot is entirely independent of the handling of circumflex and + dollar, the only relationship being that they both involve newline characters. Dot has no special meaning in a character class. MATCHING A SINGLE BYTE Outside a character class, the escape sequence \C matches any one byte, - both in and out of UTF-8 mode. Unlike a dot, it can match a newline. - The feature is provided in Perl in order to match individual bytes in - UTF-8 mode. Because it breaks up UTF-8 characters into individual - bytes, what remains in the string may be a malformed UTF-8 string. For + both in and out of UTF-8 mode. Unlike a dot, it can match a newline. + The feature is provided in Perl in order to match individual bytes in + UTF-8 mode. Because it breaks up UTF-8 characters into individual + bytes, what remains in the string may be a malformed UTF-8 string. For this reason, the \C escape sequence is best avoided. - PCRE does not allow \C to appear in lookbehind assertions (described - below), because in UTF-8 mode this would make it impossible to calcu- + PCRE does not allow \C to appear in lookbehind assertions (described + below), because in UTF-8 mode this would make it impossible to calcu- late the length of the lookbehind. @@ -2267,35 +2780,40 @@ SQUARE BRACKETS AND CHARACTER CLASSES An opening square bracket introduces a character class, terminated by a closing square bracket. A closing square bracket on its own is not spe- cial. If a closing square bracket is required as a member of the class, - it should be the first data character in the class (after an initial + it should be the first data character in the class (after an initial circumflex, if present) or escaped with a backslash. - A character class matches a single character in the subject. In UTF-8 - mode, the character may occupy more than one byte. A matched character + A character class matches a single character in the subject. In UTF-8 + mode, the character may occupy more than one byte. A matched character must be in the set of characters defined by the class, unless the first - character in the class definition is a circumflex, in which case the - subject character must not be in the set defined by the class. If a - circumflex is actually required as a member of the class, ensure it is + character in the class definition is a circumflex, in which case the + subject character must not be in the set defined by the class. If a + circumflex is actually required as a member of the class, ensure it is not the first character, or escape it with a backslash. - For example, the character class [aeiou] matches any lower case vowel, - while [^aeiou] matches any character that is not a lower case vowel. + For example, the character class [aeiou] matches any lower case vowel, + while [^aeiou] matches any character that is not a lower case vowel. Note that a circumflex is just a convenient notation for specifying the - characters that are in the class by enumerating those that are not. A - class that starts with a circumflex is not an assertion: it still con- - sumes a character from the subject string, and therefore it fails if + characters that are in the class by enumerating those that are not. A + class that starts with a circumflex is not an assertion: it still con- + sumes a character from the subject string, and therefore it fails if the current pointer is at the end of the string. - In UTF-8 mode, characters with values greater than 255 can be included - in a class as a literal string of bytes, or by using the \x{ escaping + In UTF-8 mode, characters with values greater than 255 can be included + in a class as a literal string of bytes, or by using the \x{ escaping mechanism. - When caseless matching is set, any letters in a class represent both - their upper case and lower case versions, so for example, a caseless - [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not - match "A", whereas a caseful version would. When running in UTF-8 mode, - PCRE supports the concept of case for characters with values greater - than 128 only when it is compiled with Unicode property support. + When caseless matching is set, any letters in a class represent both + their upper case and lower case versions, so for example, a caseless + [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not + match "A", whereas a caseful version would. In UTF-8 mode, PCRE always + understands the concept of case for characters whose values are less + than 128, so caseless matching is always possible. For characters with + higher values, the concept of case is supported if PCRE is compiled + with Unicode property support, but not otherwise. If you want to use + caseless matching for characters 128 and above, you must ensure that + PCRE is compiled with Unicode property support as well as with UTF-8 + support. The newline character is never treated in any special way in character classes, whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE @@ -3215,24 +3733,23 @@ CALLOUTS gether. A complete description of the interface to the callout function is given in the pcrecallout documentation. -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. ----------------------------------------------------------------------------- -PCRE(3) PCRE(3) - NAME PCRE - Perl-compatible regular expressions + PARTIAL MATCHING IN PCRE In normal use of PCRE, if the subject string that is passed to - pcre_exec() matches as far as it goes, but is too short to match the - entire pattern, PCRE_ERROR_NOMATCH is returned. There are circumstances - where it might be helpful to distinguish this case from other cases in - which there is no match. + pcre_exec() or pcre_dfa_exec() matches as far as it goes, but is too + short to match the entire pattern, PCRE_ERROR_NOMATCH is returned. + There are circumstances where it might be helpful to distinguish this + case from other cases in which there is no match. Consider, for example, an application where a human is required to type in data for a field with specific formatting requirements. An example @@ -3248,11 +3765,19 @@ PARTIAL MATCHING IN PCRE until the entire string has been entered. PCRE supports the concept of partial matching by means of the PCRE_PAR- - TIAL option, which can be set when calling pcre_exec(). When this is - done, the return code PCRE_ERROR_NOMATCH is converted into - PCRE_ERROR_PARTIAL if at any time during the matching process the - entire subject string matched part of the pattern. No captured data is - set when this occurs. + TIAL option, which can be set when calling pcre_exec() or + pcre_dfa_exec(). When this flag is set for pcre_exec(), the return code + PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if at any time + during the matching process the last part of the subject string matched + part of the pattern. Unfortunately, for non-anchored matching, it is + not possible to obtain the position of the start of the partial match. + No captured data is set when PCRE_ERROR_PARTIAL is returned. + + When PCRE_PARTIAL is set for pcre_dfa_exec(), the return code + PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end of + the subject is reached, there have been no complete matches, but there + is still at least one matching possibility. The portion of the string + that provided the partial match is set as the first matching string. Using PCRE_PARTIAL disables one of PCRE's optimizations. PCRE remembers the last literal byte in a pattern, and abandons matching immediately @@ -3263,8 +3788,9 @@ PARTIAL MATCHING IN PCRE RESTRICTED PATTERNS FOR PCRE_PARTIAL Because of the way certain internal optimizations are implemented in - PCRE, the PCRE_PARTIAL option cannot be used with all patterns. - Repeated single characters such as + the pcre_exec() function, the PCRE_PARTIAL option cannot be used with + all patterns. These restrictions do not apply when pcre_dfa_exec() is + used. For pcre_exec(), repeated single characters such as a{2,4} @@ -3272,26 +3798,26 @@ RESTRICTED PATTERNS FOR PCRE_PARTIAL \d+ - are not permitted if the maximum number of occurrences is greater than + are not permitted if the maximum number of occurrences is greater than one. Optional items such as \d? (where the maximum is one) are permit- - ted. Quantifiers with any values are permitted after parentheses, so + ted. Quantifiers with any values are permitted after parentheses, so the invalid examples above can be coded thus: (a){2,4} (\d)+ - These constructions run more slowly, but for the kinds of application - that are envisaged for this facility, this is not felt to be a major + These constructions run more slowly, but for the kinds of application + that are envisaged for this facility, this is not felt to be a major restriction. - If PCRE_PARTIAL is set for a pattern that does not conform to the - restrictions, pcre_exec() returns the error code PCRE_ERROR_BADPARTIAL + If PCRE_PARTIAL is set for a pattern that does not conform to the + restrictions, pcre_exec() returns the error code PCRE_ERROR_BADPARTIAL (-13). EXAMPLE OF PARTIAL MATCHING USING PCRETEST - If the escape sequence \P is present in a pcretest data line, the + If the escape sequence \P is present in a pcretest data line, the PCRE_PARTIAL flag is used for the match. Here is a run of pcretest that uses the date example quoted above: @@ -3308,21 +3834,103 @@ EXAMPLE OF PARTIAL MATCHING USING PCRETEST data> jP No match - The first data string is matched completely, so pcretest shows the - matched substrings. The remaining four strings do not match the com- - plete pattern, but the first two are partial matches. + The first data string is matched completely, so pcretest shows the + matched substrings. The remaining four strings do not match the com- + plete pattern, but the first two are partial matches. The same test, + using DFA matching (by means of the \D escape sequence), produces the + following output: -Last updated: 08 September 2004 -Copyright (c) 1997-2004 University of Cambridge. ------------------------------------------------------------------------------ + re> /^?(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)$/ + data> 25jun04\P\D + 0: 25jun04 + data> 23dec3\P\D + Partial match: 23dec3 + data> 3ju\P\D + Partial match: 3ju + data> 3juj\P\D + No match + data> j\P\D + No match + + Notice that in this case the portion of the string that was matched is + made available. + + +MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() + + When a partial match has been found using pcre_dfa_exec(), it is possi- + ble to continue the match by providing additional subject data and + calling pcre_dfa_exec() again with the PCRE_DFA_RESTART option and the + same working space (where details of the previous partial match are + stored). Here is an example using pcretest, where the \R escape + sequence sets the PCRE_DFA_RESTART option and the \D escape sequence + requests the use of pcre_dfa_exec(): + + re> /^?(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)$/ + data> 23ja\P\D + Partial match: 23ja + data> n05\R\D + 0: n05 + + The first call has "23ja" as the subject, and requests partial match- + ing; the second call has "n05" as the subject for the continued + (restarted) match. Notice that when the match is complete, only the + last part is shown; PCRE does not retain the previously partially- + matched string. It is up to the calling program to do that if it needs + to. + + This facility can be used to pass very long subject strings to + pcre_dfa_exec(). However, some care is needed for certain types of pat- + tern. + + 1. If the pattern contains tests for the beginning or end of a line, + you need to pass the PCRE_NOTBOL or PCRE_NOTEOL options, as appropri- + ate, when the subject string for any call does not contain the begin- + ning or end of a line. + + 2. If the pattern contains backward assertions (including \b or \B), + you need to arrange for some overlap in the subject strings to allow + for this. For example, you could pass the subject in chunks that were + 500 bytes long, but in a buffer of 700 bytes, with the starting offset + set to 200 and the previous 200 bytes at the start of the buffer. + + 3. Matching a subject string that is split into multiple segments does + not always produce exactly the same result as matching over one single + long string. The difference arises when there are multiple matching + possibilities, because a partial match result is given only when there + are no completed matches in a call to fBpcre_dfa_exec(). This means + that as soon as the shortest match has been found, continuation to a + new subject segment is no longer possible. Consider this pcretest + example: -PCRE(3) PCRE(3) + re> /dog(sbody)?/ + data> do\P\D + Partial match: do + data> gsb\R\P\D + 0: g + data> dogsbody\D + 0: dogsbody + 1: dog + + The pattern matches the words "dog" or "dogsbody". When the subject is + presented in several parts ("do" and "gsb" being the first two) the + match stops when "dog" has been found, and it is not possible to con- + tinue. On the other hand, if "dogsbody" is presented as a single + string, both matches are found. + + Because of this phenomenon, it does not usually make sense to end a + pattern that is going to be matched in this way with a variable repeat. + +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. +----------------------------------------------------------------------------- NAME PCRE - Perl-compatible regular expressions + SAVING AND RE-USING PRECOMPILED PCRE PATTERNS If you are running an application that uses a large number of regular @@ -3391,16 +3999,18 @@ SAVING A COMPILED PATTERN RE-USING A PRECOMPILED PATTERN Re-using a precompiled pattern is straightforward. Having reloaded it - into main memory, you pass its pointer to pcre_exec() in the usual way. - This should work even on another host, and even if that host has the - opposite endianness to the one where the pattern was compiled. - - However, if you passed a pointer to custom character tables when the - pattern was compiled (the tableptr argument of pcre_compile()), you - must now pass a similar pointer to pcre_exec(), because the value saved - with the compiled pattern will obviously be nonsense. A field in a - pcre_extra() block is used to pass this data, as described in the sec- - tion on matching a pattern in the pcreapi documentation. + into main memory, you pass its pointer to pcre_exec() or + pcre_dfa_exec() in the usual way. This should work even on another + host, and even if that host has the opposite endianness to the one + where the pattern was compiled. + + However, if you passed a pointer to custom character tables when the + pattern was compiled (the tableptr argument of pcre_compile()), you + must now pass a similar pointer to pcre_exec() or pcre_dfa_exec(), + because the value saved with the compiled pattern will obviously be + nonsense. A field in a pcre_extra() block is used to pass this data, as + described in the section on matching a pattern in the pcreapi documen- + tation. If you did not provide custom character tables when the pattern was compiled, the pointer in the compiled pattern is NULL, which causes @@ -3411,29 +4021,29 @@ RE-USING A PRECOMPILED PATTERN your own pcre_extra data block and set the study_data field to point to the reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in the flags field to indicate that study data is present. Then - pass the pcre_extra block to pcre_exec() in the usual way. + pass the pcre_extra block to pcre_exec() or pcre_dfa_exec() in the + usual way. COMPATIBILITY WITH DIFFERENT PCRE RELEASES - The layout of the control block that is at the start of the data that - makes up a compiled pattern was changed for release 5.0. If you have - any saved patterns that were compiled with previous releases (not a - facility that was previously advertised), you will have to recompile - them for release 5.0. However, from now on, it should be possible to + The layout of the control block that is at the start of the data that + makes up a compiled pattern was changed for release 5.0. If you have + any saved patterns that were compiled with previous releases (not a + facility that was previously advertised), you will have to recompile + them for release 5.0. However, from now on, it should be possible to make changes in a compabible manner. -Last updated: 10 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. ----------------------------------------------------------------------------- -PCRE(3) PCRE(3) - NAME PCRE - Perl-compatible regular expressions + PCRE PERFORMANCE Certain items that may appear in regular expression patterns are more @@ -3469,9 +4079,9 @@ PCRE PERFORMANCE If you are using such a pattern with subject strings that do not con- tain newlines, the best performance is obtained by setting PCRE_DOTALL, - or starting the pattern with ^.* to indicate explicit anchoring. That - saves PCRE from having to scan along the subject looking for a newline - to restart at. + or starting the pattern with ^.* or ^.*? to indicate explicit anchor- + ing. That saves PCRE from having to scan along the subject looking for + a newline to restart at. Beware of patterns that contain nested indefinite repeats. These can take a long time to run when applied to a string that does not match. @@ -3492,9 +4102,9 @@ PCRE PERFORMANCE (a+)*b where a literal character follows. Before embarking on the standard - matching procedure, PCRE checks that there is a "b" later in the - subject string, and if there is not, it fails the match immediately. - However, when there is no following literal this optimization cannot be + matching procedure, PCRE checks that there is a "b" later in the sub- + ject string, and if there is not, it fails the match immediately. How- + ever, when there is no following literal this optimization cannot be used. You can see the difference by comparing the behaviour of (a+)*\d @@ -3506,17 +4116,16 @@ PCRE PERFORMANCE In many cases, the solution to this kind of performance issue is to use an atomic group or a possessive quantifier. -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. ----------------------------------------------------------------------------- -PCRE(3) PCRE(3) - NAME PCRE - Perl-compatible regular expressions. + SYNOPSIS OF POSIX API #include <pcreposix.h> @@ -3537,7 +4146,7 @@ DESCRIPTION This set of functions provides a POSIX-style API to the PCRE regular expression package. See the pcreapi documentation for a description of - PCRE's native API, which contains additional functionality. + PCRE's native API, which contains much additional functionality. The functions described here are just wrapper functions that ultimately call the PCRE native API. Their prototypes are defined in the @@ -3581,6 +4190,12 @@ COMPILING A PATTERN The argument cflags is either zero, or contains one or more of the bits defined by the following macros: + REG_DOTALL + + The PCRE_DOTALL option is set when the expression is passed for compi- + lation to the native function. Note that REG_DOTALL is not part of the + POSIX standard. + REG_ICASE The PCRE_CASELESS option is set when the expression is passed for com- @@ -3692,21 +4307,235 @@ MEMORY USAGE AUTHOR - Philip Hazel <ph10@cam.ac.uk> + Philip Hazel University Computing Service, Cambridge CB2 3QG, England. -Last updated: 07 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. ----------------------------------------------------------------------------- -PCRE(3) PCRE(3) + + +NAME + PCRE - Perl-compatible regular expressions. + + +SYNOPSIS OF C++ WRAPPER + + #include <pcrecpp.h> + + +DESCRIPTION + + The C++ wrapper for PCRE was provided by Google Inc. This brief man + page was constructed from the notes in the pcrecpp.h file, which should + be consulted for further details. + + +MATCHING INTERFACE + + The "FullMatch" operation checks that supplied text matches a supplied + pattern exactly. If pointer arguments are supplied, it copies matched + sub-strings that match sub-patterns into them. + + Example: successful match + pcrecpp::RE re("h.*o"); + re.FullMatch("hello"); + + Example: unsuccessful match (requires full match): + pcrecpp::RE re("e"); + !re.FullMatch("hello"); + + Example: creating a temporary RE object: + pcrecpp::RE("h.*o").FullMatch("hello"); + + You can pass in a "const char*" or a "string" for "text". The examples + below tend to use a const char*. You can, as in the different examples + above, store the RE object explicitly in a variable or use a temporary + RE object. The examples below use one mode or the other arbitrarily. + Either could correctly be used for any of these examples. + + You must supply extra pointer arguments to extract matched subpieces. + + Example: extracts "ruby" into "s" and 1234 into "i" + int i; + string s; + pcrecpp::RE re("(\\w+):(\\d+)"); + re.FullMatch("ruby:1234", &s, &i); + + Example: does not try to extract any extra sub-patterns + re.FullMatch("ruby:1234", &s); + + Example: does not try to extract into NULL + re.FullMatch("ruby:1234", NULL, &i); + + Example: integer overflow causes failure + !re.FullMatch("ruby:1234567891234", NULL, &i); + + Example: fails because there aren't enough sub-patterns: + !pcrecpp::RE("\\w+:\\d+").FullMatch("ruby:1234", &s); + + Example: fails because string cannot be stored in integer + !pcrecpp::RE("(.*)").FullMatch("ruby", &i); + + The provided pointer arguments can be pointers to any scalar numeric + type, or one of: + + string (matched piece is copied to string) + StringPiece (StringPiece is mutated to point to matched piece) + T (where "bool T::ParseFrom(const char*, int)" exists) + NULL (the corresponding matched sub-pattern is not copied) + + The function returns true iff all of the following conditions are sat- + isfied: + + a. "text" matches "pattern" exactly; + + b. The number of matched sub-patterns is >= number of supplied + pointers; + + c. The "i"th argument has a suitable type for holding the + string captured as the "i"th sub-pattern. If you pass in + NULL for the "i"th argument, or pass fewer arguments than + number of sub-patterns, "i"th captured sub-pattern is + ignored. + + The matching interface supports at most 16 arguments per call. If you + need more, consider using the more general interface + pcrecpp::RE::DoMatch. See pcrecpp.h for the signature for DoMatch. + + +PARTIAL MATCHES + + You can use the "PartialMatch" operation when you want the pattern to + match any substring of the text. + + Example: simple search for a string: + pcrecpp::RE("ell").PartialMatch("hello"); + + Example: find first number in a string: + int number; + pcrecpp::RE re("(\\d+)"); + re.PartialMatch("x*100 + 20", &number); + assert(number == 100); + + +UTF-8 AND THE MATCHING INTERFACE + + By default, pattern and text are plain text, one byte per character. + The UTF8 flag, passed to the constructor, causes both pattern and + string to be treated as UTF-8 text, still a byte stream but potentially + multiple bytes per character. In practice, the text is likelier to be + UTF-8 than the pattern, but the match returned may depend on the UTF8 + flag, so always use it when matching UTF8 text. For example, "." will + match one byte normally but with UTF8 set may match up to three bytes + of a multi-byte character. + + Example: + pcrecpp::RE_Options options; + options.set_utf8(); + pcrecpp::RE re(utf8_pattern, options); + re.FullMatch(utf8_string); + + Example: using the convenience function UTF8(): + pcrecpp::RE re(utf8_pattern, pcrecpp::UTF8()); + re.FullMatch(utf8_string); + + NOTE: The UTF8 flag is ignored if pcre was not configured with the + --enable-utf8 flag. + + +SCANNING TEXT INCREMENTALLY + + The "Consume" operation may be useful if you want to repeatedly match + regular expressions at the front of a string and skip over them as they + match. This requires use of the "StringPiece" type, which represents a + sub-range of a real string. Like RE, StringPiece is defined in the + pcrecpp namespace. + + Example: read lines of the form "var = value" from a string. + string contents = ...; // Fill string somehow + pcrecpp::StringPiece input(contents); // Wrap in a StringPiece + + string var; + int value; + pcrecpp::RE re("(\\w+) = (\\d+)\n"); + while (re.Consume(&input, &var, &value)) { + ...; + } + + Each successful call to "Consume" will set "var/value", and also + advance "input" so it points past the matched text. + + The "FindAndConsume" operation is similar to "Consume" but does not + anchor your match at the beginning of the string. For example, you + could extract all words from a string by repeatedly calling + + pcrecpp::RE("(\\w+)").FindAndConsume(&input, &word) + + +PARSING HEX/OCTAL/C-RADIX NUMBERS + + By default, if you pass a pointer to a numeric value, the corresponding + text is interpreted as a base-10 number. You can instead wrap the + pointer with a call to one of the operators Hex(), Octal(), or CRadix() + to interpret the text in another base. The CRadix operator interprets + C-style "0" (base-8) and "0x" (base-16) prefixes, but defaults to + base-10. + + Example: + int a, b, c, d; + pcrecpp::RE re("(.*) (.*) (.*) (.*)"); + re.FullMatch("100 40 0100 0x40", + pcrecpp::Octal(&a), pcrecpp::Hex(&b), + pcrecpp::CRadix(&c), pcrecpp::CRadix(&d)); + + will leave 64 in a, b, c, and d. + + +REPLACING PARTS OF STRINGS + + You can replace the first match of "pattern" in "str" with "rewrite". + Within "rewrite", backslash-escaped digits (\1 to \9) can be used to + insert text matching corresponding parenthesized group from the pat- + tern. \0 in "rewrite" refers to the entire matching text. For example: + + string s = "yabba dabba doo"; + pcrecpp::RE("b+").Replace("d", &s); + + will leave "s" containing "yada dabba doo". The result is true if the + pattern matches and a replacement occurs, false otherwise. + + GlobalReplace is like Replace except that it replaces all occurrences + of the pattern in the string with the rewrite. Replacements are not + subject to re-matching. For example: + + string s = "yabba dabba doo"; + pcrecpp::RE("b+").GlobalReplace("d", &s); + + will leave "s" containing "yada dada doo". It returns the number of + replacements made. + + Extract is like Replace, except that if the pattern matches, "rewrite" + is copied into "out" (an additional argument) with substitutions. The + non-matching portions of "text" are ignored. Returns true iff a match + occurred and the extraction happened successfully; if no match occurs, + the string is left unaffected. + + +AUTHOR + + The C++ wrapper was contributed by Google Inc. + Copyright (c) 2005 Google Inc. +----------------------------------------------------------------------------- NAME PCRE - Perl-compatible regular expressions + PCRE SAMPLE PROGRAM A simple, complete demonstration program, to get you started with using diff --git a/doc/pcre_compile.3 b/doc/pcre_compile.3 index 37a5f2d..4d4cb5a 100644 --- a/doc/pcre_compile.3 +++ b/doc/pcre_compile.3 @@ -38,6 +38,7 @@ The option bits are: PCRE_EXTENDED Ignore whitespace and # comments PCRE_EXTRA PCRE extra features (not much use currently) + PCRE_FIRSTLINE Force matching to be before newline PCRE_MULTILINE ^ and $ match newlines within data PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- theses (named ones available) diff --git a/doc/pcre_compile2.3 b/doc/pcre_compile2.3 new file mode 100644 index 0000000..1c2d704 --- /dev/null +++ b/doc/pcre_compile2.3 @@ -0,0 +1,70 @@ +.TH PCRE 3 +.SH NAME +PCRE - Perl-compatible regular expressions +.SH SYNOPSIS +.rs +.sp +.B #include <pcre.h> +.PP +.SM +.br +.B pcre *pcre_compile2(const char *\fIpattern\fP, int \fIoptions\fP, +.ti +5n +.B int *\fIerrorcodeptr\fP, +.ti +5n +.B const char **\fIerrptr\fP, int *\fIerroffset\fP, +.ti +5n +.B const unsigned char *\fItableptr\fP); +. +.SH DESCRIPTION +.rs +.sp +This function compiles a regular expression into an internal form. It is the +same as \fBpcre_compile()\fP, except for the addition of the \fIerrorcodeptr\fP +argument. The arguments are: + +.sp + \fIpattern\fR A zero-terminated string containing the + regular expression to be compiled + \fIoptions\fR Zero or more option bits + \fIerrorcodeptr\fP Where to put an error code + \fIerrptr\fR Where to put an error message + \fIerroffset\fR Offset in pattern where error was found + \fItableptr\fR Pointer to character tables, or NULL to + use the built-in default +.sp +The option bits are: +.sp + PCRE_ANCHORED Force pattern anchoring + PCRE_AUTO_CALLOUT Compile automatic callouts + PCRE_CASELESS Do caseless matching + PCRE_DOLLAR_ENDONLY $ not to match newline at end + PCRE_DOTALL . matches anything including NL + PCRE_EXTENDED Ignore whitespace and # comments + PCRE_EXTRA PCRE extra features + (not much use currently) + PCRE_FIRSTLINE Force matching to be before newline + PCRE_MULTILINE ^ and $ match newlines within data + PCRE_NO_AUTO_CAPTURE Disable numbered capturing paren- + theses (named ones available) + PCRE_UNGREEDY Invert greediness of quantifiers + PCRE_UTF8 Run in UTF-8 mode + PCRE_NO_UTF8_CHECK Do not check the pattern for UTF-8 + validity (only relevant if + PCRE_UTF8 is set) +.sp +PCRE must be built with UTF-8 support in order to use PCRE_UTF8 and +PCRE_NO_UTF8_CHECK. +.P +The yield of the function is a pointer to a private data structure that +contains the compiled pattern, or NULL if an error was detected. +.P +There is a complete description of the PCRE native API in the +.\" HREF +\fBpcreapi\fR +.\" +page and a description of the POSIX API in the +.\" HREF +\fBpcreposix\fR +.\" +page. diff --git a/doc/pcre_dfa_exec.3 b/doc/pcre_dfa_exec.3 new file mode 100644 index 0000000..277c2e1 --- /dev/null +++ b/doc/pcre_dfa_exec.3 @@ -0,0 +1,80 @@ +.TH PCRE 3 +.SH NAME +PCRE - Perl-compatible regular expressions +.SH SYNOPSIS +.rs +.sp +.B #include <pcre.h> +.PP +.SM +.br +.B int pcre_dfa_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," +.ti +5n +.B "const char *\fIsubject\fP," int \fIlength\fP, int \fIstartoffset\fP, +.ti +5n +.B int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP, +.ti +5n +.B int *\fIworkspace\fP, int \fIwscount\fP); +. +.SH DESCRIPTION +.rs +.sp +This function matches a compiled regular expression against a given subject +string, using a DFA matching algorithm (\fInot\fP Perl-compatible). Note that +the main, Perl-compatible, matching function is \fBpcre_exec()\fP. The +arguments for this function are: +.sp + \fIcode\fP Points to the compiled pattern + \fIextra\fP Points to an associated \fBpcre_extra\fP structure, + or is NULL + \fIsubject\fP Points to the subject string + \fIlength\fP Length of the subject string, in bytes + \fIstartoffset\fP Offset in bytes in the subject at which to + start matching + \fIoptions\fP Option bits + \fIovector\fP Points to a vector of ints for result offsets + \fIovecsize\fP Number of elements in the vector + \fIworkspace\fP Points to a vector of ints used as working space + \fIwscount\fP Number of elements in the vector +.sp +The options are: +.sp + PCRE_ANCHORED Match only at the first position + PCRE_NOTBOL Subject is not the beginning of a line + PCRE_NOTEOL Subject is not the end of a line + PCRE_NOTEMPTY An empty string is not a valid match + PCRE_NO_UTF8_CHECK Do not check the subject for UTF-8 + validity (only relevant if PCRE_UTF8 + was set at compile time) + PCRE_PARTIAL Return PCRE_ERROR_PARTIAL for a partial match + PCRE_DFA_SHORTEST Return only the shortest match + PCRE_DFA_RESTART This is a restart after a partial match +.sp +There are restrictions on what may appear in a pattern when matching using the +DFA algorithm is requested. Details are given in the +.\" HREF +\fBpcrematching\fP +.\" +documentation. +.P +A \fBpcre_extra\fP structure contains the following fields: +.sp + \fIflags\fP Bits indicating which fields are set + \fIstudy_data\fP Opaque data from \fBpcre_study()\fP + \fImatch_limit\fP Limit on internal recursion + \fIcallout_data\fP Opaque data passed back to callouts + \fItables\fP Points to character tables or is NULL +.sp +The flag bits are PCRE_EXTRA_STUDY_DATA, PCRE_EXTRA_MATCH_LIMIT, +PCRE_EXTRA_CALLOUT_DATA, and PCRE_EXTRA_TABLES. For DFA matching, the +\fImatch_limit\fP field is not used, and must not be set. +.P +There is a complete description of the PCRE native API in the +.\" HREF +\fBpcreapi\fP +.\" +page and a description of the POSIX API in the +.\" HREF +\fBpcreposix\fP +.\" +page. diff --git a/doc/pcre_exec.3 b/doc/pcre_exec.3 index 7e071a9..113a632 100644 --- a/doc/pcre_exec.3 +++ b/doc/pcre_exec.3 @@ -18,7 +18,8 @@ PCRE - Perl-compatible regular expressions .rs .sp This function matches a compiled regular expression against a given subject -string, and returns offsets to capturing subexpressions. Its arguments are: +string, using a matching algorithm that is similar to Perl's. It returns +offsets to captured substrings. Its arguments are: .sp \fIcode\fP Points to the compiled pattern \fIextra\fP Points to an associated \fBpcre_extra\fP structure, diff --git a/doc/pcre_refcount.3 b/doc/pcre_refcount.3 new file mode 100644 index 0000000..0b94068 --- /dev/null +++ b/doc/pcre_refcount.3 @@ -0,0 +1,33 @@ +.TH PCRE 3 +.SH NAME +PCRE - Perl-compatible regular expressions +.SH SYNOPSIS +.rs +.sp +.B #include <pcre.h> +.PP +.SM +.br +.B int pcre_info(pcre *\fIcode\fP, int \fIadjust\fP); +. +.SH DESCRIPTION +.rs +.sp +This function is used to maintain a reference count inside a data block that +contains a compiled pattern. Its arguments are: +.sp + \fIcode\fP Compiled regular expression + \fIadjust\fP Adjustment to reference value +.sp +The yield of the function is the adjusted reference value, which is constrained +to lie between 0 and 65535. +.P +There is a complete description of the PCRE native API in the +.\" HREF +\fBpcreapi\fP +.\" +page and a description of the POSIX API in the +.\" HREF +\fBpcreposix\fP +.\" +page. diff --git a/doc/pcreapi.3 b/doc/pcreapi.3 index 42a4e59..9fc8182 100644 --- a/doc/pcreapi.3 +++ b/doc/pcreapi.3 @@ -15,6 +15,15 @@ PCRE - Perl-compatible regular expressions .B const unsigned char *\fItableptr\fP); .PP .br +.B pcre *pcre_compile2(const char *\fIpattern\fP, int \fIoptions\fP, +.ti +5n +.B int *\fIerrorcodeptr\fP, +.ti +5n +.B const char **\fIerrptr\fP, int *\fIerroffset\fP, +.ti +5n +.B const unsigned char *\fItableptr\fP); +.PP +.br .B pcre_extra *pcre_study(const pcre *\fIcode\fP, int \fIoptions\fP, .ti +5n .B const char **\fIerrptr\fP); @@ -27,6 +36,15 @@ PCRE - Perl-compatible regular expressions .B int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP); .PP .br +.B int pcre_dfa_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," +.ti +5n +.B "const char *\fIsubject\fP," int \fIlength\fP, int \fIstartoffset\fP, +.ti +5n +.B int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP, +.ti +5n +.B int *\fIworkspace\fP, int \fIwscount\fP); +.PP +.br .B int pcre_copy_named_substring(const pcre *\fIcode\fP, .ti +5n .B const char *\fIsubject\fP, int *\fIovector\fP, @@ -87,6 +105,9 @@ PCRE - Perl-compatible regular expressions .B *\fIfirstcharptr\fP); .PP .br +.B int pcre_refcount(pcre *\fIcode\fP, int \fIadjust\fP); +.PP +.br .B int pcre_config(int \fIwhat\fP, void *\fIwhere\fP); .PP .br @@ -111,33 +132,50 @@ PCRE - Perl-compatible regular expressions .SH "PCRE API OVERVIEW" .rs .sp -PCRE has its own native API, which is described in this document. There is also -a set of wrapper functions that correspond to the POSIX regular expression API. -These are described in the +PCRE has its own native API, which is described in this document. There is +also a set of wrapper functions that correspond to the POSIX regular expression +API. These are described in the .\" HREF \fBpcreposix\fP .\" -documentation. +documentation. Both of these APIs define a set of C function calls. A C++ +wrapper is distributed with PCRE. It is documented in the +.\" HREF +\fBpcrecpp\fP +.\" +page. .P -The native API function prototypes are defined in the header file \fBpcre.h\fP, -and on Unix systems the library itself is called \fBlibpcre\fP. It can -normally be accessed by adding \fB-lpcre\fP to the command for linking an -application that uses PCRE. The header file defines the macros PCRE_MAJOR and -PCRE_MINOR to contain the major and minor release numbers for the library. +The native API C function prototypes are defined in the header file +\fBpcre.h\fP, and on Unix systems the library itself is called \fBlibpcre\fP. +It can normally be accessed by adding \fB-lpcre\fP to the command for linking +an application that uses PCRE. The header file defines the macros PCRE_MAJOR +and PCRE_MINOR to contain the major and minor release numbers for the library. Applications can use these to include support for different releases of PCRE. .P -The functions \fBpcre_compile()\fP, \fBpcre_study()\fP, and \fBpcre_exec()\fP -are used for compiling and matching regular expressions. A sample program that -demonstrates the simplest way of using them is provided in the file called -\fIpcredemo.c\fP in the source distribution. The +The functions \fBpcre_compile()\fP, \fBpcre_compile2()\fP, \fBpcre_study()\fP, +and \fBpcre_exec()\fP are used for compiling and matching regular expressions +in a Perl-compatible manner. A sample program that demonstrates the simplest +way of using them is provided in the file called \fIpcredemo.c\fP in the source +distribution. The .\" HREF \fBpcresample\fP .\" documentation describes how to run it. .P +A second matching function, \fBpcre_dfa_exec()\fP, which is not +Perl-compatible, is also provided. This uses a different algorithm for the +matching. This allows it to find all possible matches (at a given point in the +subject), not just one. However, this algorithm does not return captured +substrings. A description of the two matching algorithms and their advantages +and disadvantages is given in the +.\" HREF +\fBpcrematching\fP +.\" +documentation. +.P In addition to the main compiling and matching functions, there are convenience -functions for extracting captured substrings from a matched subject string. -They are: +functions for extracting captured substrings from a subject string that is +matched by \fBpcre_exec()\fP. They are: .sp \fBpcre_copy_substring()\fP \fBpcre_copy_named_substring()\fP @@ -150,10 +188,10 @@ They are: provided, to free the memory used for extracted strings. .P The function \fBpcre_maketables()\fP is used to build a set of character tables -in the current locale for passing to \fBpcre_compile()\fP or \fBpcre_exec()\fP. -This is an optional facility that is provided for specialist use. Most -commonly, no special tables are passed, in which case internal tables that are -generated when PCRE is built are used. +in the current locale for passing to \fBpcre_compile()\fP, \fBpcre_exec()\fP, +or \fBpcre_dfa_exec()\fP. This is an optional facility that is provided for +specialist use. Most commonly, no special tables are passed, in which case +internal tables that are generated when PCRE is built are used. .P The function \fBpcre_fullinfo()\fP is used to find out information about a compiled pattern; \fBpcre_info()\fP is an obsolete version that returns only @@ -161,6 +199,10 @@ some of the available information, but is retained for backwards compatibility. The function \fBpcre_version()\fP returns a pointer to a string containing the version of PCRE and its date of release. .P +The function \fBpcre_refcount()\fP maintains a reference count in a data block +containing a compiled pattern. This is provided for the benefit of +object-oriented applications. +.P The global variables \fBpcre_malloc\fP and \fBpcre_free\fP initially contain the entry points of the standard \fBmalloc()\fP and \fBfree()\fP functions, respectively. PCRE calls the memory management functions via these variables, @@ -170,12 +212,13 @@ should be done before calling any PCRE functions. The global variables \fBpcre_stack_malloc\fP and \fBpcre_stack_free\fP are also indirections to memory management functions. These special functions are used only when PCRE is compiled to use the heap for remembering data, instead of -recursive function calls. This is a non-standard way of building PCRE, for use -in environments that have limited stacks. Because of the greater use of memory -management, it runs more slowly. Separate functions are provided so that -special-purpose external code can be used for this case. When used, these -functions are always called in a stack-like manner (last obtained, first -freed), and always for memory blocks of the same size. +recursive function calls, when running the \fBpcre_exec()\fP function. This is +a non-standard way of building PCRE, for use in environments that have limited +stacks. Because of the greater use of memory management, it runs more slowly. +Separate functions are provided so that special-purpose external code can be +used for this case. When used, these functions are always called in a +stack-like manner (last obtained, first freed), and always for memory blocks of +the same size. .P The global variable \fBpcre_callout\fP initially contains NULL. It can be set by the caller to a "callout" function, which PCRE will then call at specified @@ -268,12 +311,13 @@ details are given with \fBpcre_exec()\fP below. .sp PCRE_CONFIG_STACKRECURSE .sp -The output is an integer that is set to one if internal recursion is -implemented by recursive function calls that use the stack to remember their -state. This is the usual way that PCRE is compiled. The output is zero if PCRE -was compiled to use blocks of data on the heap instead of recursive function -calls. In this case, \fBpcre_stack_malloc\fP and \fBpcre_stack_free\fP are -called to manage memory blocks on the heap, thus avoiding the use of the stack. +The output is an integer that is set to one if internal recursion when running +\fBpcre_exec()\fP is implemented by recursive function calls that use the stack +to remember their state. This is the usual way that PCRE is compiled. The +output is zero if PCRE was compiled to use blocks of data on the heap instead +of recursive function calls. In this case, \fBpcre_stack_malloc\fP and +\fBpcre_stack_free\fP are called to manage memory blocks on the heap, thus +avoiding the use of the stack. . . .SH "COMPILING A PATTERN" @@ -284,14 +328,26 @@ called to manage memory blocks on the heap, thus avoiding the use of the stack. .B const char **\fIerrptr\fP, int *\fIerroffset\fP, .ti +5n .B const unsigned char *\fItableptr\fP); +.sp +.B pcre *pcre_compile2(const char *\fIpattern\fP, int \fIoptions\fP, +.ti +5n +.B int *\fIerrorcodeptr\fP, +.ti +5n +.B const char **\fIerrptr\fP, int *\fIerroffset\fP, +.ti +5n +.B const unsigned char *\fItableptr\fP); +.P +Either of the functions \fBpcre_compile()\fP or \fBpcre_compile2()\fP can be +called to compile a pattern into an internal form. The only difference between +the two interfaces is that \fBpcre_compile2()\fP has an additional argument, +\fIerrorcodeptr\fP, via which a numerical error code can be returned. .P -The function \fBpcre_compile()\fP is called to compile a pattern into an -internal form. The pattern is a C string terminated by a binary zero, and -is passed in the \fIpattern\fP argument. A pointer to a single block of memory -that is obtained via \fBpcre_malloc\fP is returned. This contains the compiled -code and related data. The \fBpcre\fP type is defined for the returned block; -this is a typedef for a structure whose contents are not externally defined. It -is up to the caller to free the memory when it is no longer required. +The pattern is a C string terminated by a binary zero, and is passed in the +\fIpattern\fP argument. A pointer to a single block of memory that is obtained +via \fBpcre_malloc\fP is returned. This contains the compiled code and related +data. The \fBpcre\fP type is defined for the returned block; this is a typedef +for a structure whose contents are not externally defined. It is up to the +caller to free the memory when it is no longer required. .P Although the compiled code of a PCRE regex is relocatable, that is, it does not depend on memory location, the complete \fBpcre\fP data block is not @@ -318,6 +374,11 @@ error message. The offset from the start of the pattern to the character where the error was discovered is placed in the variable pointed to by \fIerroffset\fP, which must not be NULL. If it is, an immediate error is given. .P +If \fBpcre_compile2()\fP is used instead of \fBpcre_compile()\fP, and the +\fIerrorcodeptr\fP argument is not NULL, a non-zero error code number is +returned via this argument in the event of an error. This is in addition to the +textual error message. Error codes and messages are listed below. +.P If the final argument, \fItableptr\fP, is NULL, PCRE uses a default set of character tables that are built when PCRE is compiled, using the default C locale. Otherwise, \fItableptr\fP must be an address that is the result of a @@ -362,9 +423,13 @@ documentation. .sp If this bit is set, letters in the pattern match both upper and lower case letters. It is equivalent to Perl's /i option, and it can be changed within a -pattern by a (?i) option setting. When running in UTF-8 mode, case support for -high-valued characters is available only when PCRE is built with Unicode -character property support. +pattern by a (?i) option setting. In UTF-8 mode, PCRE always understands the +concept of case for characters whose values are less than 128, so caseless +matching is always possible. For characters with higher values, the concept of +case is supported if PCRE is compiled with Unicode property support, but not +otherwise. If you want to use caseless matching for characters 128 and above, +you must ensure that PCRE is compiled with Unicode property support as well as +with UTF-8 support. .sp PCRE_DOLLAR_ENDONLY .sp @@ -408,6 +473,12 @@ special meaning is treated as a literal. There are at present no other features controlled by this option. It can also be set by a (?X) option setting within a pattern. .sp + PCRE_FIRSTLINE +.sp +If this option is set, an unanchored pattern is required to match before or at +the first newline character in the subject string, though the matched text may +continue over the newline. +.sp PCRE_MULTILINE .sp By default, PCRE treats the subject string as consisting of a single line of @@ -463,14 +534,72 @@ automatically checked. If an invalid UTF-8 sequence of bytes is found, valid, and you want to skip this check for performance reasons, you can set the PCRE_NO_UTF8_CHECK option. When it is set, the effect of passing an invalid UTF-8 string as a pattern is undefined. It may cause your program to crash. -Note that this option can also be passed to \fBpcre_exec()\fP, to suppress the -UTF-8 validity checking of subject strings. +Note that this option can also be passed to \fBpcre_exec()\fP and +\fBpcre_dfa_exec()\fP, to suppress the UTF-8 validity checking of subject +strings. +. +. +.SH "COMPILATION ERROR CODES" +.rs +.sp +The following table lists the error codes than may be returned by +\fBpcre_compile2()\fP, along with the error messages that may be returned by +both compiling functions. +.sp + 0 no error + 1 \e at end of pattern + 2 \ec at end of pattern + 3 unrecognized character follows \e + 4 numbers out of order in {} quantifier + 5 number too big in {} quantifier + 6 missing terminating ] for character class + 7 invalid escape sequence in character class + 8 range out of order in character class + 9 nothing to repeat + 10 operand of unlimited repeat could match the empty string + 11 internal error: unexpected repeat + 12 unrecognized character after (? + 13 POSIX named classes are supported only within a class + 14 missing ) + 15 reference to non-existent subpattern + 16 erroffset passed as NULL + 17 unknown option bit(s) set + 18 missing ) after comment + 19 parentheses nested too deeply + 20 regular expression too large + 21 failed to get memory + 22 unmatched parentheses + 23 internal error: code overflow + 24 unrecognized character after (?< + 25 lookbehind assertion is not fixed length + 26 malformed number after (?( + 27 conditional group contains more than two branches + 28 assertion expected after (?( + 29 (?R or (?digits must be followed by ) + 30 unknown POSIX class name + 31 POSIX collating elements are not supported + 32 this version of PCRE is not compiled with PCRE_UTF8 support + 33 spare error + 34 character value in \ex{...} sequence is too large + 35 invalid condition (?(0) + 36 \eC not allowed in lookbehind assertion + 37 PCRE does not support \eL, \el, \eN, \eU, or \eu + 38 number after (?C is > 255 + 39 closing ) for (?C expected + 40 recursive call could loop indefinitely + 41 unrecognized character after (?P + 42 syntax error after (?P + 43 two named groups have the same name + 44 invalid UTF-8 string + 45 support for \eP, \ep, and \eX has not been compiled + 46 malformed \eP or \ep sequence + 47 unknown property name after \eP or \ep . . .SH "STUDYING A PATTERN" .rs .sp -.B pcre_extra *pcre_study(const pcre *\fIcode\fP, int \fIoptions\fP, +.B pcre_extra *pcre_study(const pcre *\fIcode\fP, int \fIoptions\fP .ti +5n .B const char **\fIerrptr\fP); .PP @@ -492,7 +621,7 @@ below .\" in the section on matching a pattern. .P -If studying the pattern does not produce any additional information, +If studying the pattern does not produce any additional information \fBpcre_study()\fP returns NULL. In that circumstance, if the calling program wants to pass any of the other fields to \fBpcre_exec()\fP, it must set up its own \fBpcre_extra\fP block. @@ -523,12 +652,12 @@ bytes is created. .SH "LOCALE SUPPORT" .rs .sp -PCRE handles caseless matching, and determines whether characters are letters, +PCRE handles caseless matching, and determines whether characters are letters digits, or whatever, by reference to a set of tables, indexed by character -value. (When running in UTF-8 mode, this applies only to characters with codes +value. When running in UTF-8 mode, this applies only to characters with codes less than 128. Higher-valued codes never match escapes such as \ew or \ed, but can be tested with \ep if PCRE is built with Unicode character property -support.) +support. .P An internal set of tables is created in the default C locale when PCRE is built. This is used when the final argument of \fBpcre_compile()\fP is NULL, @@ -615,7 +744,7 @@ no back references. Return the number of capturing subpatterns in the pattern. The fourth argument should point to an \fBint\fP variable. .sp - PCRE_INFO_DEFAULTTABLES + PCRE_INFO_DEFAULT_TABLES .sp Return a pointer to the internal default character tables within PCRE. The fourth argument should point to an \fBunsigned char *\fP variable. This @@ -760,7 +889,30 @@ it is used to pass back information about the first character of any matched string (see PCRE_INFO_FIRSTBYTE above). . . -.SH "MATCHING A PATTERN" +.SH "REFERENCE COUNTS" +.rs +.sp +.B int pcre_refcount(pcre *\fIcode\fP, int \fIadjust\fP); +.PP +The \fBpcre_refcount()\fP function is used to maintain a reference count in the +data block that contains a compiled pattern. It is provided for the benefit of +applications that operate in an object-oriented manner, where different parts +of the application may be using the same compiled pattern, but you want to free +the block when they are all done. +.P +When a pattern is compiled, the reference count field is initialized to zero. +It is changed only by calling this function, whose action is to add the +\fIadjust\fP value (which may be positive or negative) to it. The yield of the +function is the new value. However, the value of the count is constrained to +lie between 0 and 65535, inclusive. If the new value is outside these limits, +it is forced to the appropriate limit value. +.P +Except when it is zero, the reference count is not correctly preserved if a +pattern is compiled on one host and then transferred to a host whose byte-order +is different. (This seems a highly unlikely scenario.) +. +. +.SH "MATCHING A PATTERN: THE TRADITIONAL FUNCTION" .rs .sp .B int pcre_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," @@ -772,7 +924,14 @@ string (see PCRE_INFO_FIRSTBYTE above). The function \fBpcre_exec()\fP is called to match a subject string against a compiled pattern, which is passed in the \fIcode\fP argument. If the pattern has been studied, the result of the study should be passed in the -\fIextra\fP argument. +\fIextra\fP argument. This function is the main matching facility of the +library, and it operates in a Perl-like manner. For specialist use there is +also an alternative matching function, which is described +.\" HTML <a href="#dfamatch"> +.\" </a> +below +.\" +in the section about the \fBpcre_dfa_exec()\fP function. .P In most applications, the pattern will have been compiled (and optionally studied) in the same process that calls \fBpcre_exec()\fP. However, it is @@ -796,7 +955,7 @@ Here is an example of a simple call to \fBpcre_exec()\fP: 0, /* start at offset 0 in the subject */ 0, /* default options */ ovector, /* vector of integers for substring information */ - 30); /* number of elements in the vector (NOT size in bytes) */ + 30); /* number of elements (NOT size in bytes) */ . .\" HTML <a name="extradata"></a> .SS "Extra data for \fBpcre_exec()\fR" @@ -1041,6 +1200,7 @@ subpatterns there are in a compiled pattern. The smallest size for \fIovector\fP that will allow for \fIn\fP captured substrings, in addition to the offsets of the substring matched by the whole pattern, is (\fIn\fP+1)*3. . +.\" HTML <a name="errorlist"></a> .SS "Return values from \fBpcre_exec()\fP" .rs .sp @@ -1112,7 +1272,7 @@ A string that contains an invalid UTF-8 byte sequence was passed as a subject. The UTF-8 byte sequence that was passed as a subject was valid, but the value of \fIstartoffset\fP did not point to the beginning of a UTF-8 character. .sp - PCRE_ERROR_PARTIAL (-12) + PCRE_ERROR_PARTIAL (-12) .sp The subject string did not match, but it did match partially. See the .\" HREF @@ -1120,7 +1280,7 @@ The subject string did not match, but it did match partially. See the .\" documentation for details of partial matching. .sp - PCRE_ERROR_BAD_PARTIAL (-13) + PCRE_ERROR_BADPARTIAL (-13) .sp The PCRE_PARTIAL option was used with a compiled pattern containing items that are not supported for partial matching. See the @@ -1129,12 +1289,12 @@ are not supported for partial matching. See the .\" documentation for details of partial matching. .sp - PCRE_ERROR_INTERNAL (-14) + PCRE_ERROR_INTERNAL (-14) .sp An unexpected internal error has occurred. This error could be caused by a bug in PCRE or by overwriting of the compiled pattern. .sp - PCRE_ERROR_BADCOUNT (-15) + PCRE_ERROR_BADCOUNT (-15) .sp This error is given if the value of the \fIovecsize\fP argument is negative. . @@ -1281,8 +1441,196 @@ translation table. These functions call \fBpcre_get_stringnumber()\fP, and if it succeeds, they then call \fIpcre_copy_substring()\fP or \fIpcre_get_substring()\fP, as appropriate. +. +. +.SH "FINDING ALL POSSIBLE MATCHES" +.rs +.sp +The traditional matching function uses a similar algorithm to Perl, which stops +when it finds the first match, starting at a given point in the subject. If you +want to find all possible matches, or the longest possible match, consider +using the alternative matching function (see below) instead. If you cannot use +the alternative function, but still need to find all possible matches, you +can kludge it up by making use of the callout facility, which is described in +the +.\" HREF +\fBpcrecallout\fP +.\" +documentation. +.P +What you have to do is to insert a callout right at the end of the pattern. +When your callout function is called, extract and save the current matched +substring. Then return 1, which forces \fBpcre_exec()\fP to backtrack and try +other alternatives. Ultimately, when it runs out of matches, \fBpcre_exec()\fP +will yield PCRE_ERROR_NOMATCH. +. +. +.\" HTML <a name="dfamatch"></a> +.SH "MATCHING A PATTERN: THE ALTERNATIVE FUNCTION" +.rs +.sp +.B int pcre_dfa_exec(const pcre *\fIcode\fP, "const pcre_extra *\fIextra\fP," +.ti +5n +.B "const char *\fIsubject\fP," int \fIlength\fP, int \fIstartoffset\fP, +.ti +5n +.B int \fIoptions\fP, int *\fIovector\fP, int \fIovecsize\fP, +.ti +5n +.B int *\fIworkspace\fP, int \fIwscount\fP); +.P +The function \fBpcre_dfa_exec()\fP is called to match a subject string against +a compiled pattern, using a "DFA" matching algorithm. This has different +characteristics to the normal algorithm, and is not compatible with Perl. Some +of the features of PCRE patterns are not supported. Nevertheless, there are +times when this kind of matching can be useful. For a discussion of the two +matching algorithms, see the +.\" HREF +\fBpcrematching\fP +.\" +documentation. +.P +The arguments for the \fBpcre_dfa_exec()\fP function are the same as for +\fBpcre_exec()\fP, plus two extras. The \fIovector\fP argument is used in a +different way, and this is described below. The other common arguments are used +in the same way as for \fBpcre_exec()\fP, so their description is not repeated +here. +.P +The two additional arguments provide workspace for the function. The workspace +vector should contain at least 20 elements. It is used for keeping track of +multiple paths through the pattern tree. More workspace will be needed for +patterns and subjects where there are a lot of possible matches. +.P +Here is an example of a simple call to \fBpcre_exec()\fP: +.sp + int rc; + int ovector[10]; + int wspace[20]; + rc = pcre_exec( + re, /* result of pcre_compile() */ + NULL, /* we didn't study the pattern */ + "some string", /* the subject string */ + 11, /* the length of the subject string */ + 0, /* start at offset 0 in the subject */ + 0, /* default options */ + ovector, /* vector of integers for substring information */ + 10, /* number of elements (NOT size in bytes) */ + wspace, /* working space vector */ + 20); /* number of elements (NOT size in bytes) */ +. +.SS "Option bits for \fBpcre_dfa_exec()\fP" +.rs +.sp +The unused bits of the \fIoptions\fP argument for \fBpcre_dfa_exec()\fP must be +zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NOTBOL, +PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL, +PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last three of these are +the same as for \fBpcre_exec()\fP, so their description is not repeated here. +.sp + PCRE_PARTIAL +.sp +This has the same general effect as it does for \fBpcre_exec()\fP, but the +details are slightly different. When PCRE_PARTIAL is set for +\fBpcre_dfa_exec()\fP, the return code PCRE_ERROR_NOMATCH is converted into +PCRE_ERROR_PARTIAL if the end of the subject is reached, there have been no +complete matches, but there is still at least one matching possibility. The +portion of the string that provided the partial match is set as the first +matching string. +.sp + PCRE_DFA_SHORTEST +.sp +Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to stop as +soon as it has found one match. Because of the way the DFA algorithm works, +this is necessarily the shortest possible match at the first possible matching +point in the subject string. +.sp + PCRE_DFA_RESTART +.sp +When \fBpcre_dfa_exec()\fP is called with the PCRE_PARTIAL option, and returns +a partial match, it is possible to call it again, with additional subject +characters, and have it continue with the same match. The PCRE_DFA_RESTART +option requests this action; when it is set, the \fIworkspace\fP and +\fIwscount\fP options must reference the same vector as before because data +about the match so far is left in them after a partial match. There is more +discussion of this facility in the +.\" HREF +\fBpcrepartial\fP +.\" +documentation. +. +.SS "Successful returns from \fBpcre_dfa_exec()\fP" +.rs +.sp +When \fBpcre_dfa_exec()\fP succeeds, it may have matched more than one +substring in the subject. Note, however, that all the matches from one run of +the function start at the same point in the subject. The shorter matches are +all initial substrings of the longer matches. For example, if the pattern +.sp + <.*> +.sp +is matched against the string +.sp + This is <something> <something else> <something further> no more +.sp +the three matched strings are +.sp + <something> + <something> <something else> + <something> <something else> <something further> +.sp +On success, the yield of the function is a number greater than zero, which is +the number of matched substrings. The substrings themselves are returned in +\fIovector\fP. Each string uses two elements; the first is the offset to the +start, and the second is the offset to the end. All the strings have the same +start offset. (Space could have been saved by giving this only once, but it was +decided to retain some compatibility with the way \fBpcre_exec()\fP returns +data, even though the meaning of the strings is different.) +.P +The strings are returned in reverse order of length; that is, the longest +matching string is given first. If there were too many matches to fit into +\fIovector\fP, the yield of the function is zero, and the vector is filled with +the longest matches. +. +.SS "Error returns from \fBpcre_dfa_exec()\fP" +.rs +.sp +The \fBpcre_dfa_exec()\fP function returns a negative number when it fails. +Many of the errors are the same as for \fBpcre_exec()\fP, and these are +described +.\" HTML <a href="#errorlist"> +.\" </a> +above. +.\" +There are in addition the following errors that are specific to +\fBpcre_dfa_exec()\fP: +.sp + PCRE_ERROR_DFA_UITEM (-16) +.sp +This return is given if \fBpcre_dfa_exec()\fP encounters an item in the pattern +that it does not support, for instance, the use of \eC or a back reference. +.sp + PCRE_ERROR_DFA_UCOND (-17) +.sp +This return is given if \fBpcre_dfa_exec()\fP encounters a condition item in a +pattern that uses a back reference for the condition. This is not supported. +.sp + PCRE_ERROR_DFA_UMLIMIT (-18) +.sp +This return is given if \fBpcre_dfa_exec()\fP is called with an \fIextra\fP +block that contains a setting of the \fImatch_limit\fP field. This is not +supported (it is meaningless). +.sp + PCRE_ERROR_DFA_WSSIZE (-19) +.sp +This return is given if \fBpcre_dfa_exec()\fP runs out of space in the +\fIworkspace\fP vector. +.sp + PCRE_ERROR_DFA_RECURSE (-20) +.sp +When a recursive subpattern is processed, the matching function calls itself +recursively, using private vectors for \fIovector\fP and \fIworkspace\fP. This +error is given if the output vector is not large enough. This should be +extremely rare, as a vector of size 1000 is used. .P .in 0 -Last updated: 09 September 2004 +Last updated: 16 May 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcrebuild.3 b/doc/pcrebuild.3 index 8ac5882..62c4ea2 100644 --- a/doc/pcrebuild.3 +++ b/doc/pcrebuild.3 @@ -100,10 +100,11 @@ to the \fBconfigure\fP command. .rs .sp Internally, PCRE has a function called \fBmatch()\fP, which it calls repeatedly -(possibly recursively) when matching a pattern. By controlling the maximum -number of times this function may be called during a single matching operation, -a limit can be placed on the resources used by a single call to -\fBpcre_exec()\fP. The limit can be changed at run time, as described in the +(possibly recursively) when matching a pattern with the \fBpcre_exec()\fP +function. By controlling the maximum number of times this function may be +called during a single matching operation, a limit can be placed on the +resources used by a single call to \fBpcre_exec()\fP. The limit can be changed +at run time, as described in the .\" HREF \fBpcreapi\fP .\" @@ -112,7 +113,8 @@ setting such as .sp --with-match-limit=500000 .sp -to the \fBconfigure\fP command. +to the \fBconfigure\fP command. This setting has no effect on the +\fBpcre_dfa_exec()\fP matching function. . .SH "HANDLING VERY LARGE PATTERNS" .rs @@ -138,13 +140,14 @@ of the compiled pattern, and this changes with the link size. .SH "AVOIDING EXCESSIVE STACK USAGE" .rs .sp -PCRE implements backtracking while matching by making recursive calls to an -internal function called \fBmatch()\fP. In environments where the size of the -stack is limited, this can severely limit PCRE's operation. (The Unix -environment does not usually suffer from this problem.) An alternative approach -that uses memory from the heap to remember data, instead of using recursive -function calls, has been implemented to work round this problem. If you want to -build a version of PCRE that works this way, add +When matching with the \fBpcre_exec()\fP function, PCRE implements backtracking +by making recursive calls to an internal function called \fBmatch()\fP. In +environments where the size of the stack is limited, this can severely limit +PCRE's operation. (The Unix environment does not usually suffer from this +problem.) An alternative approach that uses memory from the heap to remember +data, instead of using recursive function calls, has been implemented to work +round this problem. If you want to build a version of PCRE that works this way, +add .sp --disable-stack-for-recursion .sp @@ -155,7 +158,8 @@ predictable: the block sizes requested are always the same, and the blocks are always freed in reverse order. A calling program might be able to implement optimized functions that perform better than the standard \fBmalloc()\fP and \fBfree()\fP functions. PCRE runs noticeably more slowly when built in this -way. +way. This option affects only the \fBpcre_exec()\fP function; it is not +relevant for the the \fBpcre_dfa_exec()\fP function. . .SH "USING EBCDIC CODE" .rs @@ -169,6 +173,6 @@ compiled to run in an EBCDIC environment by adding to the \fBconfigure\fP command. .P .in 0 -Last updated: 09 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcrecallout.3 b/doc/pcrecallout.3 index 5fd8ff8..6155d43 100644 --- a/doc/pcrecallout.3 +++ b/doc/pcrecallout.3 @@ -58,9 +58,10 @@ no match, the callout is obeyed. .rs .sp During matching, when PCRE reaches a callout point, the external function -defined by \fIpcre_callout\fP is called (if it is set). The only argument is a -pointer to a \fBpcre_callout\fP block. This structure contains the following -fields: +defined by \fIpcre_callout\fP is called (if it is set). This applies to both +the \fBpcre_exec()\fP and the \fBpcre_dfa_exec()\fP matching functions. The +only argument to the callout function is a pointer to a \fBpcre_callout\fP +block. This structure contains the following fields: .sp int \fIversion\fP; int \fIcallout_number\fP; @@ -85,9 +86,11 @@ into the pattern (that is, the number after ?C for manual callouts, and 255 for automatically generated callouts). .P The \fIoffset_vector\fP field is a pointer to the vector of offsets that was -passed by the caller to \fBpcre_exec()\fP. The contents can be inspected in -order to extract substrings that have been matched so far, in the same way as -for extracting substrings after a match has completed. +passed by the caller to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP. When +\fBpcre_exec()\fP is used, the contents can be inspected in order to extract +substrings that have been matched so far, in the same way as for extracting +substrings after a match has completed. For \fBpcre_dfa_exec()\fP this field is +not useful. .P The \fIsubject\fP and \fIsubject_length\fP fields contain copies of the values that were passed to \fBpcre_exec()\fP. @@ -100,19 +103,22 @@ different starting points in the subject. The \fIcurrent_position\fP field contains the offset within the subject of the current match pointer. .P -The \fIcapture_top\fP field contains one more than the number of the highest -numbered captured substring so far. If no substrings have been captured, -the value of \fIcapture_top\fP is one. +When the \fBpcre_exec()\fP function is used, the \fIcapture_top\fP field +contains one more than the number of the highest numbered captured substring so +far. If no substrings have been captured, the value of \fIcapture_top\fP is +one. This is always the case when \fBpcre_dfa_exec()\fP is used, because it +does not support captured substrings. .P The \fIcapture_last\fP field contains the number of the most recently captured -substring. If no substrings have been captured, its value is -1. +substring. If no substrings have been captured, its value is -1. This is always +the case when \fBpcre_dfa_exec()\fP is used. .P The \fIcallout_data\fP field contains a value that is passed to -\fBpcre_exec()\fP by the caller specifically so that it can be passed back in -callouts. It is passed in the \fIpcre_callout\fP field of the \fBpcre_extra\fP -data structure. If no such data was passed, the value of \fIcallout_data\fP in -a \fBpcre_callout\fP block is NULL. There is a description of the -\fBpcre_extra\fP structure in the +\fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP specifically so that it can be +passed back in callouts. It is passed in the \fIpcre_callout\fP field of the +\fBpcre_extra\fP data structure. If no such data was passed, the value of +\fIcallout_data\fP in a \fBpcre_callout\fP block is NULL. There is a +description of the \fBpcre_extra\fP structure in the .\" HREF \fBpcreapi\fP .\" @@ -139,10 +145,10 @@ same callout number. However, they are set for all callouts. .sp The external callout function returns an integer to PCRE. If the value is zero, matching proceeds as normal. If the value is greater than zero, matching fails -at the current point, but backtracking to test other matching possibilities -goes ahead, just as if a lookahead assertion had failed. If the value is less -than zero, the match is abandoned, and \fBpcre_exec()\fP returns the negative -value. +at the current point, but the testing of other matching possibilities goes +ahead, just as if a lookahead assertion had failed. If the value is less than +zero, the match is abandoned, and \fBpcre_exec()\fP (or \fBpcre_dfa_exec()\fP) +returns the negative value. .P Negative values should normally be chosen from the set of PCRE_ERROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a standard "no match" failure. @@ -150,6 +156,6 @@ The error number PCRE_ERROR_CALLOUT is reserved for use by callout functions; it will never be used by PCRE itself. .P .in 0 -Last updated: 09 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcrecompat.3 b/doc/pcrecompat.3 index 6a853e0..05ed3cb 100644 --- a/doc/pcrecompat.3 +++ b/doc/pcrecompat.3 @@ -114,8 +114,11 @@ package. .sp (m) Patterns compiled by PCRE can be saved and re-used at a later time, even on different hosts that have the other endianness. +.sp +(n) The alternative matching function (\fBpcre_dfa_exec()\fP) matches in a +different way and is not Perl-compatible. .P .in 0 -Last updated: 09 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcrecpp.3 b/doc/pcrecpp.3 new file mode 100644 index 0000000..01eb028 --- /dev/null +++ b/doc/pcrecpp.3 @@ -0,0 +1,220 @@ +.TH PCRE 3 +.SH NAME +PCRE - Perl-compatible regular expressions. +.SH "SYNOPSIS OF C++ WRAPPER" +.rs +.sp +.B #include <pcrecpp.h> +.PP +.SM +.br +.SH DESCRIPTION +.rs +.sp +The C++ wrapper for PCRE was provided by Google Inc. This brief man page was +constructed from the notes in the \fIpcrecpp.h\fP file, which should be +consulted for further details. +. +. +.SH "MATCHING INTERFACE" +.rs +.sp +The "FullMatch" operation checks that supplied text matches a supplied pattern +exactly. If pointer arguments are supplied, it copies matched sub-strings that +match sub-patterns into them. +.sp + Example: successful match + pcrecpp::RE re("h.*o"); + re.FullMatch("hello"); +.sp + Example: unsuccessful match (requires full match): + pcrecpp::RE re("e"); + !re.FullMatch("hello"); +.sp + Example: creating a temporary RE object: + pcrecpp::RE("h.*o").FullMatch("hello"); +.sp +You can pass in a "const char*" or a "string" for "text". The examples below +tend to use a const char*. You can, as in the different examples above, store +the RE object explicitly in a variable or use a temporary RE object. The +examples below use one mode or the other arbitrarily. Either could correctly be +used for any of these examples. +.P +You must supply extra pointer arguments to extract matched subpieces. +.sp + Example: extracts "ruby" into "s" and 1234 into "i" + int i; + string s; + pcrecpp::RE re("(\e\ew+):(\e\ed+)"); + re.FullMatch("ruby:1234", &s, &i); +.sp + Example: does not try to extract any extra sub-patterns + re.FullMatch("ruby:1234", &s); +.sp + Example: does not try to extract into NULL + re.FullMatch("ruby:1234", NULL, &i); +.sp + Example: integer overflow causes failure + !re.FullMatch("ruby:1234567891234", NULL, &i); +.sp + Example: fails because there aren't enough sub-patterns: + !pcrecpp::RE("\e\ew+:\e\ed+").FullMatch("ruby:1234", &s); +.sp + Example: fails because string cannot be stored in integer + !pcrecpp::RE("(.*)").FullMatch("ruby", &i); +.sp +The provided pointer arguments can be pointers to any scalar numeric +type, or one of: +.sp + string (matched piece is copied to string) + StringPiece (StringPiece is mutated to point to matched piece) + T (where "bool T::ParseFrom(const char*, int)" exists) + NULL (the corresponding matched sub-pattern is not copied) +.sp +The function returns true iff all of the following conditions are satisfied: +.sp + a. "text" matches "pattern" exactly; +.sp + b. The number of matched sub-patterns is >= number of supplied + pointers; +.sp + c. The "i"th argument has a suitable type for holding the + string captured as the "i"th sub-pattern. If you pass in + NULL for the "i"th argument, or pass fewer arguments than + number of sub-patterns, "i"th captured sub-pattern is + ignored. +.sp +The matching interface supports at most 16 arguments per call. +If you need more, consider using the more general interface +\fBpcrecpp::RE::DoMatch\fP. See \fBpcrecpp.h\fP for the signature for +\fBDoMatch\fP. +. +.SH "PARTIAL MATCHES" +.rs +.sp +You can use the "PartialMatch" operation when you want the pattern +to match any substring of the text. +.sp + Example: simple search for a string: + pcrecpp::RE("ell").PartialMatch("hello"); +.sp + Example: find first number in a string: + int number; + pcrecpp::RE re("(\e\ed+)"); + re.PartialMatch("x*100 + 20", &number); + assert(number == 100); +. +. +.SH "UTF-8 AND THE MATCHING INTERFACE" +.rs +.sp +By default, pattern and text are plain text, one byte per character. The UTF8 +flag, passed to the constructor, causes both pattern and string to be treated +as UTF-8 text, still a byte stream but potentially multiple bytes per +character. In practice, the text is likelier to be UTF-8 than the pattern, but +the match returned may depend on the UTF8 flag, so always use it when matching +UTF8 text. For example, "." will match one byte normally but with UTF8 set may +match up to three bytes of a multi-byte character. +.sp + Example: + pcrecpp::RE_Options options; + options.set_utf8(); + pcrecpp::RE re(utf8_pattern, options); + re.FullMatch(utf8_string); +.sp + Example: using the convenience function UTF8(): + pcrecpp::RE re(utf8_pattern, pcrecpp::UTF8()); + re.FullMatch(utf8_string); +.sp +NOTE: The UTF8 flag is ignored if pcre was not configured with the + --enable-utf8 flag. +. +. +.SH "SCANNING TEXT INCREMENTALLY" +.rs +.sp +The "Consume" operation may be useful if you want to repeatedly +match regular expressions at the front of a string and skip over +them as they match. This requires use of the "StringPiece" type, +which represents a sub-range of a real string. Like RE, StringPiece +is defined in the pcrecpp namespace. +.sp + Example: read lines of the form "var = value" from a string. + string contents = ...; // Fill string somehow + pcrecpp::StringPiece input(contents); // Wrap in a StringPiece + + string var; + int value; + pcrecpp::RE re("(\e\ew+) = (\e\ed+)\en"); + while (re.Consume(&input, &var, &value)) { + ...; + } +.sp +Each successful call to "Consume" will set "var/value", and also +advance "input" so it points past the matched text. +.P +The "FindAndConsume" operation is similar to "Consume" but does not +anchor your match at the beginning of the string. For example, you +could extract all words from a string by repeatedly calling +.sp + pcrecpp::RE("(\e\ew+)").FindAndConsume(&input, &word) +. +. +.SH "PARSING HEX/OCTAL/C-RADIX NUMBERS" +.rs +.sp +By default, if you pass a pointer to a numeric value, the +corresponding text is interpreted as a base-10 number. You can +instead wrap the pointer with a call to one of the operators Hex(), +Octal(), or CRadix() to interpret the text in another base. The +CRadix operator interprets C-style "0" (base-8) and "0x" (base-16) +prefixes, but defaults to base-10. +.sp + Example: + int a, b, c, d; + pcrecpp::RE re("(.*) (.*) (.*) (.*)"); + re.FullMatch("100 40 0100 0x40", + pcrecpp::Octal(&a), pcrecpp::Hex(&b), + pcrecpp::CRadix(&c), pcrecpp::CRadix(&d)); +.sp +will leave 64 in a, b, c, and d. +. +. +.SH "REPLACING PARTS OF STRINGS" +.rs +.sp +You can replace the first match of "pattern" in "str" with "rewrite". +Within "rewrite", backslash-escaped digits (\e1 to \e9) can be +used to insert text matching corresponding parenthesized group +from the pattern. \e0 in "rewrite" refers to the entire matching +text. For example: +.sp + string s = "yabba dabba doo"; + pcrecpp::RE("b+").Replace("d", &s); +.sp +will leave "s" containing "yada dabba doo". The result is true if the pattern +matches and a replacement occurs, false otherwise. +.P +\fBGlobalReplace\fP is like \fBReplace\fP except that it replaces all +occurrences of the pattern in the string with the rewrite. Replacements are +not subject to re-matching. For example: +.sp + string s = "yabba dabba doo"; + pcrecpp::RE("b+").GlobalReplace("d", &s); +.sp +will leave "s" containing "yada dada doo". It returns the number of +replacements made. +.P +\fBExtract\fP is like \fBReplace\fP, except that if the pattern matches, +"rewrite" is copied into "out" (an additional argument) with substitutions. +The non-matching portions of "text" are ignored. Returns true iff a match +occurred and the extraction happened successfully; if no match occurs, the +string is left unaffected. +. +. +.SH AUTHOR +.rs +.sp +The C++ wrapper was contributed by Google Inc. +.br +Copyright (c) 2005 Google Inc. diff --git a/doc/pcregrep.1 b/doc/pcregrep.1 index 56c37d8..f1244e4 100644 --- a/doc/pcregrep.1 +++ b/doc/pcregrep.1 @@ -2,7 +2,7 @@ .SH NAME pcregrep - a grep with Perl-compatible regular expressions. .SH SYNOPSIS -.B pcregrep [-Vcfhilnrsuvx] [long options] [pattern] [file1 file2 ...] +.B pcregrep [options] [long options] [pattern] [file1 file2 ...] . .SH DESCRIPTION .rs @@ -19,28 +19,60 @@ PCRE supports. A pattern must be specified on the command line unless the \fB-f\fP option is used (see below). .P -If no files are specified, \fBpcregrep\fP reads the standard input. By default, -each line that matches the pattern is copied to the standard output, and if -there is more than one file, the file name is printed before each line of -output. However, there are options that can change how \fBpcregrep\fP behaves. +If no files are specified, \fBpcregrep\fP reads the standard input. The +standard input can also be referenced by a name consisting of a single hyphen. +For example: +.sp + pcregrep some-pattern /file1 - /file3 +.sp +By default, each line that matches the pattern is copied to the standard +output, and if there is more than one file, the file name is printed before +each line of output. However, there are options that can change how +\fBpcregrep\fP behaves. In particular, the \fB-M\fP option makes it possible to +search for patterns that span line boundaries. .P -Lines are limited to BUFSIZ characters. BUFSIZ is defined in \fB<stdio.h>\fP. -The newline character is removed from the end of each line before it is matched -against the pattern. +Patterns are limited to 8K or BUFSIZ characters, whichever is the greater. +BUFSIZ is defined in \fB<stdio.h>\fP. . .SH OPTIONS .rs -.sp .TP 10 -\fB-V\fP -Write the version number of the PCRE library being used to the standard error -stream. +\fB--\fP +This terminate the list of options. It is useful if the next item on the +command line starts with a hyphen, but is not an option. +.TP +\fB-A\fP \fInumber\fP +Print \fInumber\fP lines of context after each matching line. If file names +and/or line numbers are being printed, a hyphen separator is used instead of a +colon for the context lines. A line containing "--" is printed between each +group of lines, unless they are in fact contiguous in the input file. The value +of \fInumber\fP is expected to be relatively small. However, \fBpcregrep\fP +guarantees to have up to 8K of following text available for context printing. +.TP +\fB-B\fP \fInumber\fP +Print \fInumber\fP lines of context before each matching line. If file names +and/or line numbers are being printed, a hyphen separator is used instead of a +colon for the context lines. A line containing "--" is printed between each +group of lines, unless they are in fact contiguous in the input file. The value +of \fInumber\fP is expected to be relatively small. However, \fBpcregrep\fP +guarantees to have up to 8K of preceding text available for context printing. +.TP +\fB-C\fP \fInumber\fP +Print \fInumber\fP lines of context both before and after each matching line. +This is equivalent to setting both \fB-A\fP and \fB-B\fP to the same value. .TP \fB-c\fP Do not print individual lines; instead just print a count of the number of lines that would otherwise have been printed. If several files are given, a count is printed for each of them. .TP +\fB--exclude\fP=\fIpattern\fP +When \fBpcregrep\fP is searching the files in a directory as a consequence of +the \fB-r\fP (recursive search) option, any files whose names match the pattern +are excluded. The pattern is a PCRE regular expression. If a file name matches +both \fB--include\fP and \fB--exclude\fP, it is excluded. There is no short +form for this option. +.TP \fB-f\fP\fIfilename\fP Read a number of patterns from the file, one per line, and match all of them against each line of input. A line is output if any of the patterns match it. @@ -55,30 +87,73 @@ Suppress printing of filenames when searching multiple files. \fB-i\fP Ignore upper/lower case distinctions during comparisons. .TP +\fB--include\fP=\fIpattern\fP +When \fBpcregrep\fP is searching the files in a directory as a consequence of +the \fB-r\fP (recursive search) option, only files whose names match the +pattern are included. The pattern is a PCRE regular expression. If a file name +matches both \fB--include\fP and \fB--exclude\fP, it is excluded. There is no +short form for this option. +.TP +\fB-L\fP +Instead of printing lines from the files, just print the names of the files +that do not contain any lines that would have been printed. Each file name is +printed once, on a separate line. +.TP \fB-l\fP Instead of printing lines from the files, just print the names of the files containing lines that would have been printed. Each file name is printed once, on a separate line. .TP +\fB--label\fP=\fIname\fP +This option supplies a name to be used for the standard input when file names +are being printed. If not supplied, "(standard input)" is used. There is no +short form for this option. +.TP +\fB-M\fP +Allow patterns to match more than one line. When this option is given, patterns +may usefully contain literal newline characters and internal occurrences of ^ +and $ characters. The output for any one match may consist of more than one +line. When this option is set, the PCRE library is called in "multiline" mode. +There is a limit to the number of lines that can be matched, imposed by the way +that \fBpcregrep\fP buffers the input file as it scans it. However, +\fBpcregrep\fP ensures that at least 8K characters or the rest of the document +(whichever is the shorter) are available for forward matching, and similarly +the previous 8K characters (or all the previous characters, if fewer than 8K) +are guaranteed to be available for lookbehind assertions. +.TP \fB-n\fP Precede each line by its line number in the file. .TP +\fB-q\fP +Work quietly, that is, display nothing except error messages. +The exit status indicates whether or not any matches were found. +.TP \fB-r\fP -If any file is a directory, recursively scan the files it contains. Without +If any given path is a directory, recursively scan the files it contains, +taking note of any \fB--include\fP and \fB--exclude\fP settings. Without \fB-r\fP a directory is scanned as a normal file. .TP \fB-s\fP -Work silently, that is, display nothing except error messages. -The exit status indicates whether any matches were found. +Suppress error messages about non-existent or unreadable files. Such files are +quietly skipped. However, the return code is still 2, even if matches were +found in other files. .TP \fB-u\fP Operate in UTF-8 mode. This option is available only if PCRE has been compiled with UTF-8 support. Both the pattern and each subject line must be valid strings of UTF-8 characters. .TP +\fB-V\fP +Write the version numbers of \fBpcregrep\fP and the PCRE library that is being +used to the standard error stream. +.TP \fB-v\fP Invert the sense of the match, so that lines which do \fInot\fP match the -pattern are now the ones that are found. +pattern are the ones that are found. +.TP +\fB-w\fP +Force the pattern to match only whole words. This is equivalent to having \eb +at the start and end of the pattern. .TP \fB-x\fP Force the pattern to be anchored (it must start matching at the beginning of @@ -92,39 +167,66 @@ alternative branch in the regular expression. Long forms of all the options are available, as in GNU grep. They are shown in the following table: .sp + -A --after-context + -B --before-context + -C --context -c --count + --exclude (no short form) + -f --file -h --no-filename + --help (no short form) -i --ignore-case + --include (no short form) + -L --files-without-match -l --files-with-matches + --label (no short form) -n --line-number -r --recursive + -q --quiet -s --no-messages -u --utf-8 -V --version -v --invert-match -x --line-regex -x --line-regexp +. +.SH "OPTIONS WITH DATA" +.rs +.sp +There are four different ways in which an option with data can be specified. +If a short form option is used, the data may follow immediately, or in the next +command line item. For example: +.sp + -f/some/file + -f /some/file +.sp +If a long form option is used, the data may appear in the same command line +item, separated by an = character, or it may appear in the next command line +item. For example: +.sp + --file=/some/file + --file /some/file .sp -In addition, --file=\fIfilename\fP is equivalent to -f\fIfilename\fP, and ---help shows the list of options and then exits. . .SH DIAGNOSTICS .rs .sp Exit status is 0 if any matches were found, 1 if no matches were found, and 2 -for syntax errors or inacessible files (even if matches were found). +for syntax errors and non-existent or inacessible files (even if matches were +found in other files). Using the \fB-s\fP option to suppress error messages +about inaccessble files does not affect the return code. . . .SH AUTHOR .rs .sp -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel .br University Computing Service .br Cambridge CB2 3QG, England. .P .in 0 -Last updated: 09 September 2004 +Last updated: 16 May 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcregrep.txt b/doc/pcregrep.txt index 1dca003..c2374e1 100644 --- a/doc/pcregrep.txt +++ b/doc/pcregrep.txt @@ -6,7 +6,7 @@ NAME pcregrep - a grep with Perl-compatible regular expressions. SYNOPSIS - pcregrep [-Vcfhilnrsuvx] [long options] [pattern] [file1 file2 ...] + pcregrep [options] [long options] [pattern] [file1 file2 ...] DESCRIPTION @@ -20,61 +20,139 @@ DESCRIPTION A pattern must be specified on the command line unless the -f option is used (see below). - If no files are specified, pcregrep reads the standard input. By - default, each line that matches the pattern is copied to the standard - output, and if there is more than one file, the file name is printed - before each line of output. However, there are options that can change - how pcregrep behaves. + If no files are specified, pcregrep reads the standard input. The stan- + dard input can also be referenced by a name consisting of a single + hyphen. For example: - Lines are limited to BUFSIZ characters. BUFSIZ is defined in <stdio.h>. - The newline character is removed from the end of each line before it is - matched against the pattern. + pcregrep some-pattern /file1 - /file3 + By default, each line that matches the pattern is copied to the stan- + dard output, and if there is more than one file, the file name is + printed before each line of output. However, there are options that can + change how pcregrep behaves. In particular, the -M option makes it pos- + sible to search for patterns that span line boundaries. + + Patterns are limited to 8K or BUFSIZ characters, whichever is the + greater. BUFSIZ is defined in <stdio.h>. -OPTIONS +OPTIONS - -V Write the version number of the PCRE library being used to - the standard error stream. + -- This terminate the list of options. It is useful if the next + item on the command line starts with a hyphen, but is not an + option. + + -A number Print number lines of context after each matching line. If + file names and/or line numbers are being printed, a hyphen + separator is used instead of a colon for the context lines. A + line containing "--" is printed between each group of lines, + unless they are in fact contiguous in the input file. The + value of number is expected to be relatively small. However, + pcregrep guarantees to have up to 8K of following text avail- + able for context printing. + + -B number Print number lines of context before each matching line. If + file names and/or line numbers are being printed, a hyphen + separator is used instead of a colon for the context lines. A + line containing "--" is printed between each group of lines, + unless they are in fact contiguous in the input file. The + value of number is expected to be relatively small. However, + pcregrep guarantees to have up to 8K of preceding text avail- + able for context printing. + + -C number Print number lines of context both before and after each + matching line. This is equivalent to setting both -A and -B + to the same value. -c Do not print individual lines; instead just print a count of the number of lines that would otherwise have been printed. If several files are given, a count is printed for each of them. + --exclude=pattern + When pcregrep is searching the files in a directory as a con- + sequence of the -r (recursive search) option, any files whose + names match the pattern are excluded. The pattern is a PCRE + regular expression. If a file name matches both --include and + --exclude, it is excluded. There is no short form for this + option. + -ffilename - Read a number of patterns from the file, one per line, and - match all of them against each line of input. A line is out- - put if any of the patterns match it. When -f is used, no - pattern is taken from the command line; all arguments are - treated as file names. There is a maximum of 100 patterns. + Read a number of patterns from the file, one per line, and + match all of them against each line of input. A line is out- + put if any of the patterns match it. When -f is used, no + pattern is taken from the command line; all arguments are + treated as file names. There is a maximum of 100 patterns. Trailing white space is removed, and blank lines are ignored. - An empty file contains no patterns and therefore matches + An empty file contains no patterns and therefore matches nothing. -h Suppress printing of filenames when searching multiple files. -i Ignore upper/lower case distinctions during comparisons. - -l Instead of printing lines from the files, just print the - names of the files containing lines that would have been - printed. Each file name is printed once, on a separate line. + --include=pattern + When pcregrep is searching the files in a directory as a con- + sequence of the -r (recursive search) option, only files + whose names match the pattern are included. The pattern is a + PCRE regular expression. If a file name matches both + --include and --exclude, it is excluded. There is no short + form for this option. + + -L Instead of printing lines from the files, just print the + names of the files that do not contain any lines that would + have been printed. Each file name is printed once, on a sepa- + rate line. + + -l Instead of printing lines from the files, just print the + names of the files containing lines that would have been + printed. Each file name is printed once, on a separate line. + + --label=name + This option supplies a name to be used for the standard input + when file names are being printed. If not supplied, "(stan- + dard input)" is used. There is no short form for this option. + + -M Allow patterns to match more than one line. When this option + is given, patterns may usefully contain literal newline char- + acters and internal occurrences of ^ and $ characters. The + output for any one match may consist of more than one line. + When this option is set, the PCRE library is called in "mul- + tiline" mode. There is a limit to the number of lines that + can be matched, imposed by the way that pcregrep buffers the + input file as it scans it. However, pcregrep ensures that at + least 8K characters or the rest of the document (whichever is + the shorter) are available for forward matching, and simi- + larly the previous 8K characters (or all the previous charac- + ters, if fewer than 8K) are guaranteed to be available for + lookbehind assertions. -n Precede each line by its line number in the file. - -r If any file is a directory, recursively scan the files it - contains. Without -r a directory is scanned as a normal file. - - -s Work silently, that is, display nothing except error mes- - sages. The exit status indicates whether any matches were + -q Work quietly, that is, display nothing except error messages. + The exit status indicates whether or not any matches were found. + -r If any given path is a directory, recursively scan the files + it contains, taking note of any --include and --exclude set- + tings. Without -r a directory is scanned as a normal file. + + -s Suppress error messages about non-existent or unreadable + files. Such files are quietly skipped. However, the return + code is still 2, even if matches were found in other files. + -u Operate in UTF-8 mode. This option is available only if PCRE has been compiled with UTF-8 support. Both the pattern and each subject line must be valid strings of UTF-8 characters. - -v Invert the sense of the match, so that lines which do not - match the pattern are now the ones that are found. + -V Write the version numbers of pcregrep and the PCRE library + that is being used to the standard error stream. + + -v Invert the sense of the match, so that lines which do not + match the pattern are the ones that are found. + + -w Force the pattern to match only whole words. This is equiva- + lent to having \b at the start and end of the pattern. -x Force the pattern to be anchored (it must start matching at the beginning of the line) and in addition, require it to @@ -88,12 +166,22 @@ LONG OPTIONS Long forms of all the options are available, as in GNU grep. They are shown in the following table: + -A --after-context + -B --before-context + -C --context -c --count + --exclude (no short form) + -f --file -h --no-filename + --help (no short form) -i --ignore-case + --include (no short form) + -L --files-without-match -l --files-with-matches + --label (no short form) -n --line-number -r --recursive + -q --quiet -s --no-messages -u --utf-8 -V --version @@ -101,22 +189,37 @@ LONG OPTIONS -x --line-regex -x --line-regexp - In addition, --file=filename is equivalent to -ffilename, and --help - shows the list of options and then exits. + +OPTIONS WITH DATA + + There are four different ways in which an option with data can be spec- + ified. If a short form option is used, the data may follow immedi- + ately, or in the next command line item. For example: + + -f/some/file + -f /some/file + + If a long form option is used, the data may appear in the same command + line item, separated by an = character, or it may appear in the next + command line item. For example: + + --file=/some/file + --file /some/file DIAGNOSTICS Exit status is 0 if any matches were found, 1 if no matches were found, - and 2 for syntax errors or inacessible files (even if matches were - found). + and 2 for syntax errors and non-existent or inacessible files (even if + matches were found in other files). Using the -s option to suppress + error messages about inaccessble files does not affect the return code. AUTHOR - Philip Hazel <ph10@cam.ac.uk> + Philip Hazel University Computing Service Cambridge CB2 3QG, England. -Last updated: 09 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 16 May 2005 +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcrematching.3 b/doc/pcrematching.3 new file mode 100644 index 0000000..4524701 --- /dev/null +++ b/doc/pcrematching.3 @@ -0,0 +1,157 @@ +.TH PCRE 3 +.SH NAME +PCRE - Perl-compatible regular expressions +.SH "PCRE MATCHING ALGORITHMS" +.rs +.sp +This document describes the two different algorithms that are available in PCRE +for matching a compiled regular expression against a given subject string. The +"standard" algorithm is the one provided by the \fBpcre_exec()\fP function. +This works in the same was as Perl's matching function, and provides a +Perl-compatible matching operation. +.P +An alternative algorithm is provided by the \fBpcre_dfa_exec()\fP function; +this operates in a different way, and is not Perl-compatible. It has advantages +and disadvantages compared with the standard algorithm, and these are described +below. +.P +When there is only one possible way in which a given subject string can match a +pattern, the two algorithms give the same answer. A difference arises, however, +when there are multiple possibilities. For example, if the pattern +.sp + ^<.*> +.sp +is matched against the string +.sp + <something> <something else> <something further> +.sp +there are three possible answers. The standard algorithm finds only one of +them, whereas the DFA algorithm finds all three. +. +.SH "REGULAR EXPRESSIONS AS TREES" +.rs +.sp +The set of strings that are matched by a regular expression can be represented +as a tree structure. An unlimited repetition in the pattern makes the tree of +infinite size, but it is still a tree. Matching the pattern to a given subject +string (from a given starting point) can be thought of as a search of the tree. +There are two standard ways to search a tree: depth-first and breadth-first, +and these correspond to the two matching algorithms provided by PCRE. +. +.SH "THE STANDARD MATCHING ALGORITHM" +.rs +.sp +In the terminology of Jeffrey Friedl's book \fIMastering Regular +Expressions\fP, the standard algorithm is an "NFA algorithm". It conducts a +depth-first search of the pattern tree. That is, it proceeds along a single +path through the tree, checking that the subject matches what is required. When +there is a mismatch, the algorithm tries any alternatives at the current point, +and if they all fail, it backs up to the previous branch point in the tree, and +tries the next alternative branch at that level. This often involves backing up +(moving to the left) in the subject string as well. The order in which +repetition branches are tried is controlled by the greedy or ungreedy nature of +the quantifier. +.P +If a leaf node is reached, a matching string has been found, and at that point +the algorithm stops. Thus, if there is more than one possible match, this +algorithm returns the first one that it finds. Whether this is the shortest, +the longest, or some intermediate length depends on the way the greedy and +ungreedy repetition quantifiers are specified in the pattern. +.P +Because it ends up with a single path through the tree, it is relatively +straightforward for this algorithm to keep track of the substrings that are +matched by portions of the pattern in parentheses. This provides support for +capturing parentheses and back references. +. +.SH "THE DFA MATCHING ALGORITHM" +.rs +.sp +DFA stands for "deterministic finite automaton", but you do not need to +understand the origins of that name. This algorithm conducts a breadth-first +search of the tree. Starting from the first matching point in the subject, it +scans the subject string from left to right, once, character by character, and +as it does this, it remembers all the paths through the tree that represent +valid matches. +.P +The scan continues until either the end of the subject is reached, or there are +no more unterminated paths. At this point, terminated paths represent the +different matching possibilities (if there are none, the match has failed). +Thus, if there is more than one possible match, this algorithm finds all of +them, and in particular, it finds the longest. In PCRE, there is an option to +stop the algorithm after the first match (which is necessarily the shortest) +has been found. +.P +Note that all the matches that are found start at the same point in the +subject. If the pattern +.sp + cat(er(pillar)?) +.sp +is matched against the string "the caterpillar catchment", the result will be +the three strings "cat", "cater", and "caterpillar" that start at the fourth +character of the subject. The algorithm does not automatically move on to find +matches that start at later positions. +.P +There are a number of features of PCRE regular expressions that are not +supported by the DFA matching algorithm. They are as follows: +.P +1. Because the algorithm finds all possible matches, the greedy or ungreedy +nature of repetition quantifiers is not relevant. Greedy and ungreedy +quantifiers are treated in exactly the same way. +.P +2. When dealing with multiple paths through the tree simultaneously, it is not +straightforward to keep track of captured substrings for the different matching +possibilities, and PCRE's implementation of this algorithm does not attempt to +do this. This means that no captured substrings are available. +.P +3. Because no substrings are captured, back references within the pattern are +not supported, and cause errors if encountered. +.P +4. For the same reason, conditional expressions that use a backreference as the +condition are not supported. +.P +5. Callouts are supported, but the value of the \fIcapture_top\fP field is +always 1, and the value of the \fIcapture_last\fP field is always -1. +.P +6. +The \eC escape sequence, which (in the standard algorithm) matches a single +byte, even in UTF-8 mode, is not supported because the DFA algorithm moves +through the subject string one character at a time, for all active paths +through the tree. +. +.SH "ADVANTAGES OF THE DFA ALGORITHM" +.rs +.sp +Using the DFA matching algorithm provides the following advantages: +.P +1. All possible matches (at a single point in the subject) are automatically +found, and in particular, the longest match is found. To find more than one +match using the standard algorithm, you have to do kludgy things with +callouts. +.P +2. There is much better support for partial matching. The restrictions on the +content of the pattern that apply when using the standard algorithm for partial +matching do not apply to the DFA algorithm. For non-anchored patterns, the +starting position of a partial match is available. +.P +3. Because the DFA algorithm scans the subject string just once, and never +needs to backtrack, it is possible to pass very long subject strings to the +matching function in several pieces, checking for partial matching each time. +. +.SH "DISADVANTAGES OF THE DFA ALGORITHM" +.rs +.sp +The DFA algorithm suffers from a number of disadvantages: +.P +1. It is substantially slower than the standard algorithm. This is partly +because it has to search for all possible matches, but is also because it is +less susceptible to optimization. +.P +2. Capturing parentheses and back references are not supported. +.P +3. The "atomic group" feature of PCRE regular expressions is supported, but +does not provide the advantage that it does for the standard algorithm. +.P +.in 0 +Last updated: 28 February 2005 +.br +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcrepartial.3 b/doc/pcrepartial.3 index 3489c18..ffc0c6e 100644 --- a/doc/pcrepartial.3 +++ b/doc/pcrepartial.3 @@ -5,10 +5,10 @@ PCRE - Perl-compatible regular expressions .rs .sp In normal use of PCRE, if the subject string that is passed to -\fBpcre_exec()\fP matches as far as it goes, but is too short to match the -entire pattern, PCRE_ERROR_NOMATCH is returned. There are circumstances where -it might be helpful to distinguish this case from other cases in which there is -no match. +\fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP matches as far as it goes, but is +too short to match the entire pattern, PCRE_ERROR_NOMATCH is returned. There +are circumstances where it might be helpful to distinguish this case from other +cases in which there is no match. .P Consider, for example, an application where a human is required to type in data for a field with specific formatting requirements. An example might be a date @@ -24,10 +24,19 @@ user interface than a check that is delayed until the entire string has been entered. .P PCRE supports the concept of partial matching by means of the PCRE_PARTIAL -option, which can be set when calling \fBpcre_exec()\fP. When this is done, the -return code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if at any -time during the matching process the entire subject string matched part of the -pattern. No captured data is set when this occurs. +option, which can be set when calling \fBpcre_exec()\fP or +\fBpcre_dfa_exec()\fP. When this flag is set for \fBpcre_exec()\fP, the return +code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if at any time +during the matching process the last part of the subject string matched part of +the pattern. Unfortunately, for non-anchored matching, it is not possible to +obtain the position of the start of the partial match. No captured data is set +when PCRE_ERROR_PARTIAL is returned. +.P +When PCRE_PARTIAL is set for \fBpcre_dfa_exec()\fP, the return code +PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end of the +subject is reached, there have been no complete matches, but there is still at +least one matching possibility. The portion of the string that provided the +partial match is set as the first matching string. .P Using PCRE_PARTIAL disables one of PCRE's optimizations. PCRE remembers the last literal byte in a pattern, and abandons matching immediately if such a @@ -38,9 +47,10 @@ for a subject string that might match only partially. .SH "RESTRICTED PATTERNS FOR PCRE_PARTIAL" .rs .sp -Because of the way certain internal optimizations are implemented in PCRE, the -PCRE_PARTIAL option cannot be used with all patterns. Repeated single -characters such as +Because of the way certain internal optimizations are implemented in the +\fBpcre_exec()\fP function, the PCRE_PARTIAL option cannot be used with all +patterns. These restrictions do not apply when \fBpcre_dfa_exec()\fP is used. +For \fBpcre_exec()\fP, repeated single characters such as .sp a{2,4} .sp @@ -85,11 +95,90 @@ uses the date example quoted above: .sp The first data string is matched completely, so \fBpcretest\fP shows the matched substrings. The remaining four strings do not match the complete -pattern, but the first two are partial matches. +pattern, but the first two are partial matches. The same test, using DFA +matching (by means of the \eD escape sequence), produces the following output: +.sp + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 25jun04\eP\eD + 0: 25jun04 + data> 23dec3\eP\eD + Partial match: 23dec3 + data> 3ju\eP\eD + Partial match: 3ju + data> 3juj\eP\eD + No match + data> j\eP\eD + No match +.sp +Notice that in this case the portion of the string that was matched is made +available. +. +. +.SH "MULTI-SEGMENT MATCHING WITH pcre_dfa_exec()" +.rs +.sp +When a partial match has been found using \fBpcre_dfa_exec()\fP, it is possible +to continue the match by providing additional subject data and calling +\fBpcre_dfa_exec()\fP again with the PCRE_DFA_RESTART option and the same +working space (where details of the previous partial match are stored). Here is +an example using \fBpcretest\fP, where the \eR escape sequence sets the +PCRE_DFA_RESTART option and the \eD escape sequence requests the use of +\fBpcre_dfa_exec()\fP: +.sp + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 23ja\eP\eD + Partial match: 23ja + data> n05\eR\eD + 0: n05 +.sp +The first call has "23ja" as the subject, and requests partial matching; the +second call has "n05" as the subject for the continued (restarted) match. +Notice that when the match is complete, only the last part is shown; PCRE does +not retain the previously partially-matched string. It is up to the calling +program to do that if it needs to. +.P +This facility can be used to pass very long subject strings to +\fBpcre_dfa_exec()\fP. However, some care is needed for certain types of +pattern. +.P +1. If the pattern contains tests for the beginning or end of a line, you need +to pass the PCRE_NOTBOL or PCRE_NOTEOL options, as appropriate, when the +subject string for any call does not contain the beginning or end of a line. +.P +2. If the pattern contains backward assertions (including \eb or \eB), you need +to arrange for some overlap in the subject strings to allow for this. For +example, you could pass the subject in chunks that were 500 bytes long, but in +a buffer of 700 bytes, with the starting offset set to 200 and the previous 200 +bytes at the start of the buffer. +.P +3. Matching a subject string that is split into multiple segments does not +always produce exactly the same result as matching over one single long string. +The difference arises when there are multiple matching possibilities, because a +partial match result is given only when there are no completed matches in a +call to fBpcre_dfa_exec()\fP. This means that as soon as the shortest match has +been found, continuation to a new subject segment is no longer possible. +Consider this \fBpcretest\fP example: +.sp + re> /dog(sbody)?/ + data> do\eP\eD + Partial match: do + data> gsb\eR\eP\eD + 0: g + data> dogsbody\eD + 0: dogsbody + 1: dog +.sp +The pattern matches the words "dog" or "dogsbody". When the subject is +presented in several parts ("do" and "gsb" being the first two) the match stops +when "dog" has been found, and it is not possible to continue. On the other +hand, if "dogsbody" is presented as a single string, both matches are found. +.P +Because of this phenomenon, it does not usually make sense to end a pattern +that is going to be matched in this way with a variable repeat. . . .P .in 0 -Last updated: 08 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcrepattern.3 b/doc/pcrepattern.3 index 6f6a21a..fa39e53 100644 --- a/doc/pcrepattern.3 +++ b/doc/pcrepattern.3 @@ -26,15 +26,35 @@ in the main .\" page. .P +The remainder of this document discusses the patterns that are supported by +PCRE when its main matching function, \fBpcre_exec()\fP, is used. +From release 6.0, PCRE offers a second matching function, +\fBpcre_dfa_exec()\fP, which matches using a different algorithm that is not +Perl-compatible. The advantages and disadvantages of the alternative function, +and how it differs from the normal function, are discussed in the +.\" HREF +\fBpcrematching\fP +.\" +page. +.P A regular expression is a pattern that is matched against a subject string from left to right. Most characters stand for themselves in a pattern, and match the corresponding characters in the subject. As a trivial example, the pattern .sp The quick brown fox .sp -matches a portion of a subject string that is identical to itself. The power of -regular expressions comes from the ability to include alternatives and -repetitions in the pattern. These are encoded in the pattern by the use of +matches a portion of a subject string that is identical to itself. When +caseless matching is specified (the PCRE_CASELESS option), letters are matched +independently of case. In UTF-8 mode, PCRE always understands the concept of +case for characters whose values are less than 128, so caseless matching is +always possible. For characters with higher values, the concept of case is +supported if PCRE is compiled with Unicode property support, but not otherwise. +If you want to use caseless matching for characters 128 and above, you must +ensure that PCRE is compiled with Unicode property support as well as with +UTF-8 support. +.P +The power of regular expressions comes from the ability to include alternatives +and repetitions in the pattern. These are encoded in the pattern by the use of \fImetacharacters\fP, which do not stand for themselves but instead are interpreted in some special way. .P @@ -527,9 +547,13 @@ class as a literal string of bytes, or by using the \ex{ escaping mechanism. When caseless matching is set, any letters in a class represent both their upper case and lower case versions, so for example, a caseless [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a -caseful version would. When running in UTF-8 mode, PCRE supports the concept of -case for characters with values greater than 128 only when it is compiled with -Unicode property support. +caseful version would. In UTF-8 mode, PCRE always understands the concept of +case for characters whose values are less than 128, so caseless matching is +always possible. For characters with higher values, the concept of case is +supported if PCRE is compiled with Unicode property support, but not otherwise. +If you want to use caseless matching for characters 128 and above, you must +ensure that PCRE is compiled with Unicode property support as well as with +UTF-8 support. .P The newline character is never treated in any special way in character classes, whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE options is. A class @@ -1451,6 +1475,6 @@ description of the interface to the callout function is given in the documentation. .P .in 0 -Last updated: 09 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcreperform.3 b/doc/pcreperform.3 index 999268e..82e454c 100644 --- a/doc/pcreperform.3 +++ b/doc/pcreperform.3 @@ -35,8 +35,8 @@ this, PCRE has to retry the match starting after every newline in the subject. .P If you are using such a pattern with subject strings that do not contain newlines, the best performance is obtained by setting PCRE_DOTALL, or starting -the pattern with ^.* to indicate explicit anchoring. That saves PCRE from -having to scan along the subject looking for a newline to restart at. +the pattern with ^.* or ^.*? to indicate explicit anchoring. That saves PCRE +from having to scan along the subject looking for a newline to restart at. .P Beware of patterns that contain nested indefinite repeats. These can take a long time to run when applied to a string that does not match. Consider the @@ -71,6 +71,6 @@ In many cases, the solution to this kind of performance issue is to use an atomic group or a possessive quantifier. .P .in 0 -Last updated: 09 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcreposix.3 b/doc/pcreposix.3 index 321dcd7..b67d6ff 100644 --- a/doc/pcreposix.3 +++ b/doc/pcreposix.3 @@ -33,8 +33,8 @@ package. See the .\" HREF \fBpcreapi\fP .\" -documentation for a description of PCRE's native API, which contains additional -functionality. +documentation for a description of PCRE's native API, which contains much +additional functionality. .P The functions described here are just wrapper functions that ultimately call the PCRE native API. Their prototypes are defined in the \fBpcreposix.h\fP @@ -76,6 +76,11 @@ about the compiled expression. The argument \fIcflags\fP is either zero, or contains one or more of the bits defined by the following macros: .sp + REG_DOTALL +.sp +The PCRE_DOTALL option is set when the expression is passed for compilation to +the native function. Note that REG_DOTALL is not part of the POSIX standard. +.sp REG_ICASE .sp The PCRE_CASELESS option is set when the expression is passed for compilation @@ -189,13 +194,13 @@ memory, after which \fIpreg\fP may no longer be used as a compiled expression. .SH AUTHOR .rs .sp -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel .br University Computing Service, .br Cambridge CB2 3QG, England. .P .in 0 -Last updated: 07 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcreprecompile.3 b/doc/pcreprecompile.3 index f08939b..f359b96 100644 --- a/doc/pcreprecompile.3 +++ b/doc/pcreprecompile.3 @@ -79,15 +79,16 @@ return a non-NULL value before trying to save the study data. .rs .sp Re-using a precompiled pattern is straightforward. Having reloaded it into main -memory, you pass its pointer to \fBpcre_exec()\fP in the usual way. This should -work even on another host, and even if that host has the opposite endianness to -the one where the pattern was compiled. +memory, you pass its pointer to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP in +the usual way. This should work even on another host, and even if that host has +the opposite endianness to the one where the pattern was compiled. .P However, if you passed a pointer to custom character tables when the pattern was compiled (the \fItableptr\fP argument of \fBpcre_compile()\fP), you must -now pass a similar pointer to \fBpcre_exec()\fP, because the value saved with -the compiled pattern will obviously be nonsense. A field in a -\fBpcre_extra()\fP block is used to pass this data, as described in the +now pass a similar pointer to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP, +because the value saved with the compiled pattern will obviously be nonsense. A +field in a \fBpcre_extra()\fP block is used to pass this data, as described in +the .\" HTML <a href="pcreapi.html#extradata"> .\" </a> section on matching a pattern @@ -107,7 +108,8 @@ If you saved study data with the compiled pattern, you need to create your own \fBpcre_extra\fP data block and set the \fIstudy_data\fP field to point to the reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in the \fIflags\fP field to indicate that study data is present. Then pass the -\fBpcre_extra\fP block to \fBpcre_exec()\fP in the usual way. +\fBpcre_extra\fP block to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP in the +usual way. . . .SH "COMPATIBILITY WITH DIFFERENT PCRE RELEASES" @@ -120,6 +122,6 @@ advertised), you will have to recompile them for release 5.0. However, from now on, it should be possible to make changes in a compabible manner. .P .in 0 -Last updated: 10 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcretest.1 b/doc/pcretest.1 index 0c06cb7..336abcf 100644 --- a/doc/pcretest.1 +++ b/doc/pcretest.1 @@ -4,7 +4,7 @@ pcretest - a program for testing Perl-compatible regular expressions. .SH SYNOPSIS .rs .sp -.B pcretest "[-C] [-d] [-i] [-m] [-o osize] [-p] [-t] [source]" +.B pcretest "[-C] [-d] [-dfa] [-i] [-m] [-o osize] [-p] [-t] [source]" .ti +5n .B "[destination]" .P @@ -31,11 +31,16 @@ Output the version number of the PCRE library, and all available information about the optional features that are included, and then exit. .TP 10 \fB-d\fP -Behave as if each regex had the \fB/D\fP (debug) modifier; the internal +Behave as if each regex has the \fB/D\fP (debug) modifier; the internal form is output after compilation. .TP 10 +\fB-dfa\fP +Behave as if each data line contains the \eD escape sequence; this causes the +alternative matching function, \fBpcre_dfa_exec()\fP, to be used instead of the +standard \fBpcre_exec()\fP function (more detail is given below). +.TP 10 \fB-i\fP -Behave as if each regex had the \fB/I\fP modifier; information about the +Behave as if each regex has the \fB/I\fP modifier; information about the compiled pattern is given after compilation. .TP 10 \fB-m\fP @@ -50,8 +55,9 @@ for 14 capturing subexpressions. The vector size can be changed for individual matching calls by including \eO in the data line (see below). .TP 10 \fB-p\fP -Behave as if each regex has \fB/P\fP modifier; the POSIX wrapper API is used -to call PCRE. None of the other options has any effect when \fB-p\fP is set. +Behave as if each regex has the \fB/P\fP modifier; the POSIX wrapper API is +used to call PCRE. None of the other options has any effect when \fB-p\fP is +set. .TP 10 \fB-t\fP Run each compile, study, and match many times with a timer, and output @@ -131,6 +137,7 @@ not correspond to anything in Perl: \fB/A\fP PCRE_ANCHORED \fB/C\fP PCRE_AUTO_CALLOUT \fB/E\fP PCRE_DOLLAR_ENDONLY + \fB/f\fP PCRE_FIRSTLINE \fB/N\fP PCRE_NO_AUTO_CAPTURE \fB/U\fP PCRE_UNGREEDY \fB/X\fP PCRE_EXTRA @@ -257,6 +264,8 @@ recognized: .\" JOIN \eC*n pass the number n (may be negative) as callout data; this is used as the callout return value + \eD use the \fBpcre_dfa_exec()\fP match function + \eF only shortest match for \fBpcre_dfa_exec()\fP .\" JOIN \eGdd call pcre_get_substring() for substring dd after a successful match (number less than 32) @@ -272,7 +281,10 @@ recognized: .\" JOIN \eOdd set the size of the output vector passed to \fBpcre_exec()\fP to dd (any number of digits) +.\" JOIN \eP pass the PCRE_PARTIAL option to \fBpcre_exec()\fP + or \fBpcre_dfa_exec()\fP + \eR pass the PCRE_DFA_RESTART option to \fBpcre_dfa_exec()\fP \eS output details of memory get/free calls during matching \eZ pass the PCRE_NOTEOL option to \fBpcre_exec()\fP .\" JOIN @@ -308,15 +320,38 @@ any number of hexadecimal digits inside the braces. The result is from one to six bytes, encoded according to the UTF-8 rules. . . -.SH "OUTPUT FROM PCRETEST" +.SH "THE ALTERNATIVE MATCHING FUNCTION" .rs .sp +By default, \fBpcretest\fP uses the standard PCRE matching function, +\fBpcre_exec()\fP to match each data line. From release 6.0, PCRE supports an +alternative matching function, \fBpcre_dfa_test()\fP, which operates in a +different way, and has some restrictions. The differences between the two +functions are described in the +.\" HREF +\fBpcrematching\fP +.\" +documentation. +.P +If a data line contains the \eD escape sequence, or if the command line +contains the \fB-dfa\fP option, the alternative matching function is called. +This function finds all possible matches at a given point. If, however, the \eF +escape sequence is present in the data line, it stops after the first match is +found. This is always the shortest possible match. +. +. +.SH "DEFAULT OUTPUT FROM PCRETEST" +.rs +.sp +This section describes the output when the normal matching function, +\fBpcre_exec()\fP, is being used. +.P When a match succeeds, pcretest outputs the list of captured substrings that \fBpcre_exec()\fP returns, starting with number 0 for the string that matched the whole pattern. Otherwise, it outputs "No match" or "Partial match" when \fBpcre_exec()\fP returns PCRE_ERROR_NOMATCH or PCRE_ERROR_PARTIAL, respectively, and otherwise the PCRE negative error number. Here is an example -of an interactive pcretest run. +of an interactive \fBpcretest\fP run. .sp $ pcretest PCRE version 5.00 07-Sep-2004 @@ -365,13 +400,68 @@ prompt is used for continuations), data lines may not. However newlines can be included in data by means of the \en escape. . . +.SH "OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION" +.rs +.sp +When the alternative matching function, \fBpcre_dfa_exec()\fP, is used (by +means of the \eD escape sequence or the \fB-dfa\fP command line option), the +output consists of a list of all the matches that start at the first point in +the subject where there is at least one match. For example: +.sp + re> /(tang|tangerine|tan)/ + data> yellow tangerine\eD + 0: tangerine + 1: tang + 2: tan +.sp +(Using the normal matching function on this data finds only "tang".) The +longest matching string is always given first (and numbered zero). +.P +If \fB/g\P is present on the pattern, the search for further matches resumes +at the end of the longest match. For example: +.sp + re> /(tang|tangerine|tan)/g + data> yellow tangerine and tangy sultana\eD + 0: tangerine + 1: tang + 2: tan + 0: tang + 1: tan + 0: tan +.sp +Since the matching function does not support substring capture, the escape +sequences that are concerned with captured substrings are not relevant. +. +. +.SH "RESTARTING AFTER A PARTIAL MATCH" +.rs +.sp +When the alternative matching function has given the PCRE_ERROR_PARTIAL return, +indicating that the subject partially matched the pattern, you can restart the +match with additional subject data by means of the \eR escape sequence. For +example: +.sp + re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/ + data> 23ja\eP\eD + Partial match: 23ja + data> n05\eR\eD + 0: n05 +.sp +For further information about partial matching, see the +.\" HREF +\fBpcrepartial\fP +.\" +documentation. +. +. .SH CALLOUTS .rs .sp If the pattern contains any callout requests, \fBpcretest\fP's callout function -is called during matching. By default, it displays the callout number, the -start and current positions in the text at the callout time, and the next -pattern item to be tested. For example, the output +is called during matching. This works with both matching functions. By default, +the called function displays the callout number, the start and current +positions in the text at the callout time, and the next pattern item to be +tested. For example, the output .sp --->pqrabcdef 0 ^ ^ \ed @@ -396,7 +486,7 @@ example: 0: E* .sp The callout function in \fBpcretest\fP returns zero (carry on matching) by -default, but you can use an \eC item in a data line (as described above) to +default, but you can use a \eC item in a data line (as described above) to change this. .P Inserting callouts can be helpful when using \fBpcretest\fP to check @@ -471,13 +561,13 @@ result is undefined. .SH AUTHOR .rs .sp -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel .br University Computing Service, .br Cambridge CB2 3QG, England. .P .in 0 -Last updated: 10 September 2004 +Last updated: 28 February 2005 .br -Copyright (c) 1997-2004 University of Cambridge. +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/pcretest.txt b/doc/pcretest.txt index 7da6889..2badffa 100644 --- a/doc/pcretest.txt +++ b/doc/pcretest.txt @@ -7,7 +7,7 @@ NAME SYNOPSIS - pcretest [-C] [-d] [-i] [-m] [-o osize] [-p] [-t] [source] + pcretest [-C] [-d] [-dfa] [-i] [-m] [-o osize] [-p] [-t] [source] [destination] pcretest was written as a test program for the PCRE regular expression @@ -24,95 +24,100 @@ OPTIONS able information about the optional features that are included, and then exit. - -d Behave as if each regex had the /D (debug) modifier; the + -d Behave as if each regex has the /D (debug) modifier; the internal form is output after compilation. - -i Behave as if each regex had the /I modifier; information + -dfa Behave as if each data line contains the \D escape sequence; + this causes the alternative matching function, + pcre_dfa_exec(), to be used instead of the standard + pcre_exec() function (more detail is given below). + + -i Behave as if each regex has the /I modifier; information about the compiled pattern is given after compilation. - -m Output the size of each compiled pattern after it has been - compiled. This is equivalent to adding /M to each regular - expression. For compatibility with earlier versions of + -m Output the size of each compiled pattern after it has been + compiled. This is equivalent to adding /M to each regular + expression. For compatibility with earlier versions of pcretest, -s is a synonym for -m. - -o osize Set the number of elements in the output vector that is used - when calling pcre_exec() to be osize. The default value is + -o osize Set the number of elements in the output vector that is used + when calling pcre_exec() to be osize. The default value is 45, which is enough for 14 capturing subexpressions. The vec- - tor size can be changed for individual matching calls by + tor size can be changed for individual matching calls by including \O in the data line (see below). - -p Behave as if each regex has /P modifier; the POSIX wrapper - API is used to call PCRE. None of the other options has any - effect when -p is set. + -p Behave as if each regex has the /P modifier; the POSIX wrap- + per API is used to call PCRE. None of the other options has + any effect when -p is set. - -t Run each compile, study, and match many times with a timer, - and output resulting time per compile or match (in millisec- - onds). Do not set -m with -t, because you will then get the - size output a zillion times, and the timing will be dis- + -t Run each compile, study, and match many times with a timer, + and output resulting time per compile or match (in millisec- + onds). Do not set -m with -t, because you will then get the + size output a zillion times, and the timing will be dis- torted. DESCRIPTION - If pcretest is given two filename arguments, it reads from the first + If pcretest is given two filename arguments, it reads from the first and writes to the second. If it is given only one filename argument, it - reads from that file and writes to stdout. Otherwise, it reads from - stdin and writes to stdout, and prompts for each line of input, using + reads from that file and writes to stdout. Otherwise, it reads from + stdin and writes to stdout, and prompts for each line of input, using "re>" to prompt for regular expressions, and "data>" to prompt for data lines. The program handles any number of sets of input on a single input file. - Each set starts with a regular expression, and continues with any num- + Each set starts with a regular expression, and continues with any num- ber of data lines to be matched against the pattern. - Each data line is matched separately and independently. If you want to - do multiple-line matches, you have to use the \n escape sequence in a - single line of input to encode the newline characters. The maximum + Each data line is matched separately and independently. If you want to + do multiple-line matches, you have to use the \n escape sequence in a + single line of input to encode the newline characters. The maximum length of data line is 30,000 characters. - An empty line signals the end of the data lines, at which point a new - regular expression is read. The regular expressions are given enclosed + An empty line signals the end of the data lines, at which point a new + regular expression is read. The regular expressions are given enclosed in any non-alphanumeric delimiters other than backslash, for example /(a|bc)x+yz/ - White space before the initial delimiter is ignored. A regular expres- - sion may be continued over several input lines, in which case the new- - line characters are included within it. It is possible to include the + White space before the initial delimiter is ignored. A regular expres- + sion may be continued over several input lines, in which case the new- + line characters are included within it. It is possible to include the delimiter within the pattern by escaping it, for example /abc\/def/ - If you do so, the escape and the delimiter form part of the pattern, - but since delimiters are always non-alphanumeric, this does not affect - its interpretation. If the terminating delimiter is immediately fol- + If you do so, the escape and the delimiter form part of the pattern, + but since delimiters are always non-alphanumeric, this does not affect + its interpretation. If the terminating delimiter is immediately fol- lowed by a backslash, for example, /abc/\ - then a backslash is added to the end of the pattern. This is done to - provide a way of testing the error condition that arises if a pattern + then a backslash is added to the end of the pattern. This is done to + provide a way of testing the error condition that arises if a pattern finishes with a backslash, because /abc\/ - is interpreted as the first line of a pattern that starts with "abc/", + is interpreted as the first line of a pattern that starts with "abc/", causing pcretest to read the next line as a continuation of the regular expression. PATTERN MODIFIERS - A pattern may be followed by any number of modifiers, which are mostly - single characters. Following Perl usage, these are referred to below - as, for example, "the /i modifier", even though the delimiter of the - pattern need not always be a slash, and no slash is used when writing - modifiers. Whitespace may appear between the final pattern delimiter + A pattern may be followed by any number of modifiers, which are mostly + single characters. Following Perl usage, these are referred to below + as, for example, "the /i modifier", even though the delimiter of the + pattern need not always be a slash, and no slash is used when writing + modifiers. Whitespace may appear between the final pattern delimiter and the first modifier, and between the modifiers themselves. The /i, /m, /s, and /x modifiers set the PCRE_CASELESS, PCRE_MULTILINE, - PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when pcre_com- - pile() is called. These four modifier letters have the same effect as + PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when pcre_com- + pile() is called. These four modifier letters have the same effect as they do in Perl. For example: /caseless/i @@ -123,95 +128,96 @@ PATTERN MODIFIERS /A PCRE_ANCHORED /C PCRE_AUTO_CALLOUT /E PCRE_DOLLAR_ENDONLY + /f PCRE_FIRSTLINE /N PCRE_NO_AUTO_CAPTURE /U PCRE_UNGREEDY /X PCRE_EXTRA - Searching for all possible matches within each subject string can be - requested by the /g or /G modifier. After finding a match, PCRE is + Searching for all possible matches within each subject string can be + requested by the /g or /G modifier. After finding a match, PCRE is called again to search the remainder of the subject string. The differ- ence between /g and /G is that the former uses the startoffset argument - to pcre_exec() to start searching at a new point within the entire - string (which is in effect what Perl does), whereas the latter passes - over a shortened substring. This makes a difference to the matching + to pcre_exec() to start searching at a new point within the entire + string (which is in effect what Perl does), whereas the latter passes + over a shortened substring. This makes a difference to the matching process if the pattern begins with a lookbehind assertion (including \b or \B). - If any call to pcre_exec() in a /g or /G sequence matches an empty - string, the next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED - flags set in order to search for another, non-empty, match at the same - point. If this second match fails, the start offset is advanced by - one, and the normal match is retried. This imitates the way Perl han- + If any call to pcre_exec() in a /g or /G sequence matches an empty + string, the next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED + flags set in order to search for another, non-empty, match at the same + point. If this second match fails, the start offset is advanced by + one, and the normal match is retried. This imitates the way Perl han- dles such cases when using the /g modifier or the split() function. There are yet more modifiers for controlling the way pcretest operates. - The /+ modifier requests that as well as outputting the substring that - matched the entire pattern, pcretest should in addition output the - remainder of the subject string. This is useful for tests where the + The /+ modifier requests that as well as outputting the substring that + matched the entire pattern, pcretest should in addition output the + remainder of the subject string. This is useful for tests where the subject contains multiple copies of the same substring. - The /L modifier must be followed directly by the name of a locale, for + The /L modifier must be followed directly by the name of a locale, for example, /pattern/Lfr_FR For this reason, it must be the last modifier. The given locale is set, - pcre_maketables() is called to build a set of character tables for the - locale, and this is then passed to pcre_compile() when compiling the - regular expression. Without an /L modifier, NULL is passed as the - tables pointer; that is, /L applies only to the expression on which it + pcre_maketables() is called to build a set of character tables for the + locale, and this is then passed to pcre_compile() when compiling the + regular expression. Without an /L modifier, NULL is passed as the + tables pointer; that is, /L applies only to the expression on which it appears. - The /I modifier requests that pcretest output information about the - compiled pattern (whether it is anchored, has a fixed first character, - and so on). It does this by calling pcre_fullinfo() after compiling a - pattern. If the pattern is studied, the results of that are also out- + The /I modifier requests that pcretest output information about the + compiled pattern (whether it is anchored, has a fixed first character, + and so on). It does this by calling pcre_fullinfo() after compiling a + pattern. If the pattern is studied, the results of that are also out- put. The /D modifier is a PCRE debugging feature, which also assumes /I. It - causes the internal form of compiled regular expressions to be output + causes the internal form of compiled regular expressions to be output after compilation. If the pattern was studied, the information returned is also output. The /F modifier causes pcretest to flip the byte order of the fields in - the compiled pattern that contain 2-byte and 4-byte numbers. This - facility is for testing the feature in PCRE that allows it to execute + the compiled pattern that contain 2-byte and 4-byte numbers. This + facility is for testing the feature in PCRE that allows it to execute patterns that were compiled on a host with a different endianness. This - feature is not available when the POSIX interface to PCRE is being - used, that is, when the /P pattern modifier is specified. See also the + feature is not available when the POSIX interface to PCRE is being + used, that is, when the /P pattern modifier is specified. See also the section about saving and reloading compiled patterns below. - The /S modifier causes pcre_study() to be called after the expression + The /S modifier causes pcre_study() to be called after the expression has been compiled, and the results used when the expression is matched. - The /M modifier causes the size of memory block used to hold the com- + The /M modifier causes the size of memory block used to hold the com- piled pattern to be output. - The /P modifier causes pcretest to call PCRE via the POSIX wrapper API - rather than its native API. When this is done, all other modifiers - except /i, /m, and /+ are ignored. REG_ICASE is set if /i is present, - and REG_NEWLINE is set if /m is present. The wrapper functions force - PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set. + The /P modifier causes pcretest to call PCRE via the POSIX wrapper API + rather than its native API. When this is done, all other modifiers + except /i, /m, and /+ are ignored. REG_ICASE is set if /i is present, + and REG_NEWLINE is set if /m is present. The wrapper functions force + PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set. - The /8 modifier causes pcretest to call PCRE with the PCRE_UTF8 option - set. This turns on support for UTF-8 character handling in PCRE, pro- - vided that it was compiled with this support enabled. This modifier + The /8 modifier causes pcretest to call PCRE with the PCRE_UTF8 option + set. This turns on support for UTF-8 character handling in PCRE, pro- + vided that it was compiled with this support enabled. This modifier also causes any non-printing characters in output strings to be printed using the \x{hh...} notation if they are valid UTF-8 sequences. - If the /? modifier is used with /8, it causes pcretest to call - pcre_compile() with the PCRE_NO_UTF8_CHECK option, to suppress the + If the /? modifier is used with /8, it causes pcretest to call + pcre_compile() with the PCRE_NO_UTF8_CHECK option, to suppress the checking of the string for UTF-8 validity. DATA LINES - Before each data line is passed to pcre_exec(), leading and trailing - whitespace is removed, and it is then scanned for \ escapes. Some of - these are pretty esoteric features, intended for checking out some of - the more complicated features of PCRE. If you are just testing "ordi- - nary" regular expressions, you probably don't need any of these. The + Before each data line is passed to pcre_exec(), leading and trailing + whitespace is removed, and it is then scanned for \ escapes. Some of + these are pretty esoteric features, intended for checking out some of + the more complicated features of PCRE. If you are just testing "ordi- + nary" regular expressions, you probably don't need any of these. The following escapes are recognized: \a alarm (= BEL) @@ -242,6 +248,8 @@ DATA LINES reached for the nth time \C*n pass the number n (may be negative) as callout data; this is used as the callout return value + \D use the pcre_dfa_exec() match function + \F only shortest match for pcre_dfa_exec() \Gdd call pcre_get_substring() for substring dd after a successful match (number less than 32) \Gname call pcre_get_named_substring() for substring @@ -254,6 +262,8 @@ DATA LINES \Odd set the size of the output vector passed to pcre_exec() to dd (any number of digits) \P pass the PCRE_PARTIAL option to pcre_exec() + or pcre_dfa_exec() + \R pass the PCRE_DFA_RESTART option to pcre_dfa_exec() \S output details of memory get/free calls during matching \Z pass the PCRE_NOTEOL option to pcre_exec() \? pass the PCRE_NO_UTF8_CHECK option to @@ -261,35 +271,53 @@ DATA LINES \>dd start the match at offset dd (any number of digits); this sets the startoffset argument for pcre_exec() - A backslash followed by anything else just escapes the anything else. - If the very last character is a backslash, it is ignored. This gives a - way of passing an empty line as data, since a real empty line termi- + A backslash followed by anything else just escapes the anything else. + If the very last character is a backslash, it is ignored. This gives a + way of passing an empty line as data, since a real empty line termi- nates the data input. - If \M is present, pcretest calls pcre_exec() several times, with dif- - ferent values in the match_limit field of the pcre_extra data struc- - ture, until it finds the minimum number that is needed for pcre_exec() - to complete. This number is a measure of the amount of recursion and - backtracking that takes place, and checking it out can be instructive. - For most simple matches, the number is quite small, but for patterns - with very large numbers of matching possibilities, it can become large + If \M is present, pcretest calls pcre_exec() several times, with dif- + ferent values in the match_limit field of the pcre_extra data struc- + ture, until it finds the minimum number that is needed for pcre_exec() + to complete. This number is a measure of the amount of recursion and + backtracking that takes place, and checking it out can be instructive. + For most simple matches, the number is quite small, but for patterns + with very large numbers of matching possibilities, it can become large very quickly with increasing length of subject string. - When \O is used, the value specified may be higher or lower than the + When \O is used, the value specified may be higher or lower than the size set by the -O command line option (or defaulted to 45); \O applies only to the call of pcre_exec() for the line in which it appears. - If the /P modifier was present on the pattern, causing the POSIX wrap- - per API to be used, only \B and \Z have any effect, causing REG_NOTBOL + If the /P modifier was present on the pattern, causing the POSIX wrap- + per API to be used, only \B and \Z have any effect, causing REG_NOTBOL and REG_NOTEOL to be passed to regexec() respectively. - The use of \x{hh...} to represent UTF-8 characters is not dependent on - the use of the /8 modifier on the pattern. It is recognized always. - There may be any number of hexadecimal digits inside the braces. The - result is from one to six bytes, encoded according to the UTF-8 rules. + The use of \x{hh...} to represent UTF-8 characters is not dependent on + the use of the /8 modifier on the pattern. It is recognized always. + There may be any number of hexadecimal digits inside the braces. The + result is from one to six bytes, encoded according to the UTF-8 rules. + + +THE ALTERNATIVE MATCHING FUNCTION + + By default, pcretest uses the standard PCRE matching function, + pcre_exec() to match each data line. From release 6.0, PCRE supports an + alternative matching function, pcre_dfa_test(), which operates in a + different way, and has some restrictions. The differences between the + two functions are described in the pcrematching documentation. + + If a data line contains the \D escape sequence, or if the command line + contains the -dfa option, the alternative matching function is called. + This function finds all possible matches at a given point. If, however, + the \F escape sequence is present in the data line, it stops after the + first match is found. This is always the shortest possible match. -OUTPUT FROM PCRETEST +DEFAULT OUTPUT FROM PCRETEST + + This section describes the output when the normal matching function, + pcre_exec(), is being used. When a match succeeds, pcretest outputs the list of captured substrings that pcre_exec() returns, starting with number 0 for the string that @@ -345,25 +373,76 @@ OUTPUT FROM PCRETEST lines can be included in data by means of the \n escape. +OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION + + When the alternative matching function, pcre_dfa_exec(), is used (by + means of the \D escape sequence or the -dfa command line option), the + output consists of a list of all the matches that start at the first + point in the subject where there is at least one match. For example: + + re> /(tang|tangerine|tan)/ + data> yellow tangerine\D + 0: tangerine + 1: tang + 2: tan + + (Using the normal matching function on this data finds only "tang".) + The longest matching string is always given first (and numbered zero). + + If /gP is present on the pattern, the search for further matches + resumes at the end of the longest match. For example: + + re> /(tang|tangerine|tan)/g + data> yellow tangerine and tangy sultana\D + 0: tangerine + 1: tang + 2: tan + 0: tang + 1: tan + 0: tan + + Since the matching function does not support substring capture, the + escape sequences that are concerned with captured substrings are not + relevant. + + +RESTARTING AFTER A PARTIAL MATCH + + When the alternative matching function has given the PCRE_ERROR_PARTIAL + return, indicating that the subject partially matched the pattern, you + can restart the match with additional subject data by means of the \R + escape sequence. For example: + + re> /^?(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)$/ + data> 23ja\P\D + Partial match: 23ja + data> n05\R\D + 0: n05 + + For further information about partial matching, see the pcrepartial + documentation. + + CALLOUTS If the pattern contains any callout requests, pcretest's callout func- - tion is called during matching. By default, it displays the callout - number, the start and current positions in the text at the callout - time, and the next pattern item to be tested. For example, the output + tion is called during matching. This works with both matching func- + tions. By default, the called function displays the callout number, the + start and current positions in the text at the callout time, and the + next pattern item to be tested. For example, the output --->pqrabcdef 0 ^ ^ \d - indicates that callout number 0 occurred for a match attempt starting - at the fourth character of the subject string, when the pointer was at - the seventh character of the data, and when the next pattern item was - \d. Just one circumflex is output if the start and current positions + indicates that callout number 0 occurred for a match attempt starting + at the fourth character of the subject string, when the pointer was at + the seventh character of the data, and when the next pattern item was + \d. Just one circumflex is output if the start and current positions are the same. Callouts numbered 255 are assumed to be automatic callouts, inserted as - a result of the /C pattern modifier. In this case, instead of showing - the callout number, the offset in the pattern, preceded by a plus, is + a result of the /C pattern modifier. In this case, instead of showing + the callout number, the offset in the pattern, preceded by a plus, is output. For example: re> /\d?[A-E]\*/C @@ -375,76 +454,76 @@ CALLOUTS +10 ^ ^ 0: E* - The callout function in pcretest returns zero (carry on matching) by - default, but you can use an \C item in a data line (as described above) + The callout function in pcretest returns zero (carry on matching) by + default, but you can use a \C item in a data line (as described above) to change this. - Inserting callouts can be helpful when using pcretest to check compli- - cated regular expressions. For further information about callouts, see + Inserting callouts can be helpful when using pcretest to check compli- + cated regular expressions. For further information about callouts, see the pcrecallout documentation. SAVING AND RELOADING COMPILED PATTERNS - The facilities described in this section are not available when the + The facilities described in this section are not available when the POSIX inteface to PCRE is being used, that is, when the /P pattern mod- ifier is specified. When the POSIX interface is not in use, you can cause pcretest to write - a compiled pattern to a file, by following the modifiers with > and a + a compiled pattern to a file, by following the modifiers with > and a file name. For example: /pattern/im >/some/file - See the pcreprecompile documentation for a discussion about saving and + See the pcreprecompile documentation for a discussion about saving and re-using compiled patterns. - The data that is written is binary. The first eight bytes are the - length of the compiled pattern data followed by the length of the - optional study data, each written as four bytes in big-endian order - (most significant byte first). If there is no study data (either the + The data that is written is binary. The first eight bytes are the + length of the compiled pattern data followed by the length of the + optional study data, each written as four bytes in big-endian order + (most significant byte first). If there is no study data (either the pattern was not studied, or studying did not return any data), the sec- - ond length is zero. The lengths are followed by an exact copy of the + ond length is zero. The lengths are followed by an exact copy of the compiled pattern. If there is additional study data, this follows imme- - diately after the compiled pattern. After writing the file, pcretest + diately after the compiled pattern. After writing the file, pcretest expects to read a new pattern. A saved pattern can be reloaded into pcretest by specifing < and a file - name instead of a pattern. The name of the file must not contain a < - character, as otherwise pcretest will interpret the line as a pattern + name instead of a pattern. The name of the file must not contain a < + character, as otherwise pcretest will interpret the line as a pattern delimited by < characters. For example: re> </some/file Compiled regex loaded from /some/file No study data - When the pattern has been loaded, pcretest proceeds to read data lines + When the pattern has been loaded, pcretest proceeds to read data lines in the usual way. - You can copy a file written by pcretest to a different host and reload - it there, even if the new host has opposite endianness to the one on - which the pattern was compiled. For example, you can compile on an i86 + You can copy a file written by pcretest to a different host and reload + it there, even if the new host has opposite endianness to the one on + which the pattern was compiled. For example, you can compile on an i86 machine and run on a SPARC machine. - File names for saving and reloading can be absolute or relative, but - note that the shell facility of expanding a file name that starts with + File names for saving and reloading can be absolute or relative, but + note that the shell facility of expanding a file name that starts with a tilde (~) is not available. - The ability to save and reload files in pcretest is intended for test- - ing and experimentation. It is not intended for production use because - only a single pattern can be written to a file. Furthermore, there is - no facility for supplying custom character tables for use with a - reloaded pattern. If the original pattern was compiled with custom - tables, an attempt to match a subject string using a reloaded pattern - is likely to cause pcretest to crash. Finally, if you attempt to load + The ability to save and reload files in pcretest is intended for test- + ing and experimentation. It is not intended for production use because + only a single pattern can be written to a file. Furthermore, there is + no facility for supplying custom character tables for use with a + reloaded pattern. If the original pattern was compiled with custom + tables, an attempt to match a subject string using a reloaded pattern + is likely to cause pcretest to crash. Finally, if you attempt to load a file that is not in the correct format, the result is undefined. AUTHOR - Philip Hazel <ph10@cam.ac.uk> + Philip Hazel University Computing Service, Cambridge CB2 3QG, England. -Last updated: 10 September 2004 -Copyright (c) 1997-2004 University of Cambridge. +Last updated: 28 February 2005 +Copyright (c) 1997-2005 University of Cambridge. diff --git a/doc/perltest.txt b/doc/perltest.txt index f1d2c15..ca02690 100644 --- a/doc/perltest.txt +++ b/doc/perltest.txt @@ -29,5 +29,5 @@ make use of the special upper case modifiers and escapes that pcretest uses to test some features of PCRE. Some of these files also contains malformed regular expressions, in order to check that PCRE diagnoses them correctly. -Philip Hazel <ph10@cam.ac.uk> +Philip Hazel September 2004 diff --git a/libpcre.def b/libpcre.def index 2b35d10..01db4bd 100644 --- a/libpcre.def +++ b/libpcre.def @@ -6,6 +6,7 @@ pcre_config pcre_callout pcre_compile pcre_copy_substring +pcre_dfa_exec pcre_exec pcre_get_substring pcre_get_stringnumber diff --git a/libpcreposix.def b/libpcreposix.def index 5723440..5f30247 100644 --- a/libpcreposix.def +++ b/libpcreposix.def @@ -6,6 +6,7 @@ pcre_config pcre_callout pcre_compile pcre_copy_substring +pcre_dfa_exec pcre_exec pcre_get_substring pcre_get_stringnumber @@ -5,6 +5,7 @@ pcre_free DATA pcre_compile pcre_copy_substring +pcre_dfa_exec pcre_exec pcre_get_substring pcre_get_substring_list @@ -5,7 +5,7 @@ /* In its original form, this is the .in file that is transformed by "configure" into pcre.h. - Copyright (c) 1997-2004 University of Cambridge + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -46,7 +46,7 @@ make changes to pcre.in. */ #define PCRE_MINOR @PCRE_MINOR@ #define PCRE_DATE @PCRE_DATE@ -/* Win32 uses DLL by default */ +/* Win32 uses DLL by default; it needs special stuff for exported functions. */ #ifdef _WIN32 # ifdef PCRE_DEFINITION @@ -59,8 +59,15 @@ make changes to pcre.in. */ # endif # endif #endif + +/* For other operating systems, we use the standard "extern". */ + #ifndef PCRE_DATA_SCOPE -# define PCRE_DATA_SCOPE extern +# ifdef __cplusplus +# define PCRE_DATA_SCOPE extern "C" +# else +# define PCRE_DATA_SCOPE extern +# endif #endif /* Have to include stdlib.h in order to ensure that size_t is defined; @@ -76,22 +83,25 @@ extern "C" { /* Options */ -#define PCRE_CASELESS 0x0001 -#define PCRE_MULTILINE 0x0002 -#define PCRE_DOTALL 0x0004 -#define PCRE_EXTENDED 0x0008 -#define PCRE_ANCHORED 0x0010 -#define PCRE_DOLLAR_ENDONLY 0x0020 -#define PCRE_EXTRA 0x0040 -#define PCRE_NOTBOL 0x0080 -#define PCRE_NOTEOL 0x0100 -#define PCRE_UNGREEDY 0x0200 -#define PCRE_NOTEMPTY 0x0400 -#define PCRE_UTF8 0x0800 -#define PCRE_NO_AUTO_CAPTURE 0x1000 -#define PCRE_NO_UTF8_CHECK 0x2000 -#define PCRE_AUTO_CALLOUT 0x4000 -#define PCRE_PARTIAL 0x8000 +#define PCRE_CASELESS 0x00000001 +#define PCRE_MULTILINE 0x00000002 +#define PCRE_DOTALL 0x00000004 +#define PCRE_EXTENDED 0x00000008 +#define PCRE_ANCHORED 0x00000010 +#define PCRE_DOLLAR_ENDONLY 0x00000020 +#define PCRE_EXTRA 0x00000040 +#define PCRE_NOTBOL 0x00000080 +#define PCRE_NOTEOL 0x00000100 +#define PCRE_UNGREEDY 0x00000200 +#define PCRE_NOTEMPTY 0x00000400 +#define PCRE_UTF8 0x00000800 +#define PCRE_NO_AUTO_CAPTURE 0x00001000 +#define PCRE_NO_UTF8_CHECK 0x00002000 +#define PCRE_AUTO_CALLOUT 0x00004000 +#define PCRE_PARTIAL 0x00008000 +#define PCRE_DFA_SHORTEST 0x00010000 +#define PCRE_DFA_RESTART 0x00020000 +#define PCRE_FIRSTLINE 0x00040000 /* Exec-time and get/set-time error codes */ @@ -110,6 +120,11 @@ extern "C" { #define PCRE_ERROR_BADPARTIAL (-13) #define PCRE_ERROR_INTERNAL (-14) #define PCRE_ERROR_BADCOUNT (-15) +#define PCRE_ERROR_DFA_UITEM (-16) +#define PCRE_ERROR_DFA_UCOND (-17) +#define PCRE_ERROR_DFA_UMLIMIT (-18) +#define PCRE_ERROR_DFA_WSSIZE (-19) +#define PCRE_ERROR_DFA_RECURSE (-20) /* Request types for pcre_fullinfo() */ @@ -187,9 +202,8 @@ typedef struct pcre_callout_block { /* Indirection for store get and free functions. These can be set to alternative malloc/free functions if required. Special ones are used in the non-recursive case for "frames". There is also an optional callout function -that is triggered by the (?) regex item. Some magic is required for Win32 DLL; -it is null on other OS. For Virtual Pascal, these have to be different again. -*/ +that is triggered by the (?) regex item. For Virtual Pascal, these definitions +have to take another form. */ #ifndef VPCOMPAT PCRE_DATA_SCOPE void *(*pcre_malloc)(size_t); @@ -198,39 +212,44 @@ PCRE_DATA_SCOPE void *(*pcre_stack_malloc)(size_t); PCRE_DATA_SCOPE void (*pcre_stack_free)(void *); PCRE_DATA_SCOPE int (*pcre_callout)(pcre_callout_block *); #else /* VPCOMPAT */ -extern void *pcre_malloc(size_t); -extern void pcre_free(void *); -extern void *pcre_stack_malloc(size_t); -extern void pcre_stack_free(void *); -extern int pcre_callout(pcre_callout_block *); +PCRE_DATA_SCOPE void *pcre_malloc(size_t); +PCRE_DATA_SCOPE void pcre_free(void *); +PCRE_DATA_SCOPE void *pcre_stack_malloc(size_t); +PCRE_DATA_SCOPE void pcre_stack_free(void *); +PCRE_DATA_SCOPE int pcre_callout(pcre_callout_block *); #endif /* VPCOMPAT */ /* Exported PCRE functions */ -extern pcre *pcre_compile(const char *, int, const char **, - int *, const unsigned char *); -extern int pcre_config(int, void *); -extern int pcre_copy_named_substring(const pcre *, const char *, - int *, int, const char *, char *, int); -extern int pcre_copy_substring(const char *, int *, int, int, - char *, int); -extern int pcre_exec(const pcre *, const pcre_extra *, - const char *, int, int, int, int *, int); -extern void pcre_free_substring(const char *); -extern void pcre_free_substring_list(const char **); -extern int pcre_fullinfo(const pcre *, const pcre_extra *, int, - void *); -extern int pcre_get_named_substring(const pcre *, const char *, - int *, int, const char *, const char **); -extern int pcre_get_stringnumber(const pcre *, const char *); -extern int pcre_get_substring(const char *, int *, int, int, - const char **); -extern int pcre_get_substring_list(const char *, int *, int, - const char ***); -extern int pcre_info(const pcre *, int *, int *); -extern const unsigned char *pcre_maketables(void); -extern pcre_extra *pcre_study(const pcre *, int, const char **); -extern const char *pcre_version(void); +PCRE_DATA_SCOPE pcre *pcre_compile(const char *, int, const char **, int *, + const unsigned char *); +PCRE_DATA_SCOPE pcre *pcre_compile2(const char *, int, int *, const char **, + int *, const unsigned char *); +PCRE_DATA_SCOPE int pcre_config(int, void *); +PCRE_DATA_SCOPE int pcre_copy_named_substring(const pcre *, const char *, + int *, int, const char *, char *, int); +PCRE_DATA_SCOPE int pcre_copy_substring(const char *, int *, int, int, char *, + int); +PCRE_DATA_SCOPE int pcre_dfa_exec(const pcre *, const pcre_extra *, + const char *, int, int, int, int *, int , int *, int); +PCRE_DATA_SCOPE int pcre_exec(const pcre *, const pcre_extra *, const char *, + int, int, int, int *, int); +PCRE_DATA_SCOPE void pcre_free_substring(const char *); +PCRE_DATA_SCOPE void pcre_free_substring_list(const char **); +PCRE_DATA_SCOPE int pcre_fullinfo(const pcre *, const pcre_extra *, int, + void *); +PCRE_DATA_SCOPE int pcre_get_named_substring(const pcre *, const char *, + int *, int, const char *, const char **); +PCRE_DATA_SCOPE int pcre_get_stringnumber(const pcre *, const char *); +PCRE_DATA_SCOPE int pcre_get_substring(const char *, int *, int, int, + const char **); +PCRE_DATA_SCOPE int pcre_get_substring_list(const char *, int *, int, + const char ***); +PCRE_DATA_SCOPE int pcre_info(const pcre *, int *, int *); +PCRE_DATA_SCOPE const unsigned char *pcre_maketables(void); +PCRE_DATA_SCOPE int pcre_refcount(pcre *, int); +PCRE_DATA_SCOPE pcre_extra *pcre_study(const pcre *, int, const char **); +PCRE_DATA_SCOPE const char *pcre_version(void); #ifdef __cplusplus } /* extern "C" */ @@ -2,14 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -This is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. See -the file Tech.Notes for some information on the internals. +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2004 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -41,31 +38,16 @@ POSSIBILITY OF SUCH DAMAGE. */ -/* Define DEBUG to get debugging output on stdout. */ -/* #define DEBUG */ +/* This module contains the external function pcre_compile(), along with +supporting internal functions that are not used by other modules. */ -/* Use a macro for debugging printing, 'cause that eliminates the use of #ifdef -inline, and there are *still* stupid compilers about that don't like indented -pre-processor statements. I suppose it's only been 10 years... */ -#ifdef DEBUG -#define DPRINTF(p) printf p -#else -#define DPRINTF(p) /*nothing*/ -#endif +#include "pcre_internal.h" -/* Include the internals header, which itself includes "config.h", the Standard -C headers, and the external pcre header. */ -#include "internal.h" - -/* If Unicode Property support is wanted, include a private copy of the -function that does it, and the table that translates names to numbers. */ - -#ifdef SUPPORT_UCP -#include "ucp.c" -#include "ucptypetable.c" -#endif +/************************************************* +* Code parameters and static tables * +*************************************************/ /* Maximum number of items on the nested bracket stacks at compile time. This applies to the nesting of all kinds of parentheses. It does not limit @@ -76,29 +58,6 @@ compile time. */ #define BRASTACK_SIZE 200 -/* Maximum number of ints of offset to save on the stack for recursive calls. -If the offset vector is bigger, malloc is used. This should be a multiple of 3, -because the offset vector is always a multiple of 3 long. */ - -#define REC_STACK_SAVE_MAX 30 - - -/* The maximum remaining length of subject we are prepared to search for a -req_byte match. */ - -#define REQ_BYTE_MAX 1000 - - -/* Table of sizes for the fixed-length opcodes. It's defined in a macro so that -the definition is next to the definition of the opcodes in internal.h. */ - -static const uschar OP_lengths[] = { OP_LENGTHS }; - -/* Min and max values for the common repeats; for the maxima, 0 => infinity */ - -static const char rep_min[] = { 0, 0, 1, 1, 0, 0 }; -static const char rep_max[] = { 0, 0, 0, 0, 1, 1 }; - /* Table for handling escaped characters in the range '0'-'z'. Positive returns are simple data values; negative values are for special things like \d and so on. Zero means further processing is needed (for things like \x), or the escape @@ -180,6 +139,71 @@ static const int posix_class_maps[] = { cbit_xdigit,-1, -1 /* xdigit */ }; + +/* The texts of compile-time error messages. These are "char *" because they +are passed to the outside world. */ + +static const char *error_texts[] = { + "no error", + "\\ at end of pattern", + "\\c at end of pattern", + "unrecognized character follows \\", + "numbers out of order in {} quantifier", + /* 5 */ + "number too big in {} quantifier", + "missing terminating ] for character class", + "invalid escape sequence in character class", + "range out of order in character class", + "nothing to repeat", + /* 10 */ + "operand of unlimited repeat could match the empty string", + "internal error: unexpected repeat", + "unrecognized character after (?", + "POSIX named classes are supported only within a class", + "missing )", + /* 15 */ + "reference to non-existent subpattern", + "erroffset passed as NULL", + "unknown option bit(s) set", + "missing ) after comment", + "parentheses nested too deeply", + /* 20 */ + "regular expression too large", + "failed to get memory", + "unmatched parentheses", + "internal error: code overflow", + "unrecognized character after (?<", + /* 25 */ + "lookbehind assertion is not fixed length", + "malformed number after (?(", + "conditional group contains more than two branches", + "assertion expected after (?(", + "(?R or (?digits must be followed by )", + /* 30 */ + "unknown POSIX class name", + "POSIX collating elements are not supported", + "this version of PCRE is not compiled with PCRE_UTF8 support", + "spare error", + "character value in \\x{...} sequence is too large", + /* 35 */ + "invalid condition (?(0)", + "\\C not allowed in lookbehind assertion", + "PCRE does not support \\L, \\l, \\N, \\U, or \\u", + "number after (?C is > 255", + "closing ) for (?C expected", + /* 40 */ + "recursive call could loop indefinitely", + "unrecognized character after (?P", + "syntax error after (?P", + "two named groups have the same name", + "invalid UTF-8 string", + /* 45 */ + "support for \\P, \\p, and \\X has not been compiled", + "malformed \\P or \\p sequence", + "unknown property name after \\P or \\p" +}; + + /* Table to identify digits and hex digits. This is used when compiling patterns. Note that the tables in chartables are dependent on the locale, and may mark arbitrary characters as digits - but the PCRE compiling code expects @@ -307,579 +331,8 @@ static const unsigned char ebcdic_chartab[] = { /* chartable partial dup */ /* Definition to allow mutual recursion */ static BOOL - compile_regex(int, int, int *, uschar **, const uschar **, const char **, - BOOL, int, int *, int *, branch_chain *, compile_data *); - -/* Structure for building a chain of data that actually lives on the -stack, for holding the values of the subject pointer at the start of each -subpattern, so as to detect when an empty string has been matched by a -subpattern - to break infinite loops. When NO_RECURSE is set, these blocks -are on the heap, not on the stack. */ - -typedef struct eptrblock { - struct eptrblock *epb_prev; - const uschar *epb_saved_eptr; -} eptrblock; - -/* Flag bits for the match() function */ - -#define match_condassert 0x01 /* Called to check a condition assertion */ -#define match_isgroup 0x02 /* Set if start of bracketed group */ - -/* Non-error returns from the match() function. Error returns are externally -defined PCRE_ERROR_xxx codes, which are all negative. */ - -#define MATCH_MATCH 1 -#define MATCH_NOMATCH 0 - - - -/************************************************* -* Global variables * -*************************************************/ - -/* PCRE is thread-clean and doesn't use any global variables in the normal -sense. However, it calls memory allocation and free functions via the four -indirections below, and it can optionally do callouts. These values can be -changed by the caller, but are shared between all threads. However, when -compiling for Virtual Pascal, things are done differently (see pcre.in). */ - -#ifndef VPCOMPAT -#ifdef __cplusplus -extern "C" void *(*pcre_malloc)(size_t) = malloc; -extern "C" void (*pcre_free)(void *) = free; -extern "C" void *(*pcre_stack_malloc)(size_t) = malloc; -extern "C" void (*pcre_stack_free)(void *) = free; -extern "C" int (*pcre_callout)(pcre_callout_block *) = NULL; -#else -void *(*pcre_malloc)(size_t) = malloc; -void (*pcre_free)(void *) = free; -void *(*pcre_stack_malloc)(size_t) = malloc; -void (*pcre_stack_free)(void *) = free; -int (*pcre_callout)(pcre_callout_block *) = NULL; -#endif -#endif - - -/************************************************* -* Macros and tables for character handling * -*************************************************/ - -/* When UTF-8 encoding is being used, a character is no longer just a single -byte. The macros for character handling generate simple sequences when used in -byte-mode, and more complicated ones for UTF-8 characters. */ - -#ifndef SUPPORT_UTF8 -#define GETCHAR(c, eptr) c = *eptr; -#define GETCHARINC(c, eptr) c = *eptr++; -#define GETCHARINCTEST(c, eptr) c = *eptr++; -#define GETCHARLEN(c, eptr, len) c = *eptr; -#define BACKCHAR(eptr) - -#else /* SUPPORT_UTF8 */ - -/* Get the next UTF-8 character, not advancing the pointer. This is called when -we know we are in UTF-8 mode. */ - -#define GETCHAR(c, eptr) \ - c = *eptr; \ - if ((c & 0xc0) == 0xc0) \ - { \ - int gcii; \ - int gcaa = utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ - int gcss = 6*gcaa; \ - c = (c & utf8_table3[gcaa]) << gcss; \ - for (gcii = 1; gcii <= gcaa; gcii++) \ - { \ - gcss -= 6; \ - c |= (eptr[gcii] & 0x3f) << gcss; \ - } \ - } - -/* Get the next UTF-8 character, advancing the pointer. This is called when we -know we are in UTF-8 mode. */ - -#define GETCHARINC(c, eptr) \ - c = *eptr++; \ - if ((c & 0xc0) == 0xc0) \ - { \ - int gcaa = utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ - int gcss = 6*gcaa; \ - c = (c & utf8_table3[gcaa]) << gcss; \ - while (gcaa-- > 0) \ - { \ - gcss -= 6; \ - c |= (*eptr++ & 0x3f) << gcss; \ - } \ - } - -/* Get the next character, testing for UTF-8 mode, and advancing the pointer */ - -#define GETCHARINCTEST(c, eptr) \ - c = *eptr++; \ - if (md->utf8 && (c & 0xc0) == 0xc0) \ - { \ - int gcaa = utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ - int gcss = 6*gcaa; \ - c = (c & utf8_table3[gcaa]) << gcss; \ - while (gcaa-- > 0) \ - { \ - gcss -= 6; \ - c |= (*eptr++ & 0x3f) << gcss; \ - } \ - } - -/* Get the next UTF-8 character, not advancing the pointer, incrementing length -if there are extra bytes. This is called when we know we are in UTF-8 mode. */ - -#define GETCHARLEN(c, eptr, len) \ - c = *eptr; \ - if ((c & 0xc0) == 0xc0) \ - { \ - int gcii; \ - int gcaa = utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ - int gcss = 6*gcaa; \ - c = (c & utf8_table3[gcaa]) << gcss; \ - for (gcii = 1; gcii <= gcaa; gcii++) \ - { \ - gcss -= 6; \ - c |= (eptr[gcii] & 0x3f) << gcss; \ - } \ - len += gcaa; \ - } - -/* If the pointer is not at the start of a character, move it back until -it is. Called only in UTF-8 mode. */ - -#define BACKCHAR(eptr) while((*eptr & 0xc0) == 0x80) eptr--; - -#endif - - - -/************************************************* -* Default character tables * -*************************************************/ - -/* A default set of character tables is included in the PCRE binary. Its source -is built by the maketables auxiliary program, which uses the default C ctypes -functions, and put in the file chartables.c. These tables are used by PCRE -whenever the caller of pcre_compile() does not provide an alternate set of -tables. */ - -#include "chartables.c" - - - -#ifdef SUPPORT_UTF8 -/************************************************* -* Tables for UTF-8 support * -*************************************************/ - -/* These are the breakpoints for different numbers of bytes in a UTF-8 -character. */ - -static const int utf8_table1[] = - { 0x7f, 0x7ff, 0xffff, 0x1fffff, 0x3ffffff, 0x7fffffff}; - -/* These are the indicator bits and the mask for the data bits to set in the -first byte of a character, indexed by the number of additional bytes. */ - -static const int utf8_table2[] = { 0, 0xc0, 0xe0, 0xf0, 0xf8, 0xfc}; -static const int utf8_table3[] = { 0xff, 0x1f, 0x0f, 0x07, 0x03, 0x01}; - -/* Table of the number of extra characters, indexed by the first character -masked with 0x3f. The highest number for a valid UTF-8 character is in fact -0x3d. */ - -static const uschar utf8_table4[] = { - 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, - 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, - 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2, - 3,3,3,3,3,3,3,3,4,4,4,4,5,5,5,5 }; - - -/************************************************* -* Convert character value to UTF-8 * -*************************************************/ - -/* This function takes an integer value in the range 0 - 0x7fffffff -and encodes it as a UTF-8 character in 0 to 6 bytes. - -Arguments: - cvalue the character value - buffer pointer to buffer for result - at least 6 bytes long - -Returns: number of characters placed in the buffer -*/ - -static int -ord2utf8(int cvalue, uschar *buffer) -{ -register int i, j; -for (i = 0; i < sizeof(utf8_table1)/sizeof(int); i++) - if (cvalue <= utf8_table1[i]) break; -buffer += i; -for (j = i; j > 0; j--) - { - *buffer-- = 0x80 | (cvalue & 0x3f); - cvalue >>= 6; - } -*buffer = utf8_table2[i] | cvalue; -return i + 1; -} -#endif - - - -/************************************************* -* Print compiled regex * -*************************************************/ - -/* The code for doing this is held in a separate file that is also included in -pcretest.c. It defines a function called print_internals(). */ - -#ifdef DEBUG -#include "printint.c" -#endif - - - -/************************************************* -* Return version string * -*************************************************/ - -#define STRING(a) # a -#define XSTRING(s) STRING(s) - -EXPORT const char * -pcre_version(void) -{ -return XSTRING(PCRE_MAJOR) "." XSTRING(PCRE_MINOR) " " XSTRING(PCRE_DATE); -} - - - - -/************************************************* -* Flip bytes in an integer * -*************************************************/ - -/* This function is called when the magic number in a regex doesn't match in -order to flip its bytes to see if we are dealing with a pattern that was -compiled on a host of different endianness. If so, this function is used to -flip other byte values. - -Arguments: - value the number to flip - n the number of bytes to flip (assumed to be 2 or 4) - -Returns: the flipped value -*/ - -static long int -byteflip(long int value, int n) -{ -if (n == 2) return ((value & 0x00ff) << 8) | ((value & 0xff00) >> 8); -return ((value & 0x000000ff) << 24) | - ((value & 0x0000ff00) << 8) | - ((value & 0x00ff0000) >> 8) | - ((value & 0xff000000) >> 24); -} - - - -/************************************************* -* Test for a byte-flipped compiled regex * -*************************************************/ - -/* This function is called from pce_exec() and also from pcre_fullinfo(). Its -job is to test whether the regex is byte-flipped - that is, it was compiled on -a system of opposite endianness. The function is called only when the native -MAGIC_NUMBER test fails. If the regex is indeed flipped, we flip all the -relevant values into a different data block, and return it. - -Arguments: - re points to the regex - study points to study data, or NULL - internal_re points to a new regex block - internal_study points to a new study block - -Returns: the new block if is is indeed a byte-flipped regex - NULL if it is not -*/ - -static real_pcre * -try_flipped(const real_pcre *re, real_pcre *internal_re, - const pcre_study_data *study, pcre_study_data *internal_study) -{ -if (byteflip(re->magic_number, sizeof(re->magic_number)) != MAGIC_NUMBER) - return NULL; - -*internal_re = *re; /* To copy other fields */ -internal_re->size = byteflip(re->size, sizeof(re->size)); -internal_re->options = byteflip(re->options, sizeof(re->options)); -internal_re->top_bracket = byteflip(re->top_bracket, sizeof(re->top_bracket)); -internal_re->top_backref = byteflip(re->top_backref, sizeof(re->top_backref)); -internal_re->first_byte = byteflip(re->first_byte, sizeof(re->first_byte)); -internal_re->req_byte = byteflip(re->req_byte, sizeof(re->req_byte)); -internal_re->name_table_offset = byteflip(re->name_table_offset, - sizeof(re->name_table_offset)); -internal_re->name_entry_size = byteflip(re->name_entry_size, - sizeof(re->name_entry_size)); -internal_re->name_count = byteflip(re->name_count, sizeof(re->name_count)); - -if (study != NULL) - { - *internal_study = *study; /* To copy other fields */ - internal_study->size = byteflip(study->size, sizeof(study->size)); - internal_study->options = byteflip(study->options, sizeof(study->options)); - } - -return internal_re; -} - - - -/************************************************* -* (Obsolete) Return info about compiled pattern * -*************************************************/ - -/* This is the original "info" function. It picks potentially useful data out -of the private structure, but its interface was too rigid. It remains for -backwards compatibility. The public options are passed back in an int - though -the re->options field has been expanded to a long int, all the public options -at the low end of it, and so even on 16-bit systems this will still be OK. -Therefore, I haven't changed the API for pcre_info(). - -Arguments: - argument_re points to compiled code - optptr where to pass back the options - first_byte where to pass back the first character, - or -1 if multiline and all branches start ^, - or -2 otherwise - -Returns: number of capturing subpatterns - or negative values on error -*/ - -EXPORT int -pcre_info(const pcre *argument_re, int *optptr, int *first_byte) -{ -real_pcre internal_re; -const real_pcre *re = (const real_pcre *)argument_re; -if (re == NULL) return PCRE_ERROR_NULL; -if (re->magic_number != MAGIC_NUMBER) - { - re = try_flipped(re, &internal_re, NULL, NULL); - if (re == NULL) return PCRE_ERROR_BADMAGIC; - } -if (optptr != NULL) *optptr = (int)(re->options & PUBLIC_OPTIONS); -if (first_byte != NULL) - *first_byte = ((re->options & PCRE_FIRSTSET) != 0)? re->first_byte : - ((re->options & PCRE_STARTLINE) != 0)? -1 : -2; -return re->top_bracket; -} - - - -/************************************************* -* Return info about compiled pattern * -*************************************************/ - -/* This is a newer "info" function which has an extensible interface so -that additional items can be added compatibly. - -Arguments: - argument_re points to compiled code - extra_data points extra data, or NULL - what what information is required - where where to put the information - -Returns: 0 if data returned, negative on error -*/ - -EXPORT int -pcre_fullinfo(const pcre *argument_re, const pcre_extra *extra_data, int what, - void *where) -{ -real_pcre internal_re; -pcre_study_data internal_study; -const real_pcre *re = (const real_pcre *)argument_re; -const pcre_study_data *study = NULL; - -if (re == NULL || where == NULL) return PCRE_ERROR_NULL; - -if (extra_data != NULL && (extra_data->flags & PCRE_EXTRA_STUDY_DATA) != 0) - study = (const pcre_study_data *)extra_data->study_data; - -if (re->magic_number != MAGIC_NUMBER) - { - re = try_flipped(re, &internal_re, study, &internal_study); - if (re == NULL) return PCRE_ERROR_BADMAGIC; - if (study != NULL) study = &internal_study; - } - -switch (what) - { - case PCRE_INFO_OPTIONS: - *((unsigned long int *)where) = re->options & PUBLIC_OPTIONS; - break; - - case PCRE_INFO_SIZE: - *((size_t *)where) = re->size; - break; - - case PCRE_INFO_STUDYSIZE: - *((size_t *)where) = (study == NULL)? 0 : study->size; - break; - - case PCRE_INFO_CAPTURECOUNT: - *((int *)where) = re->top_bracket; - break; - - case PCRE_INFO_BACKREFMAX: - *((int *)where) = re->top_backref; - break; - - case PCRE_INFO_FIRSTBYTE: - *((int *)where) = - ((re->options & PCRE_FIRSTSET) != 0)? re->first_byte : - ((re->options & PCRE_STARTLINE) != 0)? -1 : -2; - break; - - /* Make sure we pass back the pointer to the bit vector in the external - block, not the internal copy (with flipped integer fields). */ - - case PCRE_INFO_FIRSTTABLE: - *((const uschar **)where) = - (study != NULL && (study->options & PCRE_STUDY_MAPPED) != 0)? - ((const pcre_study_data *)extra_data->study_data)->start_bits : NULL; - break; - - case PCRE_INFO_LASTLITERAL: - *((int *)where) = - ((re->options & PCRE_REQCHSET) != 0)? re->req_byte : -1; - break; - - case PCRE_INFO_NAMEENTRYSIZE: - *((int *)where) = re->name_entry_size; - break; - - case PCRE_INFO_NAMECOUNT: - *((int *)where) = re->name_count; - break; - - case PCRE_INFO_NAMETABLE: - *((const uschar **)where) = (const uschar *)re + re->name_table_offset; - break; - - case PCRE_INFO_DEFAULT_TABLES: - *((const uschar **)where) = (const uschar *)pcre_default_tables; - break; - - default: return PCRE_ERROR_BADOPTION; - } - -return 0; -} - - - -/************************************************* -* Return info about what features are configured * -*************************************************/ - -/* This is function which has an extensible interface so that additional items -can be added compatibly. - -Arguments: - what what information is required - where where to put the information - -Returns: 0 if data returned, negative on error -*/ - -EXPORT int -pcre_config(int what, void *where) -{ -switch (what) - { - case PCRE_CONFIG_UTF8: -#ifdef SUPPORT_UTF8 - *((int *)where) = 1; -#else - *((int *)where) = 0; -#endif - break; - - case PCRE_CONFIG_UNICODE_PROPERTIES: -#ifdef SUPPORT_UCP - *((int *)where) = 1; -#else - *((int *)where) = 0; -#endif - break; - - case PCRE_CONFIG_NEWLINE: - *((int *)where) = NEWLINE; - break; - - case PCRE_CONFIG_LINK_SIZE: - *((int *)where) = LINK_SIZE; - break; - - case PCRE_CONFIG_POSIX_MALLOC_THRESHOLD: - *((int *)where) = POSIX_MALLOC_THRESHOLD; - break; - - case PCRE_CONFIG_MATCH_LIMIT: - *((unsigned int *)where) = MATCH_LIMIT; - break; - - case PCRE_CONFIG_STACKRECURSE: -#ifdef NO_RECURSE - *((int *)where) = 0; -#else - *((int *)where) = 1; -#endif - break; - - default: return PCRE_ERROR_BADOPTION; - } - -return 0; -} - - - -#ifdef DEBUG -/************************************************* -* Debugging function to print chars * -*************************************************/ - -/* Print a sequence of chars in printable format, stopping at the end of the -subject if the requested. - -Arguments: - p points to characters - length number to print - is_subject TRUE if printing from within md->start_subject - md pointer to matching data block, if is_subject is TRUE - -Returns: nothing -*/ - -static void -pchars(const uschar *p, int length, BOOL is_subject, match_data *md) -{ -int c; -if (is_subject && length > md->end_subject - p) length = md->end_subject - p; -while (length-- > 0) - if (isprint(c = *(p++))) printf("%c", c); else printf("\\x%02x", c); -} -#endif - + compile_regex(int, int, int *, uschar **, const uschar **, int *, BOOL, int, + int *, int *, branch_chain *, compile_data *); @@ -894,19 +347,19 @@ a positive value greater than 255 may be returned. On entry, ptr is pointing at the \. On exit, it is on the final character of the escape sequence. Arguments: - ptrptr points to the pattern position pointer - errorptr points to the pointer to the error message - bracount number of previous extracting brackets - options the options bits - isclass TRUE if inside a character class - -Returns: zero or positive => a data character - negative => a special escape sequence - on error, errorptr is set + ptrptr points to the pattern position pointer + errorcodeptr points to the errorcode variable + bracount number of previous extracting brackets + options the options bits + isclass TRUE if inside a character class + +Returns: zero or positive => a data character + negative => a special escape sequence + on error, errorptr is set */ static int -check_escape(const uschar **ptrptr, const char **errorptr, int bracount, +check_escape(const uschar **ptrptr, int *errorcodeptr, int bracount, int options, BOOL isclass) { const uschar *ptr = *ptrptr; @@ -915,7 +368,7 @@ int c, i; /* If backslash is at the end of the pattern, it's an error. */ c = *(++ptr); -if (c == 0) *errorptr = ERR1; +if (c == 0) *errorcodeptr = ERR1; /* Non-alphamerics are literals. For digits or letters, do an initial lookup in a table. A non-zero result is something that can be returned immediately. @@ -945,7 +398,7 @@ else case 'N': case 'u': case 'U': - *errorptr = ERR37; + *errorcodeptr = ERR37; break; /* The handling of escape sequences consisting of a string of digits @@ -1022,7 +475,7 @@ else } if (*pt == '}') { - if (c < 0 || count > 8) *errorptr = ERR34; + if (c < 0 || count > 8) *errorcodeptr = ERR34; ptr = pt; break; } @@ -1054,7 +507,7 @@ else c = *(++ptr); if (c == 0) { - *errorptr = ERR2; + *errorcodeptr = ERR2; return 0; } @@ -1081,7 +534,7 @@ else if ((options & PCRE_EXTRA) != 0) switch(c) { default: - *errorptr = ERR3; + *errorcodeptr = ERR3; break; } break; @@ -1105,15 +558,15 @@ pointing at the P or p. On exit, it is pointing at the final character of the escape sequence. Argument: - ptrptr points to the pattern position pointer - negptr points to a boolean that is set TRUE for negation else FALSE - errorptr points to the pointer to the error message + ptrptr points to the pattern position pointer + negptr points to a boolean that is set TRUE for negation else FALSE + errorcodeptr points to the error code variable Returns: value from ucp_type_table, or -1 for an invalid type */ static int -get_ucp(const uschar **ptrptr, BOOL *negptr, const char **errorptr) +get_ucp(const uschar **ptrptr, BOOL *negptr, int *errorcodeptr) { int c, i, bot, top; const uschar *ptr = *ptrptr; @@ -1162,23 +615,23 @@ else /* Search for a recognized property name using binary chop */ bot = 0; -top = sizeof(utt)/sizeof(ucp_type_table); +top = _pcre_utt_size; while (bot < top) { i = (bot + top)/2; - c = strcmp(name, utt[i].name); - if (c == 0) return utt[i].value; + c = strcmp(name, _pcre_utt[i].name); + if (c == 0) return _pcre_utt[i].value; if (c > 0) bot = i + 1; else top = i; } UNKNOWN_RETURN: -*errorptr = ERR47; +*errorcodeptr = ERR47; *ptrptr = ptr; return -1; ERROR_RETURN: -*errorptr = ERR46; +*errorcodeptr = ERR46; *ptrptr = ptr; return -1; } @@ -1229,18 +682,18 @@ after is_counted_repeat() has confirmed that a repeat-count quantifier exists, so the syntax is guaranteed to be correct, but we need to check the values. Arguments: - p pointer to first char after '{' - minp pointer to int for min - maxp pointer to int for max - returned as -1 if no max - errorptr points to pointer to error message - -Returns: pointer to '}' on success; - current ptr on error, with errorptr set + p pointer to first char after '{' + minp pointer to int for min + maxp pointer to int for max + returned as -1 if no max + errorcodeptr points to error code variable + +Returns: pointer to '}' on success; + current ptr on error, with errorcodeptr set non-zero */ static const uschar * -read_repeat_counts(const uschar *p, int *minp, int *maxp, const char **errorptr) +read_repeat_counts(const uschar *p, int *minp, int *maxp, int *errorcodeptr) { int min = 0; int max = -1; @@ -1255,7 +708,7 @@ if (*p == '}') max = min; else while((digitab[*p] & ctype_digit) != 0) max = max * 10 + *p++ - '0'; if (max < min) { - *errorptr = ERR4; + *errorcodeptr = ERR4; return p; } } @@ -1265,7 +718,7 @@ if (*p == '}') max = min; else pointer to the terminating '}'. */ if (min > 65535 || max > 65535) - *errorptr = ERR5; + *errorcodeptr = ERR5; else { *minp = min; @@ -1315,7 +768,7 @@ for (;;) case OP_ASSERTBACK_NOT: if (!skipassert) return code; do code += GET(code, 1); while (*code == OP_ALT); - code += OP_lengths[*code]; + code += _pcre_OP_lengths[*code]; break; case OP_WORD_BOUNDARY: @@ -1326,7 +779,7 @@ for (;;) case OP_CALLOUT: case OP_CREF: case OP_BRANUMBER: - code += OP_lengths[*code]; + code += _pcre_OP_lengths[*code]; break; default: @@ -1424,7 +877,7 @@ for (;;) case OP_DOLL: case OP_NOT_WORD_BOUNDARY: case OP_WORD_BOUNDARY: - cc += OP_lengths[*cc]; + cc += _pcre_OP_lengths[*cc]; break; /* Handle literal characters */ @@ -1558,11 +1011,11 @@ for (;;) int n = c - OP_BRA; if (n > EXTRACT_BASIC_MAX) n = GET2(code, 2+LINK_SIZE); if (n == number) return (uschar *)code; - code += OP_lengths[OP_BRA]; + code += _pcre_OP_lengths[OP_BRA]; } else { - code += OP_lengths[c]; + code += _pcre_OP_lengths[c]; #ifdef SUPPORT_UTF8 @@ -1630,11 +1083,11 @@ for (;;) else if (c == OP_RECURSE) return code; else if (c > OP_BRA) { - code += OP_lengths[OP_BRA]; + code += _pcre_OP_lengths[OP_BRA]; } else { - code += OP_lengths[c]; + code += _pcre_OP_lengths[c]; #ifdef SUPPORT_UTF8 @@ -1698,7 +1151,7 @@ could_be_empty_branch(const uschar *code, const uschar *endcode, BOOL utf8) register int c; for (code = first_significant_code(code + 1 + LINK_SIZE, NULL, 0, TRUE); code < endcode; - code = first_significant_code(code + OP_lengths[c], NULL, 0, TRUE)) + code = first_significant_code(code + _pcre_OP_lengths[c], NULL, 0, TRUE)) { const uschar *ccode; @@ -2028,7 +1481,8 @@ int c, chartype, othercase, next; for (c = *cptr; c <= d; c++) { - if (ucp_findchar(c, &chartype, &othercase) == ucp_L && othercase != 0) break; + if (_pcre_ucp_findchar(c, &chartype, &othercase) == ucp_L && othercase != 0) + break; } if (c > d) return FALSE; @@ -2038,7 +1492,8 @@ next = othercase + 1; for (++c; c <= d; c++) { - if (ucp_findchar(c, &chartype, &othercase) != ucp_L || othercase != next) + if (_pcre_ucp_findchar(c, &chartype, &othercase) != ucp_L || + othercase != next) break; next++; } @@ -2064,19 +1519,19 @@ Arguments: brackets points to number of extracting brackets used codeptr points to the pointer to the current code point ptrptr points to the current pattern pointer - errorptr points to pointer to error message + errorcodeptr points to error code variable firstbyteptr set to initial literal character, or < 0 (REQ_UNSET, REQ_NONE) reqbyteptr set to the last literal character required, else < 0 bcptr points to current branch chain cd contains pointers to tables etc. Returns: TRUE on success - FALSE, with *errorptr set on error + FALSE, with *errorcodeptr set non-zero on error */ static BOOL compile_branch(int *optionsptr, int *brackets, uschar **codeptr, - const uschar **ptrptr, const char **errorptr, int *firstbyteptr, + const uschar **ptrptr, int *errorcodeptr, int *firstbyteptr, int *reqbyteptr, branch_chain *bcptr, compile_data *cd) { int repeat_type, op_type; @@ -2277,7 +1732,7 @@ for (;; ptr++) if ((ptr[1] == ':' || ptr[1] == '.' || ptr[1] == '=') && check_posix_syntax(ptr, &tempptr, cd)) { - *errorptr = (ptr[1] == ':')? ERR13 : ERR31; + *errorcodeptr = (ptr[1] == ':')? ERR13 : ERR31; goto FAILED; } @@ -2356,7 +1811,7 @@ for (;; ptr++) if (ptr[1] != ':') { - *errorptr = ERR31; + *errorcodeptr = ERR31; goto FAILED; } @@ -2370,7 +1825,7 @@ for (;; ptr++) posix_class = check_posix_name(ptr, tempptr - ptr); if (posix_class < 0) { - *errorptr = ERR30; + *errorcodeptr = ERR30; goto FAILED; } @@ -2422,7 +1877,7 @@ for (;; ptr++) if (c == '\\') { - c = check_escape(&ptr, errorptr, *brackets, options, TRUE); + c = check_escape(&ptr, errorcodeptr, *brackets, options, TRUE); if (-c == ESC_b) c = '\b'; /* \b is backslash in a class */ else if (-c == ESC_X) c = 'X'; /* \X is literal X in a class */ @@ -2473,7 +1928,7 @@ for (;; ptr++) case ESC_P: { BOOL negated; - int property = get_ucp(&ptr, &negated, errorptr); + int property = get_ucp(&ptr, &negated, errorcodeptr); if (property < 0) goto FAILED; class_utf8 = TRUE; *class_utf8data++ = ((-c == ESC_p) != negated)? @@ -2491,7 +1946,7 @@ for (;; ptr++) default: if ((options & PCRE_EXTRA) != 0) { - *errorptr = ERR7; + *errorcodeptr = ERR7; goto FAILED; } c = *ptr; /* The final character */ @@ -2529,7 +1984,7 @@ for (;; ptr++) if (d == '\\') { const uschar *oldptr = ptr; - d = check_escape(&ptr, errorptr, *brackets, options, TRUE); + d = check_escape(&ptr, errorcodeptr, *brackets, options, TRUE); /* \b is backslash; \X is literal X; any other special means the '-' was literal */ @@ -2592,9 +2047,9 @@ for (;; ptr++) else { *class_utf8data++ = XCL_RANGE; - class_utf8data += ord2utf8(occ, class_utf8data); + class_utf8data += _pcre_ord2utf8(occ, class_utf8data); } - class_utf8data += ord2utf8(ocd, class_utf8data); + class_utf8data += _pcre_ord2utf8(ocd, class_utf8data); } } #endif /* SUPPORT_UCP */ @@ -2603,8 +2058,8 @@ for (;; ptr++) overlapping ranges. */ *class_utf8data++ = XCL_RANGE; - class_utf8data += ord2utf8(c, class_utf8data); - class_utf8data += ord2utf8(d, class_utf8data); + class_utf8data += _pcre_ord2utf8(c, class_utf8data); + class_utf8data += _pcre_ord2utf8(d, class_utf8data); /* With UCP support, we are done. Without UCP support, there is no caseless matching for UTF-8 characters > 127; we can use the bit map @@ -2655,17 +2110,18 @@ for (;; ptr++) { class_utf8 = TRUE; *class_utf8data++ = XCL_SINGLE; - class_utf8data += ord2utf8(c, class_utf8data); + class_utf8data += _pcre_ord2utf8(c, class_utf8data); #ifdef SUPPORT_UCP if ((options & PCRE_CASELESS) != 0) { int chartype; int othercase; - if (ucp_findchar(c, &chartype, &othercase) >= 0 && othercase > 0) + if (_pcre_ucp_findchar(c, &chartype, &othercase) >= 0 && + othercase > 0) { *class_utf8data++ = XCL_SINGLE; - class_utf8data += ord2utf8(othercase, class_utf8data); + class_utf8data += _pcre_ord2utf8(othercase, class_utf8data); } } #endif /* SUPPORT_UCP */ @@ -2733,7 +2189,7 @@ for (;; ptr++) #ifdef SUPPORT_UTF8 if (utf8 && class_lastchar > 127) - mclength = ord2utf8(class_lastchar, mcbuffer); + mclength = _pcre_ord2utf8(class_lastchar, mcbuffer); else #endif { @@ -2813,8 +2269,8 @@ for (;; ptr++) case '{': if (!is_quantifier) goto NORMAL_CHAR; - ptr = read_repeat_counts(ptr+1, &repeat_min, &repeat_max, errorptr); - if (*errorptr != NULL) goto FAILED; + ptr = read_repeat_counts(ptr+1, &repeat_min, &repeat_max, errorcodeptr); + if (*errorcodeptr != 0) goto FAILED; goto REPEAT; case '*': @@ -2834,7 +2290,7 @@ for (;; ptr++) REPEAT: if (previous == NULL) { - *errorptr = ERR9; + *errorcodeptr = ERR9; goto FAILED; } @@ -3273,7 +2729,7 @@ for (;; ptr++) else { - *errorptr = ERR11; + *errorcodeptr = ERR11; goto FAILED; } @@ -3357,7 +2813,7 @@ for (;; ptr++) while (*(++ptr) != ')') condref = condref*10 + *ptr - '0'; if (condref == 0) { - *errorptr = ERR35; + *errorcodeptr = ERR35; goto FAILED; } ptr++; @@ -3409,7 +2865,7 @@ for (;; ptr++) n = n * 10 + *ptr - '0'; if (n > 255) { - *errorptr = ERR38; + *errorcodeptr = ERR38; goto FAILED; } *code++ = n; @@ -3438,7 +2894,7 @@ for (;; ptr++) { if (slot[2+namelen] == 0) { - *errorptr = ERR43; + *errorcodeptr = ERR43; goto FAILED; } crc = -1; /* Current name is substring */ @@ -3476,7 +2932,7 @@ for (;; ptr++) } if (i >= cd->names_found) { - *errorptr = ERR15; + *errorcodeptr = ERR15; goto FAILED; } @@ -3526,7 +2982,7 @@ for (;; ptr++) if (called == NULL) { - *errorptr = ERR15; + *errorcodeptr = ERR15; goto FAILED; } @@ -3536,7 +2992,7 @@ for (;; ptr++) if (GET(called, 1) == 0 && could_be_empty(called, code, bcptr, utf8)) { - *errorptr = ERR40; + *errorcodeptr = ERR40; goto FAILED; } @@ -3656,7 +3112,7 @@ for (;; ptr++) brackets, /* Extracting bracket count */ &tempcode, /* Where to put code (updated) */ &ptr, /* Input pointer (updated) */ - errorptr, /* Where to put an error message */ + errorcodeptr, /* Where to put an error message */ (bravalue == OP_ASSERTBACK || bravalue == OP_ASSERTBACK_NOT), /* TRUE if back assert */ skipbytes, /* Skip over OP_COND/OP_BRANUMBER */ @@ -3687,7 +3143,7 @@ for (;; ptr++) if (condcount > 2) { - *errorptr = ERR27; + *errorcodeptr = ERR27; goto FAILED; } @@ -3757,7 +3213,7 @@ for (;; ptr++) if (*ptr != ')') { - *errorptr = ERR14; + *errorcodeptr = ERR14; goto FAILED; } break; @@ -3768,7 +3224,7 @@ for (;; ptr++) case '\\': tempptr = ptr; - c = check_escape(&ptr, errorptr, *brackets, options, FALSE); + c = check_escape(&ptr, errorcodeptr, *brackets, options, FALSE); /* Handle metacharacters introduced by \. For ones like \d, the ESC_ values are arranged to be the negation of the corresponding OP_values. For the @@ -3814,7 +3270,7 @@ for (;; ptr++) else if (-c == ESC_P || -c == ESC_p) { BOOL negated; - int value = get_ucp(&ptr, &negated, errorptr); + int value = get_ucp(&ptr, &negated, errorcodeptr); previous = code; *code++ = ((-c == ESC_p) != negated)? OP_PROP : OP_NOTPROP; *code++ = value; @@ -3838,7 +3294,7 @@ for (;; ptr++) #ifdef SUPPORT_UTF8 if (utf8 && c > 127) - mclength = ord2utf8(c, mcbuffer); + mclength = _pcre_ord2utf8(c, mcbuffer); else #endif @@ -3940,7 +3396,7 @@ Argument: brackets -> int containing the number of extracting brackets used codeptr -> the address of the current code pointer ptrptr -> the address of the current pattern pointer - errorptr -> pointer to error message + errorcodeptr -> pointer to error code variable lookbehind TRUE if this is a lookbehind assertion skipbytes skip this many bytes at start (for OP_COND, OP_BRANUMBER) firstbyteptr place to put the first required character, or a negative number @@ -3953,7 +3409,7 @@ Returns: TRUE on success static BOOL compile_regex(int options, int oldims, int *brackets, uschar **codeptr, - const uschar **ptrptr, const char **errorptr, BOOL lookbehind, int skipbytes, + const uschar **ptrptr, int *errorcodeptr, BOOL lookbehind, int skipbytes, int *firstbyteptr, int *reqbyteptr, branch_chain *bcptr, compile_data *cd) { const uschar *ptr = *ptrptr; @@ -3998,7 +3454,7 @@ for (;;) /* Now compile the branch */ - if (!compile_branch(&options, brackets, &code, &ptr, errorptr, + if (!compile_branch(&options, brackets, &code, &ptr, errorcodeptr, &branchfirstbyte, &branchreqbyte, &bc, cd)) { *ptrptr = ptr; @@ -4056,7 +3512,7 @@ for (;;) DPRINTF(("fixed length = %d\n", length)); if (length < 0) { - *errorptr = (length == -2)? ERR36 : ERR25; + *errorcodeptr = (length == -2)? ERR36 : ERR25; *ptrptr = ptr; return FALSE; } @@ -4350,118 +3806,42 @@ return c; - -#ifdef SUPPORT_UTF8 -/************************************************* -* Validate a UTF-8 string * -*************************************************/ - -/* This function is called (optionally) at the start of compile or match, to -validate that a supposed UTF-8 string is actually valid. The early check means -that subsequent code can assume it is dealing with a valid string. The check -can be turned off for maximum performance, but then consequences of supplying -an invalid string are then undefined. - -Arguments: - string points to the string - length length of string, or -1 if the string is zero-terminated - -Returns: < 0 if the string is a valid UTF-8 string - >= 0 otherwise; the value is the offset of the bad byte -*/ - -static int -valid_utf8(const uschar *string, int length) -{ -register const uschar *p; - -if (length < 0) - { - for (p = string; *p != 0; p++); - length = p - string; - } - -for (p = string; length-- > 0; p++) - { - register int ab; - register int c = *p; - if (c < 128) continue; - if ((c & 0xc0) != 0xc0) return p - string; - ab = utf8_table4[c & 0x3f]; /* Number of additional bytes */ - if (length < ab) return p - string; - length -= ab; - - /* Check top bits in the second byte */ - if ((*(++p) & 0xc0) != 0x80) return p - string; - - /* Check for overlong sequences for each different length */ - switch (ab) - { - /* Check for xx00 000x */ - case 1: - if ((c & 0x3e) == 0) return p - string; - continue; /* We know there aren't any more bytes to check */ - - /* Check for 1110 0000, xx0x xxxx */ - case 2: - if (c == 0xe0 && (*p & 0x20) == 0) return p - string; - break; - - /* Check for 1111 0000, xx00 xxxx */ - case 3: - if (c == 0xf0 && (*p & 0x30) == 0) return p - string; - break; - - /* Check for 1111 1000, xx00 0xxx */ - case 4: - if (c == 0xf8 && (*p & 0x38) == 0) return p - string; - break; - - /* Check for leading 0xfe or 0xff, and then for 1111 1100, xx00 00xx */ - case 5: - if (c == 0xfe || c == 0xff || - (c == 0xfc && (*p & 0x3c) == 0)) return p - string; - break; - } - - /* Check for valid bytes after the 2nd, if any; all must start 10 */ - while (--ab > 0) - { - if ((*(++p) & 0xc0) != 0x80) return p - string; - } - } - -return -1; -} -#endif - - - /************************************************* * Compile a Regular Expression * *************************************************/ /* This function takes a string and returns a pointer to a block of store -holding a compiled version of the expression. +holding a compiled version of the expression. The original API for this +function had no error code return variable; it is retained for backwards +compatibility. The new function is given a new name. Arguments: - pattern the regular expression - options various option bits - errorptr pointer to pointer to error text - erroroffset ptr offset in pattern where error was detected - tables pointer to character tables or NULL - -Returns: pointer to compiled data block, or NULL on error, - with errorptr and erroroffset set + pattern the regular expression + options various option bits + errorcodeptr pointer to error code variable (pcre_compile2() only) + can be NULL if you don't want a code value + errorptr pointer to pointer to error text + erroroffset ptr offset in pattern where error was detected + tables pointer to character tables or NULL + +Returns: pointer to compiled data block, or NULL on error, + with errorptr and erroroffset set */ EXPORT pcre * pcre_compile(const char *pattern, int options, const char **errorptr, int *erroroffset, const unsigned char *tables) { +return pcre_compile2(pattern, options, NULL, errorptr, erroroffset, tables); +} + + +EXPORT pcre * +pcre_compile2(const char *pattern, int options, int *errorcodeptr, + const char **errorptr, int *erroroffset, const unsigned char *tables) +{ real_pcre *re; int length = 1 + LINK_SIZE; /* For initial BRA plus length */ -int runlength; int c, firstbyte, reqbyte; int bracount = 0; int branch_extra = 0; @@ -4470,6 +3850,7 @@ int item_count = -1; int name_count = 0; int max_name_size = 0; int lastitemlength = 0; +int errorcode = 0; #ifdef SUPPORT_UTF8 BOOL utf8; BOOL class_utf8; @@ -4485,18 +3866,26 @@ int brastack[BRASTACK_SIZE]; uschar bralenstack[BRASTACK_SIZE]; /* We can't pass back an error message if errorptr is NULL; I guess the best we -can do is just return NULL. */ +can do is just return NULL, but we can set a code value if there is a code +pointer. */ + +if (errorptr == NULL) + { + if (errorcodeptr != NULL) *errorcodeptr = 99; + return NULL; + } -if (errorptr == NULL) return NULL; *errorptr = NULL; +if (errorcodeptr != NULL) *errorcodeptr = ERR0; /* However, we can give a message for this error */ if (erroroffset == NULL) { - *errorptr = ERR16; - return NULL; + errorcode = ERR16; + goto PCRE_EARLY_ERROR_RETURN; } + *erroroffset = 0; /* Can't support UTF8 unless PCRE has been compiled to include the code. */ @@ -4504,28 +3893,28 @@ if (erroroffset == NULL) #ifdef SUPPORT_UTF8 utf8 = (options & PCRE_UTF8) != 0; if (utf8 && (options & PCRE_NO_UTF8_CHECK) == 0 && - (*erroroffset = valid_utf8((uschar *)pattern, -1)) >= 0) + (*erroroffset = _pcre_valid_utf8((uschar *)pattern, -1)) >= 0) { - *errorptr = ERR44; - return NULL; + errorcode = ERR44; + goto PCRE_EARLY_ERROR_RETURN; } #else if ((options & PCRE_UTF8) != 0) { - *errorptr = ERR32; - return NULL; + errorcode = ERR32; + goto PCRE_EARLY_ERROR_RETURN; } #endif if ((options & ~PUBLIC_OPTIONS) != 0) { - *errorptr = ERR17; - return NULL; + errorcode = ERR17; + goto PCRE_EARLY_ERROR_RETURN; } /* Set up pointers to the individual character tables */ -if (tables == NULL) tables = pcre_default_tables; +if (tables == NULL) tables = _pcre_default_tables; compile_block.lcc = tables + lcc_offset; compile_block.fcc = tables + fcc_offset; compile_block.cbits = tables + cbits_offset; @@ -4597,8 +3986,8 @@ while ((c = *(++ptr)) != 0) character type. */ case '\\': - c = check_escape(&ptr, errorptr, bracount, options, FALSE); - if (*errorptr != NULL) goto PCRE_ERROR_RETURN; + c = check_escape(&ptr, &errorcode, bracount, options, FALSE); + if (errorcode != 0) goto PCRE_ERROR_RETURN; lastitemlength = 1; /* Default length of last item for repeats */ @@ -4610,8 +3999,8 @@ while ((c = *(++ptr)) != 0) if (utf8 && c > 127) { int i; - for (i = 0; i < sizeof(utf8_table1)/sizeof(int); i++) - if (c <= utf8_table1[i]) break; + for (i = 0; i < _pcre_utf8_table1_size; i++) + if (c <= _pcre_utf8_table1[i]) break; length += i; lastitemlength += i; } @@ -4633,7 +4022,7 @@ while ((c = *(++ptr)) != 0) #ifndef SUPPORT_UCP if (-c == ESC_X) { - *errorptr = ERR45; + errorcode = ERR45; goto PCRE_ERROR_RETURN; } #endif @@ -4647,10 +4036,10 @@ while ((c = *(++ptr)) != 0) BOOL negated; length += 2; lastitemlength = 2; - if (get_ucp(&ptr, &negated, errorptr) < 0) goto PCRE_ERROR_RETURN; + if (get_ucp(&ptr, &negated, &errorcode) < 0) goto PCRE_ERROR_RETURN; continue; #else - *errorptr = ERR45; + errorcode = ERR45; goto PCRE_ERROR_RETURN; #endif } @@ -4672,8 +4061,8 @@ while ((c = *(++ptr)) != 0) length += 2; /* For single back reference */ if (ptr[1] == '{' && is_counted_repeat(ptr+2)) { - ptr = read_repeat_counts(ptr+2, &min, &max, errorptr); - if (*errorptr != NULL) goto PCRE_ERROR_RETURN; + ptr = read_repeat_counts(ptr+2, &min, &max, &errorcode); + if (errorcode != 0) goto PCRE_ERROR_RETURN; if ((min == 0 && (max == 1 || max == -1)) || (min == 1 && max == -1)) length++; @@ -4701,8 +4090,8 @@ while ((c = *(++ptr)) != 0) case '{': if (!is_counted_repeat(ptr+1)) goto NORMAL_CHAR; - ptr = read_repeat_counts(ptr+1, &min, &max, errorptr); - if (*errorptr != NULL) goto PCRE_ERROR_RETURN; + ptr = read_repeat_counts(ptr+1, &min, &max, &errorcode); + if (errorcode != 0) goto PCRE_ERROR_RETURN; /* These special cases just insert one extra opcode */ @@ -4779,8 +4168,8 @@ while ((c = *(++ptr)) != 0) if (*ptr == '\\') { - c = check_escape(&ptr, errorptr, bracount, options, TRUE); - if (*errorptr != NULL) goto PCRE_ERROR_RETURN; + c = check_escape(&ptr, &errorcode, bracount, options, TRUE); + if (errorcode != 0) goto PCRE_ERROR_RETURN; /* \b is backspace inside a class; \X is literal */ @@ -4864,8 +4253,8 @@ while ((c = *(++ptr)) != 0) if (ptr[1] == '\\') { ptr++; - d = check_escape(&ptr, errorptr, bracount, options, TRUE); - if (*errorptr != NULL) goto PCRE_ERROR_RETURN; + d = check_escape(&ptr, &errorcode, bracount, options, TRUE); + if (errorcode != 0) goto PCRE_ERROR_RETURN; if (-d == ESC_b) d = '\b'; /* backspace */ else if (-d == ESC_X) d = 'X'; /* literal X in a class */ } @@ -4894,7 +4283,7 @@ while ((c = *(++ptr)) != 0) class_optcount = 10; /* Ensure > 1 */ if (d < c) { - *errorptr = ERR8; + errorcode = ERR8; goto PCRE_ERROR_RETURN; } @@ -4937,15 +4326,15 @@ while ((c = *(++ptr)) != 0) /* An extra item is needed */ - length += 1 + ord2utf8(occ, buffer) + - ((occ == ocd)? 0 : ord2utf8(ocd, buffer)); + length += 1 + _pcre_ord2utf8(occ, buffer) + + ((occ == ocd)? 0 : _pcre_ord2utf8(ocd, buffer)); } } #endif /* SUPPORT_UCP */ /* The length of the (possibly extended) range */ - length += 1 + ord2utf8(c, buffer) + ord2utf8(d, buffer); + length += 1 + _pcre_ord2utf8(c, buffer) + _pcre_ord2utf8(d, buffer); } #endif /* SUPPORT_UTF8 */ @@ -4970,9 +4359,9 @@ while ((c = *(++ptr)) != 0) } #ifdef SUPPORT_UCP length += (((options & PCRE_CASELESS) != 0)? 2 : 1) * - (1 + ord2utf8(c, buffer)); + (1 + _pcre_ord2utf8(c, buffer)); #else /* SUPPORT_UCP */ - length += 1 + ord2utf8(c, buffer); + length += 1 + _pcre_ord2utf8(c, buffer); #endif /* SUPPORT_UCP */ } #endif /* SUPPORT_UTF8 */ @@ -4983,7 +4372,7 @@ while ((c = *(++ptr)) != 0) if (*ptr == 0) /* Missing terminating ']' */ { - *errorptr = ERR6; + errorcode = ERR6; goto PCRE_ERROR_RETURN; } @@ -5000,8 +4389,8 @@ while ((c = *(++ptr)) != 0) if (*ptr != 0 && ptr[1] == '{' && is_counted_repeat(ptr+2)) { - ptr = read_repeat_counts(ptr+2, &min, &max, errorptr); - if (*errorptr != NULL) goto PCRE_ERROR_RETURN; + ptr = read_repeat_counts(ptr+2, &min, &max, &errorcode); + if (errorcode != 0) goto PCRE_ERROR_RETURN; if ((min == 0 && (max == 1 || max == -1)) || (min == 1 && max == -1)) length++; @@ -5037,7 +4426,7 @@ while ((c = *(++ptr)) != 0) while (*ptr != 0 && *ptr != ')') ptr++; if (*ptr == 0) { - *errorptr = ERR18; + errorcode = ERR18; goto PCRE_ERROR_RETURN; } continue; @@ -5072,7 +4461,7 @@ while ((c = *(++ptr)) != 0) while ((digitab[*(++ptr)] & ctype_digit) != 0); if (*ptr != ')') { - *errorptr = ERR29; + errorcode = ERR29; goto PCRE_ERROR_RETURN; } length += 1 + LINK_SIZE; @@ -5098,7 +4487,7 @@ while ((c = *(++ptr)) != 0) while ((digitab[*(++ptr)] & ctype_digit) != 0); if (*ptr != ')') { - *errorptr = ERR39; + errorcode = ERR39; goto PCRE_ERROR_RETURN; } length += 2 + 2*LINK_SIZE; @@ -5115,7 +4504,7 @@ while ((c = *(++ptr)) != 0) while ((compile_block.ctypes[*ptr] & ctype_word) != 0) ptr++; if (*ptr != '>') { - *errorptr = ERR42; + errorcode = ERR42; goto PCRE_ERROR_RETURN; } name_count++; @@ -5128,7 +4517,7 @@ while ((c = *(++ptr)) != 0) while ((compile_block.ctypes[*(++ptr)] & ctype_word) != 0); if (*ptr != ')') { - *errorptr = ERR42; + errorcode = ERR42; goto PCRE_ERROR_RETURN; } break; @@ -5136,7 +4525,7 @@ while ((c = *(++ptr)) != 0) /* Unknown character after (?P */ - *errorptr = ERR41; + errorcode = ERR41; goto PCRE_ERROR_RETURN; /* Lookbehinds are in Perl from version 5.005 */ @@ -5149,7 +4538,7 @@ while ((c = *(++ptr)) != 0) length += 1 + LINK_SIZE; /* For the first branch */ break; } - *errorptr = ERR24; + errorcode = ERR24; goto PCRE_ERROR_RETURN; /* Conditionals are in Perl from version 5.005. The bracket must either @@ -5169,7 +4558,7 @@ while ((c = *(++ptr)) != 0) while ((digitab[*ptr] & ctype_digit) != 0) ptr++; if (*ptr != ')') { - *errorptr = ERR26; + errorcode = ERR26; goto PCRE_ERROR_RETURN; } } @@ -5180,7 +4569,7 @@ while ((c = *(++ptr)) != 0) (ptr[3] != '=' && ptr[3] != '!' && ptr[3] != '<') ) { ptr += 2; /* To get right offset in message */ - *errorptr = ERR28; + errorcode = ERR28; goto PCRE_ERROR_RETURN; } } @@ -5277,7 +4666,7 @@ while ((c = *(++ptr)) != 0) /* Unrecognized option character */ default: - *errorptr = ERR12; + errorcode = ERR12; goto PCRE_ERROR_RETURN; } } @@ -5321,7 +4710,7 @@ while ((c = *(++ptr)) != 0) if (brastackptr >= sizeof(brastack)/sizeof(int)) { - *errorptr = ERR19; + errorcode = ERR19; goto PCRE_ERROR_RETURN; } @@ -5359,8 +4748,8 @@ while ((c = *(++ptr)) != 0) if ((c = ptr[1]) == '{' && is_counted_repeat(ptr+2)) { - ptr = read_repeat_counts(ptr+2, &min, &max, errorptr); - if (*errorptr != NULL) goto PCRE_ERROR_RETURN; + ptr = read_repeat_counts(ptr+2, &min, &max, &errorcode); + if (errorcode != 0) goto PCRE_ERROR_RETURN; } else if (c == '*') { min = 0; max = -1; ptr++; } else if (c == '+') { min = 1; max = -1; ptr++; } @@ -5443,8 +4832,8 @@ if ((options & PCRE_AUTO_CALLOUT) != 0) if (length > MAX_PATTERN_SIZE) { - *errorptr = ERR20; - return NULL; + errorcode = ERR20; + goto PCRE_EARLY_ERROR_RETURN; } /* Compute the size of data block needed and get it, either from malloc or @@ -5455,8 +4844,8 @@ re = (real_pcre *)(pcre_malloc)(size); if (re == NULL) { - *errorptr = ERR21; - return NULL; + errorcode = ERR21; + goto PCRE_EARLY_ERROR_RETURN; } /* Put in the magic number, and save the sizes, options, and character table @@ -5467,11 +4856,12 @@ the end; it's there to help in the case when a regex compiled on a system with re->magic_number = MAGIC_NUMBER; re->size = size; re->options = options; -re->dummy1 = re->dummy2 = 0; +re->dummy1 = 0; re->name_table_offset = sizeof(real_pcre); re->name_entry_size = max_name_size + 3; re->name_count = name_count; -re->tables = (tables == pcre_default_tables)? NULL : tables; +re->ref_count = 0; +re->tables = (tables == _pcre_default_tables)? NULL : tables; re->nullpad = NULL; /* The starting points of the name/number translation table and of the code are @@ -5487,7 +4877,7 @@ compile_block.req_varyopt = 0; compile_block.nopartial = FALSE; /* Set up a starting, non-extracting bracket, then compile the expression. On -error, *errorptr will be set non-NULL, so we don't need to look at the result +error, errorcode will be set non-zero, so we don't need to look at the result of the function here. */ ptr = (const uschar *)pattern; @@ -5495,7 +4885,7 @@ code = (uschar *)codestart; *code = OP_BRA; bracount = 0; (void)compile_regex(options, options & PCRE_IMS, &bracount, &code, &ptr, - errorptr, FALSE, 0, &firstbyte, &reqbyte, NULL, &compile_block); + &errorcode, FALSE, 0, &firstbyte, &reqbyte, NULL, &compile_block); re->top_bracket = bracount; re->top_backref = compile_block.top_backref; @@ -5503,7 +4893,7 @@ if (compile_block.nopartial) re->options |= PCRE_NOPARTIAL; /* If not reached end of pattern on success, there's an excess bracket. */ -if (*errorptr == NULL && *ptr != 0) *errorptr = ERR22; +if (errorcode == 0 && *ptr != 0) errorcode = ERR22; /* Fill in the terminating state and check for disastrous overflow, but if debugging, leave the test till after things are printed out. */ @@ -5511,21 +4901,24 @@ if debugging, leave the test till after things are printed out. */ *code++ = OP_END; #ifndef DEBUG -if (code - codestart > length) *errorptr = ERR23; +if (code - codestart > length) errorcode = ERR23; #endif /* Give an error if there's back reference to a non-existent capturing subpattern. */ -if (re->top_backref > re->top_bracket) *errorptr = ERR15; +if (re->top_backref > re->top_bracket) errorcode = ERR15; /* Failed to compile, or error while post-processing */ -if (*errorptr != NULL) +if (errorcode != 0) { (pcre_free)(re); PCRE_ERROR_RETURN: *erroroffset = ptr - (const uschar *)pattern; + PCRE_EARLY_ERROR_RETURN: + *errorptr = error_texts[errorcode]; + if (errorcodeptr != NULL) *errorcodeptr = errorcode; return NULL; } @@ -5611,16 +5004,17 @@ if ((re->options & PCRE_REQCHSET) != 0) else printf("Req char = \\x%02x%s\n", ch, caseless); } -print_internals(re, stdout); +_pcre_printint(re, stdout); /* This check is done here in the debugging case so that the code that was compiled can be seen. */ if (code - codestart > length) { - *errorptr = ERR23; (pcre_free)(re); + *errorptr = error_texts[ERR23]; *erroroffset = ptr - (uschar *)pattern; + if (errorcodeptr != NULL) *errorcodeptr = ERR23; return NULL; } #endif @@ -5628,3568 +5022,4 @@ if (code - codestart > length) return (pcre *)re; } - - -/************************************************* -* Match a back-reference * -*************************************************/ - -/* If a back reference hasn't been set, the length that is passed is greater -than the number of characters left in the string, so the match fails. - -Arguments: - offset index into the offset vector - eptr points into the subject - length length to be matched - md points to match data block - ims the ims flags - -Returns: TRUE if matched -*/ - -static BOOL -match_ref(int offset, register const uschar *eptr, int length, match_data *md, - unsigned long int ims) -{ -const uschar *p = md->start_subject + md->offset_vector[offset]; - -#ifdef DEBUG -if (eptr >= md->end_subject) - printf("matching subject <null>"); -else - { - printf("matching subject "); - pchars(eptr, length, TRUE, md); - } -printf(" against backref "); -pchars(p, length, FALSE, md); -printf("\n"); -#endif - -/* Always fail if not enough characters left */ - -if (length > md->end_subject - eptr) return FALSE; - -/* Separate the caselesss case for speed */ - -if ((ims & PCRE_CASELESS) != 0) - { - while (length-- > 0) - if (md->lcc[*p++] != md->lcc[*eptr++]) return FALSE; - } -else - { while (length-- > 0) if (*p++ != *eptr++) return FALSE; } - -return TRUE; -} - - -#ifdef SUPPORT_UTF8 -/************************************************* -* Match character against an XCLASS * -*************************************************/ - -/* This function is called from within the XCLASS code below, to match a -character against an extended class which might match values > 255. - -Arguments: - c the character - data points to the flag byte of the XCLASS data - -Returns: TRUE if character matches, else FALSE -*/ - -static BOOL -match_xclass(int c, const uschar *data) -{ -int t; -BOOL negated = (*data & XCL_NOT) != 0; - -/* Character values < 256 are matched against a bitmap, if one is present. If -not, we still carry on, because there may be ranges that start below 256 in the -additional data. */ - -if (c < 256) - { - if ((*data & XCL_MAP) != 0 && (data[1 + c/8] & (1 << (c&7))) != 0) - return !negated; /* char found */ - } - -/* First skip the bit map if present. Then match against the list of Unicode -properties or large chars or ranges that end with a large char. We won't ever -encounter XCL_PROP or XCL_NOTPROP when UCP support is not compiled. */ - -if ((*data++ & XCL_MAP) != 0) data += 32; - -while ((t = *data++) != XCL_END) - { - int x, y; - if (t == XCL_SINGLE) - { - GETCHARINC(x, data); - if (c == x) return !negated; - } - else if (t == XCL_RANGE) - { - GETCHARINC(x, data); - GETCHARINC(y, data); - if (c >= x && c <= y) return !negated; - } - -#ifdef SUPPORT_UCP - else /* XCL_PROP & XCL_NOTPROP */ - { - int chartype, othercase; - int rqdtype = *data++; - int category = ucp_findchar(c, &chartype, &othercase); - if (rqdtype >= 128) - { - if ((rqdtype - 128 == category) == (t == XCL_PROP)) return !negated; - } - else - { - if ((rqdtype == chartype) == (t == XCL_PROP)) return !negated; - } - } -#endif /* SUPPORT_UCP */ - } - -return negated; /* char did not match */ -} -#endif - - -/*************************************************************************** -**************************************************************************** - RECURSION IN THE match() FUNCTION - -The match() function is highly recursive. Some regular expressions can cause -it to recurse thousands of times. I was writing for Unix, so I just let it -call itself recursively. This uses the stack for saving everything that has -to be saved for a recursive call. On Unix, the stack can be large, and this -works fine. - -It turns out that on non-Unix systems there are problems with programs that -use a lot of stack. (This despite the fact that every last chip has oodles -of memory these days, and techniques for extending the stack have been known -for decades.) So.... - -There is a fudge, triggered by defining NO_RECURSE, which avoids recursive -calls by keeping local variables that need to be preserved in blocks of memory -obtained from malloc instead instead of on the stack. Macros are used to -achieve this so that the actual code doesn't look very different to what it -always used to. -**************************************************************************** -***************************************************************************/ - - -/* These versions of the macros use the stack, as normal */ - -#ifndef NO_RECURSE -#define REGISTER register -#define RMATCH(rx,ra,rb,rc,rd,re,rf,rg) rx = match(ra,rb,rc,rd,re,rf,rg) -#define RRETURN(ra) return ra -#else - - -/* These versions of the macros manage a private stack on the heap. Note -that the rd argument of RMATCH isn't actually used. It's the md argument of -match(), which never changes. */ - -#define REGISTER - -#define RMATCH(rx,ra,rb,rc,rd,re,rf,rg)\ - {\ - heapframe *newframe = (pcre_stack_malloc)(sizeof(heapframe));\ - if (setjmp(frame->Xwhere) == 0)\ - {\ - newframe->Xeptr = ra;\ - newframe->Xecode = rb;\ - newframe->Xoffset_top = rc;\ - newframe->Xims = re;\ - newframe->Xeptrb = rf;\ - newframe->Xflags = rg;\ - newframe->Xprevframe = frame;\ - frame = newframe;\ - DPRINTF(("restarting from line %d\n", __LINE__));\ - goto HEAP_RECURSE;\ - }\ - else\ - {\ - DPRINTF(("longjumped back to line %d\n", __LINE__));\ - frame = md->thisframe;\ - rx = frame->Xresult;\ - }\ - } - -#define RRETURN(ra)\ - {\ - heapframe *newframe = frame;\ - frame = newframe->Xprevframe;\ - (pcre_stack_free)(newframe);\ - if (frame != NULL)\ - {\ - frame->Xresult = ra;\ - md->thisframe = frame;\ - longjmp(frame->Xwhere, 1);\ - }\ - return ra;\ - } - - -/* Structure for remembering the local variables in a private frame */ - -typedef struct heapframe { - struct heapframe *Xprevframe; - - /* Function arguments that may change */ - - const uschar *Xeptr; - const uschar *Xecode; - int Xoffset_top; - long int Xims; - eptrblock *Xeptrb; - int Xflags; - - /* Function local variables */ - - const uschar *Xcallpat; - const uschar *Xcharptr; - const uschar *Xdata; - const uschar *Xnext; - const uschar *Xpp; - const uschar *Xprev; - const uschar *Xsaved_eptr; - - recursion_info Xnew_recursive; - - BOOL Xcur_is_word; - BOOL Xcondition; - BOOL Xminimize; - BOOL Xprev_is_word; - - unsigned long int Xoriginal_ims; - -#ifdef SUPPORT_UCP - int Xprop_type; - int Xprop_fail_result; - int Xprop_category; - int Xprop_chartype; - int Xprop_othercase; - int Xprop_test_against; - int *Xprop_test_variable; -#endif - - int Xctype; - int Xfc; - int Xfi; - int Xlength; - int Xmax; - int Xmin; - int Xnumber; - int Xoffset; - int Xop; - int Xsave_capture_last; - int Xsave_offset1, Xsave_offset2, Xsave_offset3; - int Xstacksave[REC_STACK_SAVE_MAX]; - - eptrblock Xnewptrb; - - /* Place to pass back result, and where to jump back to */ - - int Xresult; - jmp_buf Xwhere; - -} heapframe; - -#endif - - -/*************************************************************************** -***************************************************************************/ - - - -/************************************************* -* Match from current position * -*************************************************/ - -/* On entry ecode points to the first opcode, and eptr to the first character -in the subject string, while eptrb holds the value of eptr at the start of the -last bracketed group - used for breaking infinite loops matching zero-length -strings. This function is called recursively in many circumstances. Whenever it -returns a negative (error) response, the outer incarnation must also return the -same response. - -Performance note: It might be tempting to extract commonly used fields from the -md structure (e.g. utf8, end_subject) into individual variables to improve -performance. Tests using gcc on a SPARC disproved this; in the first case, it -made performance worse. - -Arguments: - eptr pointer in subject - ecode position in code - offset_top current top pointer - md pointer to "static" info for the match - ims current /i, /m, and /s options - eptrb pointer to chain of blocks containing eptr at start of - brackets - for testing for empty matches - flags can contain - match_condassert - this is an assertion condition - match_isgroup - this is the start of a bracketed group - -Returns: MATCH_MATCH if matched ) these values are >= 0 - MATCH_NOMATCH if failed to match ) - a negative PCRE_ERROR_xxx value if aborted by an error condition - (e.g. stopped by recursion limit) -*/ - -static int -match(REGISTER const uschar *eptr, REGISTER const uschar *ecode, - int offset_top, match_data *md, unsigned long int ims, eptrblock *eptrb, - int flags) -{ -/* These variables do not need to be preserved over recursion in this function, -so they can be ordinary variables in all cases. Mark them with "register" -because they are used a lot in loops. */ - -register int rrc; /* Returns from recursive calls */ -register int i; /* Used for loops not involving calls to RMATCH() */ -register int c; /* Character values not kept over RMATCH() calls */ - -/* When recursion is not being used, all "local" variables that have to be -preserved over calls to RMATCH() are part of a "frame" which is obtained from -heap storage. Set up the top-level frame here; others are obtained from the -heap whenever RMATCH() does a "recursion". See the macro definitions above. */ - -#ifdef NO_RECURSE -heapframe *frame = (pcre_stack_malloc)(sizeof(heapframe)); -frame->Xprevframe = NULL; /* Marks the top level */ - -/* Copy in the original argument variables */ - -frame->Xeptr = eptr; -frame->Xecode = ecode; -frame->Xoffset_top = offset_top; -frame->Xims = ims; -frame->Xeptrb = eptrb; -frame->Xflags = flags; - -/* This is where control jumps back to to effect "recursion" */ - -HEAP_RECURSE: - -/* Macros make the argument variables come from the current frame */ - -#define eptr frame->Xeptr -#define ecode frame->Xecode -#define offset_top frame->Xoffset_top -#define ims frame->Xims -#define eptrb frame->Xeptrb -#define flags frame->Xflags - -/* Ditto for the local variables */ - -#ifdef SUPPORT_UTF8 -#define charptr frame->Xcharptr -#endif -#define callpat frame->Xcallpat -#define data frame->Xdata -#define next frame->Xnext -#define pp frame->Xpp -#define prev frame->Xprev -#define saved_eptr frame->Xsaved_eptr - -#define new_recursive frame->Xnew_recursive - -#define cur_is_word frame->Xcur_is_word -#define condition frame->Xcondition -#define minimize frame->Xminimize -#define prev_is_word frame->Xprev_is_word - -#define original_ims frame->Xoriginal_ims - -#ifdef SUPPORT_UCP -#define prop_type frame->Xprop_type -#define prop_fail_result frame->Xprop_fail_result -#define prop_category frame->Xprop_category -#define prop_chartype frame->Xprop_chartype -#define prop_othercase frame->Xprop_othercase -#define prop_test_against frame->Xprop_test_against -#define prop_test_variable frame->Xprop_test_variable -#endif - -#define ctype frame->Xctype -#define fc frame->Xfc -#define fi frame->Xfi -#define length frame->Xlength -#define max frame->Xmax -#define min frame->Xmin -#define number frame->Xnumber -#define offset frame->Xoffset -#define op frame->Xop -#define save_capture_last frame->Xsave_capture_last -#define save_offset1 frame->Xsave_offset1 -#define save_offset2 frame->Xsave_offset2 -#define save_offset3 frame->Xsave_offset3 -#define stacksave frame->Xstacksave - -#define newptrb frame->Xnewptrb - -/* When recursion is being used, local variables are allocated on the stack and -get preserved during recursion in the normal way. In this environment, fi and -i, and fc and c, can be the same variables. */ - -#else -#define fi i -#define fc c - - -#ifdef SUPPORT_UTF8 /* Many of these variables are used ony */ -const uschar *charptr; /* small blocks of the code. My normal */ -#endif /* style of coding would have declared */ -const uschar *callpat; /* them within each of those blocks. */ -const uschar *data; /* However, in order to accommodate the */ -const uschar *next; /* version of this code that uses an */ -const uschar *pp; /* external "stack" implemented on the */ -const uschar *prev; /* heap, it is easier to declare them */ -const uschar *saved_eptr; /* all here, so the declarations can */ - /* be cut out in a block. The only */ -recursion_info new_recursive; /* declarations within blocks below are */ - /* for variables that do not have to */ -BOOL cur_is_word; /* be preserved over a recursive call */ -BOOL condition; /* to RMATCH(). */ -BOOL minimize; -BOOL prev_is_word; - -unsigned long int original_ims; - -#ifdef SUPPORT_UCP -int prop_type; -int prop_fail_result; -int prop_category; -int prop_chartype; -int prop_othercase; -int prop_test_against; -int *prop_test_variable; -#endif - -int ctype; -int length; -int max; -int min; -int number; -int offset; -int op; -int save_capture_last; -int save_offset1, save_offset2, save_offset3; -int stacksave[REC_STACK_SAVE_MAX]; - -eptrblock newptrb; -#endif - -/* These statements are here to stop the compiler complaining about unitialized -variables. */ - -#ifdef SUPPORT_UCP -prop_fail_result = 0; -prop_test_against = 0; -prop_test_variable = NULL; -#endif - -/* OK, now we can get on with the real code of the function. Recursion is -specified by the macros RMATCH and RRETURN. When NO_RECURSE is *not* defined, -these just turn into a recursive call to match() and a "return", respectively. -However, RMATCH isn't like a function call because it's quite a complicated -macro. It has to be used in one particular way. This shouldn't, however, impact -performance when true recursion is being used. */ - -if (md->match_call_count++ >= md->match_limit) RRETURN(PCRE_ERROR_MATCHLIMIT); - -original_ims = ims; /* Save for resetting on ')' */ - -/* At the start of a bracketed group, add the current subject pointer to the -stack of such pointers, to be re-instated at the end of the group when we hit -the closing ket. When match() is called in other circumstances, we don't add to -this stack. */ - -if ((flags & match_isgroup) != 0) - { - newptrb.epb_prev = eptrb; - newptrb.epb_saved_eptr = eptr; - eptrb = &newptrb; - } - -/* Now start processing the operations. */ - -for (;;) - { - op = *ecode; - minimize = FALSE; - - /* For partial matching, remember if we ever hit the end of the subject after - matching at least one subject character. */ - - if (md->partial && - eptr >= md->end_subject && - eptr > md->start_match) - md->hitend = TRUE; - - /* Opening capturing bracket. If there is space in the offset vector, save - the current subject position in the working slot at the top of the vector. We - mustn't change the current values of the data slot, because they may be set - from a previous iteration of this group, and be referred to by a reference - inside the group. - - If the bracket fails to match, we need to restore this value and also the - values of the final offsets, in case they were set by a previous iteration of - the same bracket. - - If there isn't enough space in the offset vector, treat this as if it were a - non-capturing bracket. Don't worry about setting the flag for the error case - here; that is handled in the code for KET. */ - - if (op > OP_BRA) - { - number = op - OP_BRA; - - /* For extended extraction brackets (large number), we have to fish out the - number from a dummy opcode at the start. */ - - if (number > EXTRACT_BASIC_MAX) - number = GET2(ecode, 2+LINK_SIZE); - offset = number << 1; - -#ifdef DEBUG - printf("start bracket %d subject=", number); - pchars(eptr, 16, TRUE, md); - printf("\n"); -#endif - - if (offset < md->offset_max) - { - save_offset1 = md->offset_vector[offset]; - save_offset2 = md->offset_vector[offset+1]; - save_offset3 = md->offset_vector[md->offset_end - number]; - save_capture_last = md->capture_last; - - DPRINTF(("saving %d %d %d\n", save_offset1, save_offset2, save_offset3)); - md->offset_vector[md->offset_end - number] = eptr - md->start_subject; - - do - { - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, - match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - md->capture_last = save_capture_last; - ecode += GET(ecode, 1); - } - while (*ecode == OP_ALT); - - DPRINTF(("bracket %d failed\n", number)); - - md->offset_vector[offset] = save_offset1; - md->offset_vector[offset+1] = save_offset2; - md->offset_vector[md->offset_end - number] = save_offset3; - - RRETURN(MATCH_NOMATCH); - } - - /* Insufficient room for saving captured contents */ - - else op = OP_BRA; - } - - /* Other types of node can be handled by a switch */ - - switch(op) - { - case OP_BRA: /* Non-capturing bracket: optimized */ - DPRINTF(("start bracket 0\n")); - do - { - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, - match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - ecode += GET(ecode, 1); - } - while (*ecode == OP_ALT); - DPRINTF(("bracket 0 failed\n")); - RRETURN(MATCH_NOMATCH); - - /* Conditional group: compilation checked that there are no more than - two branches. If the condition is false, skipping the first branch takes us - past the end if there is only one branch, but that's OK because that is - exactly what going to the ket would do. */ - - case OP_COND: - if (ecode[LINK_SIZE+1] == OP_CREF) /* Condition extract or recurse test */ - { - offset = GET2(ecode, LINK_SIZE+2) << 1; /* Doubled ref number */ - condition = (offset == CREF_RECURSE * 2)? - (md->recursive != NULL) : - (offset < offset_top && md->offset_vector[offset] >= 0); - RMATCH(rrc, eptr, ecode + (condition? - (LINK_SIZE + 4) : (LINK_SIZE + 1 + GET(ecode, 1))), - offset_top, md, ims, eptrb, match_isgroup); - RRETURN(rrc); - } - - /* The condition is an assertion. Call match() to evaluate it - setting - the final argument TRUE causes it to stop at the end of an assertion. */ - - else - { - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, NULL, - match_condassert | match_isgroup); - if (rrc == MATCH_MATCH) - { - ecode += 1 + LINK_SIZE + GET(ecode, LINK_SIZE+2); - while (*ecode == OP_ALT) ecode += GET(ecode, 1); - } - else if (rrc != MATCH_NOMATCH) - { - RRETURN(rrc); /* Need braces because of following else */ - } - else ecode += GET(ecode, 1); - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, - match_isgroup); - RRETURN(rrc); - } - /* Control never reaches here */ - - /* Skip over conditional reference or large extraction number data if - encountered. */ - - case OP_CREF: - case OP_BRANUMBER: - ecode += 3; - break; - - /* End of the pattern. If we are in a recursion, we should restore the - offsets appropriately and continue from after the call. */ - - case OP_END: - if (md->recursive != NULL && md->recursive->group_num == 0) - { - recursion_info *rec = md->recursive; - DPRINTF(("Hit the end in a (?0) recursion\n")); - md->recursive = rec->prevrec; - memmove(md->offset_vector, rec->offset_save, - rec->saved_max * sizeof(int)); - md->start_match = rec->save_start; - ims = original_ims; - ecode = rec->after_call; - break; - } - - /* Otherwise, if PCRE_NOTEMPTY is set, fail if we have matched an empty - string - backtracking will then try other alternatives, if any. */ - - if (md->notempty && eptr == md->start_match) RRETURN(MATCH_NOMATCH); - md->end_match_ptr = eptr; /* Record where we ended */ - md->end_offset_top = offset_top; /* and how many extracts were taken */ - RRETURN(MATCH_MATCH); - - /* Change option settings */ - - case OP_OPT: - ims = ecode[1]; - ecode += 2; - DPRINTF(("ims set to %02lx\n", ims)); - break; - - /* Assertion brackets. Check the alternative branches in turn - the - matching won't pass the KET for an assertion. If any one branch matches, - the assertion is true. Lookbehind assertions have an OP_REVERSE item at the - start of each branch to move the current point backwards, so the code at - this level is identical to the lookahead case. */ - - case OP_ASSERT: - case OP_ASSERTBACK: - do - { - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, NULL, - match_isgroup); - if (rrc == MATCH_MATCH) break; - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - ecode += GET(ecode, 1); - } - while (*ecode == OP_ALT); - if (*ecode == OP_KET) RRETURN(MATCH_NOMATCH); - - /* If checking an assertion for a condition, return MATCH_MATCH. */ - - if ((flags & match_condassert) != 0) RRETURN(MATCH_MATCH); - - /* Continue from after the assertion, updating the offsets high water - mark, since extracts may have been taken during the assertion. */ - - do ecode += GET(ecode,1); while (*ecode == OP_ALT); - ecode += 1 + LINK_SIZE; - offset_top = md->end_offset_top; - continue; - - /* Negative assertion: all branches must fail to match */ - - case OP_ASSERT_NOT: - case OP_ASSERTBACK_NOT: - do - { - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, NULL, - match_isgroup); - if (rrc == MATCH_MATCH) RRETURN(MATCH_NOMATCH); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - ecode += GET(ecode,1); - } - while (*ecode == OP_ALT); - - if ((flags & match_condassert) != 0) RRETURN(MATCH_MATCH); - - ecode += 1 + LINK_SIZE; - continue; - - /* Move the subject pointer back. This occurs only at the start of - each branch of a lookbehind assertion. If we are too close to the start to - move back, this match function fails. When working with UTF-8 we move - back a number of characters, not bytes. */ - - case OP_REVERSE: -#ifdef SUPPORT_UTF8 - if (md->utf8) - { - c = GET(ecode,1); - for (i = 0; i < c; i++) - { - eptr--; - if (eptr < md->start_subject) RRETURN(MATCH_NOMATCH); - BACKCHAR(eptr) - } - } - else -#endif - - /* No UTF-8 support, or not in UTF-8 mode: count is byte count */ - - { - eptr -= GET(ecode,1); - if (eptr < md->start_subject) RRETURN(MATCH_NOMATCH); - } - - /* Skip to next op code */ - - ecode += 1 + LINK_SIZE; - break; - - /* The callout item calls an external function, if one is provided, passing - details of the match so far. This is mainly for debugging, though the - function is able to force a failure. */ - - case OP_CALLOUT: - if (pcre_callout != NULL) - { - pcre_callout_block cb; - cb.version = 1; /* Version 1 of the callout block */ - cb.callout_number = ecode[1]; - cb.offset_vector = md->offset_vector; - cb.subject = (const char *)md->start_subject; - cb.subject_length = md->end_subject - md->start_subject; - cb.start_match = md->start_match - md->start_subject; - cb.current_position = eptr - md->start_subject; - cb.pattern_position = GET(ecode, 2); - cb.next_item_length = GET(ecode, 2 + LINK_SIZE); - cb.capture_top = offset_top/2; - cb.capture_last = md->capture_last; - cb.callout_data = md->callout_data; - if ((rrc = (*pcre_callout)(&cb)) > 0) RRETURN(MATCH_NOMATCH); - if (rrc < 0) RRETURN(rrc); - } - ecode += 2 + 2*LINK_SIZE; - break; - - /* Recursion either matches the current regex, or some subexpression. The - offset data is the offset to the starting bracket from the start of the - whole pattern. (This is so that it works from duplicated subpatterns.) - - If there are any capturing brackets started but not finished, we have to - save their starting points and reinstate them after the recursion. However, - we don't know how many such there are (offset_top records the completed - total) so we just have to save all the potential data. There may be up to - 65535 such values, which is too large to put on the stack, but using malloc - for small numbers seems expensive. As a compromise, the stack is used when - there are no more than REC_STACK_SAVE_MAX values to store; otherwise malloc - is used. A problem is what to do if the malloc fails ... there is no way of - returning to the top level with an error. Save the top REC_STACK_SAVE_MAX - values on the stack, and accept that the rest may be wrong. - - There are also other values that have to be saved. We use a chained - sequence of blocks that actually live on the stack. Thanks to Robin Houston - for the original version of this logic. */ - - case OP_RECURSE: - { - callpat = md->start_code + GET(ecode, 1); - new_recursive.group_num = *callpat - OP_BRA; - - /* For extended extraction brackets (large number), we have to fish out - the number from a dummy opcode at the start. */ - - if (new_recursive.group_num > EXTRACT_BASIC_MAX) - new_recursive.group_num = GET2(callpat, 2+LINK_SIZE); - - /* Add to "recursing stack" */ - - new_recursive.prevrec = md->recursive; - md->recursive = &new_recursive; - - /* Find where to continue from afterwards */ - - ecode += 1 + LINK_SIZE; - new_recursive.after_call = ecode; - - /* Now save the offset data. */ - - new_recursive.saved_max = md->offset_end; - if (new_recursive.saved_max <= REC_STACK_SAVE_MAX) - new_recursive.offset_save = stacksave; - else - { - new_recursive.offset_save = - (int *)(pcre_malloc)(new_recursive.saved_max * sizeof(int)); - if (new_recursive.offset_save == NULL) RRETURN(PCRE_ERROR_NOMEMORY); - } - - memcpy(new_recursive.offset_save, md->offset_vector, - new_recursive.saved_max * sizeof(int)); - new_recursive.save_start = md->start_match; - md->start_match = eptr; - - /* OK, now we can do the recursion. For each top-level alternative we - restore the offset and recursion data. */ - - DPRINTF(("Recursing into group %d\n", new_recursive.group_num)); - do - { - RMATCH(rrc, eptr, callpat + 1 + LINK_SIZE, offset_top, md, ims, - eptrb, match_isgroup); - if (rrc == MATCH_MATCH) - { - md->recursive = new_recursive.prevrec; - if (new_recursive.offset_save != stacksave) - (pcre_free)(new_recursive.offset_save); - RRETURN(MATCH_MATCH); - } - else if (rrc != MATCH_NOMATCH) RRETURN(rrc); - - md->recursive = &new_recursive; - memcpy(md->offset_vector, new_recursive.offset_save, - new_recursive.saved_max * sizeof(int)); - callpat += GET(callpat, 1); - } - while (*callpat == OP_ALT); - - DPRINTF(("Recursion didn't match\n")); - md->recursive = new_recursive.prevrec; - if (new_recursive.offset_save != stacksave) - (pcre_free)(new_recursive.offset_save); - RRETURN(MATCH_NOMATCH); - } - /* Control never reaches here */ - - /* "Once" brackets are like assertion brackets except that after a match, - the point in the subject string is not moved back. Thus there can never be - a move back into the brackets. Friedl calls these "atomic" subpatterns. - Check the alternative branches in turn - the matching won't pass the KET - for this kind of subpattern. If any one branch matches, we carry on as at - the end of a normal bracket, leaving the subject pointer. */ - - case OP_ONCE: - { - prev = ecode; - saved_eptr = eptr; - - do - { - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, - eptrb, match_isgroup); - if (rrc == MATCH_MATCH) break; - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - ecode += GET(ecode,1); - } - while (*ecode == OP_ALT); - - /* If hit the end of the group (which could be repeated), fail */ - - if (*ecode != OP_ONCE && *ecode != OP_ALT) RRETURN(MATCH_NOMATCH); - - /* Continue as from after the assertion, updating the offsets high water - mark, since extracts may have been taken. */ - - do ecode += GET(ecode,1); while (*ecode == OP_ALT); - - offset_top = md->end_offset_top; - eptr = md->end_match_ptr; - - /* For a non-repeating ket, just continue at this level. This also - happens for a repeating ket if no characters were matched in the group. - This is the forcible breaking of infinite loops as implemented in Perl - 5.005. If there is an options reset, it will get obeyed in the normal - course of events. */ - - if (*ecode == OP_KET || eptr == saved_eptr) - { - ecode += 1+LINK_SIZE; - break; - } - - /* The repeating kets try the rest of the pattern or restart from the - preceding bracket, in the appropriate order. We need to reset any options - that changed within the bracket before re-running it, so check the next - opcode. */ - - if (ecode[1+LINK_SIZE] == OP_OPT) - { - ims = (ims & ~PCRE_IMS) | ecode[4]; - DPRINTF(("ims set to %02lx at group repeat\n", ims)); - } - - if (*ecode == OP_KETRMIN) - { - RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - else /* OP_KETRMAX */ - { - RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - RMATCH(rrc, eptr, ecode + 1+LINK_SIZE, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - } - RRETURN(MATCH_NOMATCH); - - /* An alternation is the end of a branch; scan along to find the end of the - bracketed group and go to there. */ - - case OP_ALT: - do ecode += GET(ecode,1); while (*ecode == OP_ALT); - break; - - /* BRAZERO and BRAMINZERO occur just before a bracket group, indicating - that it may occur zero times. It may repeat infinitely, or not at all - - i.e. it could be ()* or ()? in the pattern. Brackets with fixed upper - repeat limits are compiled as a number of copies, with the optional ones - preceded by BRAZERO or BRAMINZERO. */ - - case OP_BRAZERO: - { - next = ecode+1; - RMATCH(rrc, eptr, next, offset_top, md, ims, eptrb, match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - do next += GET(next,1); while (*next == OP_ALT); - ecode = next + 1+LINK_SIZE; - } - break; - - case OP_BRAMINZERO: - { - next = ecode+1; - do next += GET(next,1); while (*next == OP_ALT); - RMATCH(rrc, eptr, next + 1+LINK_SIZE, offset_top, md, ims, eptrb, - match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - ecode++; - } - break; - - /* End of a group, repeated or non-repeating. If we are at the end of - an assertion "group", stop matching and return MATCH_MATCH, but record the - current high water mark for use by positive assertions. Do this also - for the "once" (not-backup up) groups. */ - - case OP_KET: - case OP_KETRMIN: - case OP_KETRMAX: - { - prev = ecode - GET(ecode, 1); - saved_eptr = eptrb->epb_saved_eptr; - - /* Back up the stack of bracket start pointers. */ - - eptrb = eptrb->epb_prev; - - if (*prev == OP_ASSERT || *prev == OP_ASSERT_NOT || - *prev == OP_ASSERTBACK || *prev == OP_ASSERTBACK_NOT || - *prev == OP_ONCE) - { - md->end_match_ptr = eptr; /* For ONCE */ - md->end_offset_top = offset_top; - RRETURN(MATCH_MATCH); - } - - /* In all other cases except a conditional group we have to check the - group number back at the start and if necessary complete handling an - extraction by setting the offsets and bumping the high water mark. */ - - if (*prev != OP_COND) - { - number = *prev - OP_BRA; - - /* For extended extraction brackets (large number), we have to fish out - the number from a dummy opcode at the start. */ - - if (number > EXTRACT_BASIC_MAX) number = GET2(prev, 2+LINK_SIZE); - offset = number << 1; - -#ifdef DEBUG - printf("end bracket %d", number); - printf("\n"); -#endif - - /* Test for a numbered group. This includes groups called as a result - of recursion. Note that whole-pattern recursion is coded as a recurse - into group 0, so it won't be picked up here. Instead, we catch it when - the OP_END is reached. */ - - if (number > 0) - { - md->capture_last = number; - if (offset >= md->offset_max) md->offset_overflow = TRUE; else - { - md->offset_vector[offset] = - md->offset_vector[md->offset_end - number]; - md->offset_vector[offset+1] = eptr - md->start_subject; - if (offset_top <= offset) offset_top = offset + 2; - } - - /* Handle a recursively called group. Restore the offsets - appropriately and continue from after the call. */ - - if (md->recursive != NULL && md->recursive->group_num == number) - { - recursion_info *rec = md->recursive; - DPRINTF(("Recursion (%d) succeeded - continuing\n", number)); - md->recursive = rec->prevrec; - md->start_match = rec->save_start; - memcpy(md->offset_vector, rec->offset_save, - rec->saved_max * sizeof(int)); - ecode = rec->after_call; - ims = original_ims; - break; - } - } - } - - /* Reset the value of the ims flags, in case they got changed during - the group. */ - - ims = original_ims; - DPRINTF(("ims reset to %02lx\n", ims)); - - /* For a non-repeating ket, just continue at this level. This also - happens for a repeating ket if no characters were matched in the group. - This is the forcible breaking of infinite loops as implemented in Perl - 5.005. If there is an options reset, it will get obeyed in the normal - course of events. */ - - if (*ecode == OP_KET || eptr == saved_eptr) - { - ecode += 1 + LINK_SIZE; - break; - } - - /* The repeating kets try the rest of the pattern or restart from the - preceding bracket, in the appropriate order. */ - - if (*ecode == OP_KETRMIN) - { - RMATCH(rrc, eptr, ecode + 1+LINK_SIZE, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - else /* OP_KETRMAX */ - { - RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - RMATCH(rrc, eptr, ecode + 1+LINK_SIZE, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - } - - RRETURN(MATCH_NOMATCH); - - /* Start of subject unless notbol, or after internal newline if multiline */ - - case OP_CIRC: - if (md->notbol && eptr == md->start_subject) RRETURN(MATCH_NOMATCH); - if ((ims & PCRE_MULTILINE) != 0) - { - if (eptr != md->start_subject && eptr[-1] != NEWLINE) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - } - /* ... else fall through */ - - /* Start of subject assertion */ - - case OP_SOD: - if (eptr != md->start_subject) RRETURN(MATCH_NOMATCH); - ecode++; - break; - - /* Start of match assertion */ - - case OP_SOM: - if (eptr != md->start_subject + md->start_offset) RRETURN(MATCH_NOMATCH); - ecode++; - break; - - /* Assert before internal newline if multiline, or before a terminating - newline unless endonly is set, else end of subject unless noteol is set. */ - - case OP_DOLL: - if ((ims & PCRE_MULTILINE) != 0) - { - if (eptr < md->end_subject) - { if (*eptr != NEWLINE) RRETURN(MATCH_NOMATCH); } - else - { if (md->noteol) RRETURN(MATCH_NOMATCH); } - ecode++; - break; - } - else - { - if (md->noteol) RRETURN(MATCH_NOMATCH); - if (!md->endonly) - { - if (eptr < md->end_subject - 1 || - (eptr == md->end_subject - 1 && *eptr != NEWLINE)) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - } - } - /* ... else fall through */ - - /* End of subject assertion (\z) */ - - case OP_EOD: - if (eptr < md->end_subject) RRETURN(MATCH_NOMATCH); - ecode++; - break; - - /* End of subject or ending \n assertion (\Z) */ - - case OP_EODN: - if (eptr < md->end_subject - 1 || - (eptr == md->end_subject - 1 && *eptr != NEWLINE)) RRETURN(MATCH_NOMATCH); - ecode++; - break; - - /* Word boundary assertions */ - - case OP_NOT_WORD_BOUNDARY: - case OP_WORD_BOUNDARY: - { - - /* Find out if the previous and current characters are "word" characters. - It takes a bit more work in UTF-8 mode. Characters > 255 are assumed to - be "non-word" characters. */ - -#ifdef SUPPORT_UTF8 - if (md->utf8) - { - if (eptr == md->start_subject) prev_is_word = FALSE; else - { - const uschar *lastptr = eptr - 1; - while((*lastptr & 0xc0) == 0x80) lastptr--; - GETCHAR(c, lastptr); - prev_is_word = c < 256 && (md->ctypes[c] & ctype_word) != 0; - } - if (eptr >= md->end_subject) cur_is_word = FALSE; else - { - GETCHAR(c, eptr); - cur_is_word = c < 256 && (md->ctypes[c] & ctype_word) != 0; - } - } - else -#endif - - /* More streamlined when not in UTF-8 mode */ - - { - prev_is_word = (eptr != md->start_subject) && - ((md->ctypes[eptr[-1]] & ctype_word) != 0); - cur_is_word = (eptr < md->end_subject) && - ((md->ctypes[*eptr] & ctype_word) != 0); - } - - /* Now see if the situation is what we want */ - - if ((*ecode++ == OP_WORD_BOUNDARY)? - cur_is_word == prev_is_word : cur_is_word != prev_is_word) - RRETURN(MATCH_NOMATCH); - } - break; - - /* Match a single character type; inline for speed */ - - case OP_ANY: - if ((ims & PCRE_DOTALL) == 0 && eptr < md->end_subject && *eptr == NEWLINE) - RRETURN(MATCH_NOMATCH); - if (eptr++ >= md->end_subject) RRETURN(MATCH_NOMATCH); -#ifdef SUPPORT_UTF8 - if (md->utf8) - while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; -#endif - ecode++; - break; - - /* Match a single byte, even in UTF-8 mode. This opcode really does match - any byte, even newline, independent of the setting of PCRE_DOTALL. */ - - case OP_ANYBYTE: - if (eptr++ >= md->end_subject) RRETURN(MATCH_NOMATCH); - ecode++; - break; - - case OP_NOT_DIGIT: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - if ( -#ifdef SUPPORT_UTF8 - c < 256 && -#endif - (md->ctypes[c] & ctype_digit) != 0 - ) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - - case OP_DIGIT: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - if ( -#ifdef SUPPORT_UTF8 - c >= 256 || -#endif - (md->ctypes[c] & ctype_digit) == 0 - ) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - - case OP_NOT_WHITESPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - if ( -#ifdef SUPPORT_UTF8 - c < 256 && -#endif - (md->ctypes[c] & ctype_space) != 0 - ) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - - case OP_WHITESPACE: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - if ( -#ifdef SUPPORT_UTF8 - c >= 256 || -#endif - (md->ctypes[c] & ctype_space) == 0 - ) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - - case OP_NOT_WORDCHAR: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - if ( -#ifdef SUPPORT_UTF8 - c < 256 && -#endif - (md->ctypes[c] & ctype_word) != 0 - ) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - - case OP_WORDCHAR: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - if ( -#ifdef SUPPORT_UTF8 - c >= 256 || -#endif - (md->ctypes[c] & ctype_word) == 0 - ) - RRETURN(MATCH_NOMATCH); - ecode++; - break; - -#ifdef SUPPORT_UCP - /* Check the next character by Unicode property. We will get here only - if the support is in the binary; otherwise a compile-time error occurs. */ - - case OP_PROP: - case OP_NOTPROP: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - { - int chartype, rqdtype; - int othercase; - int category = ucp_findchar(c, &chartype, &othercase); - - rqdtype = *(++ecode); - ecode++; - - if (rqdtype >= 128) - { - if ((rqdtype - 128 != category) == (op == OP_PROP)) - RRETURN(MATCH_NOMATCH); - } - else - { - if ((rqdtype != chartype) == (op == OP_PROP)) - RRETURN(MATCH_NOMATCH); - } - } - break; - - /* Match an extended Unicode sequence. We will get here only if the support - is in the binary; otherwise a compile-time error occurs. */ - - case OP_EXTUNI: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - { - int chartype; - int othercase; - int category = ucp_findchar(c, &chartype, &othercase); - if (category == ucp_M) RRETURN(MATCH_NOMATCH); - while (eptr < md->end_subject) - { - int len = 1; - if (!md->utf8) c = *eptr; else - { - GETCHARLEN(c, eptr, len); - } - category = ucp_findchar(c, &chartype, &othercase); - if (category != ucp_M) break; - eptr += len; - } - } - ecode++; - break; -#endif - - - /* Match a back reference, possibly repeatedly. Look past the end of the - item to see if there is repeat information following. The code is similar - to that for character classes, but repeated for efficiency. Then obey - similar code to character type repeats - written out again for speed. - However, if the referenced string is the empty string, always treat - it as matched, any number of times (otherwise there could be infinite - loops). */ - - case OP_REF: - { - offset = GET2(ecode, 1) << 1; /* Doubled ref number */ - ecode += 3; /* Advance past item */ - - /* If the reference is unset, set the length to be longer than the amount - of subject left; this ensures that every attempt at a match fails. We - can't just fail here, because of the possibility of quantifiers with zero - minima. */ - - length = (offset >= offset_top || md->offset_vector[offset] < 0)? - md->end_subject - eptr + 1 : - md->offset_vector[offset+1] - md->offset_vector[offset]; - - /* Set up for repetition, or handle the non-repeated case */ - - switch (*ecode) - { - case OP_CRSTAR: - case OP_CRMINSTAR: - case OP_CRPLUS: - case OP_CRMINPLUS: - case OP_CRQUERY: - case OP_CRMINQUERY: - c = *ecode++ - OP_CRSTAR; - minimize = (c & 1) != 0; - min = rep_min[c]; /* Pick up values from tables; */ - max = rep_max[c]; /* zero for max => infinity */ - if (max == 0) max = INT_MAX; - break; - - case OP_CRRANGE: - case OP_CRMINRANGE: - minimize = (*ecode == OP_CRMINRANGE); - min = GET2(ecode, 1); - max = GET2(ecode, 3); - if (max == 0) max = INT_MAX; - ecode += 5; - break; - - default: /* No repeat follows */ - if (!match_ref(offset, eptr, length, md, ims)) RRETURN(MATCH_NOMATCH); - eptr += length; - continue; /* With the main loop */ - } - - /* If the length of the reference is zero, just continue with the - main loop. */ - - if (length == 0) continue; - - /* First, ensure the minimum number of matches are present. We get back - the length of the reference string explicitly rather than passing the - address of eptr, so that eptr can be a register variable. */ - - for (i = 1; i <= min; i++) - { - if (!match_ref(offset, eptr, length, md, ims)) RRETURN(MATCH_NOMATCH); - eptr += length; - } - - /* If min = max, continue at the same level without recursion. - They are not both allowed to be zero. */ - - if (min == max) continue; - - /* If minimizing, keep trying and advancing the pointer */ - - if (minimize) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || !match_ref(offset, eptr, length, md, ims)) - RRETURN(MATCH_NOMATCH); - eptr += length; - } - /* Control never gets here */ - } - - /* If maximizing, find the longest string and work backwards */ - - else - { - pp = eptr; - for (i = min; i < max; i++) - { - if (!match_ref(offset, eptr, length, md, ims)) break; - eptr += length; - } - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - eptr -= length; - } - RRETURN(MATCH_NOMATCH); - } - } - /* Control never gets here */ - - - - /* Match a bit-mapped character class, possibly repeatedly. This op code is - used when all the characters in the class have values in the range 0-255, - and either the matching is caseful, or the characters are in the range - 0-127 when UTF-8 processing is enabled. The only difference between - OP_CLASS and OP_NCLASS occurs when a data character outside the range is - encountered. - - First, look past the end of the item to see if there is repeat information - following. Then obey similar code to character type repeats - written out - again for speed. */ - - case OP_NCLASS: - case OP_CLASS: - { - data = ecode + 1; /* Save for matching */ - ecode += 33; /* Advance past the item */ - - switch (*ecode) - { - case OP_CRSTAR: - case OP_CRMINSTAR: - case OP_CRPLUS: - case OP_CRMINPLUS: - case OP_CRQUERY: - case OP_CRMINQUERY: - c = *ecode++ - OP_CRSTAR; - minimize = (c & 1) != 0; - min = rep_min[c]; /* Pick up values from tables; */ - max = rep_max[c]; /* zero for max => infinity */ - if (max == 0) max = INT_MAX; - break; - - case OP_CRRANGE: - case OP_CRMINRANGE: - minimize = (*ecode == OP_CRMINRANGE); - min = GET2(ecode, 1); - max = GET2(ecode, 3); - if (max == 0) max = INT_MAX; - ecode += 5; - break; - - default: /* No repeat follows */ - min = max = 1; - break; - } - - /* First, ensure the minimum number of matches are present. */ - -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINC(c, eptr); - if (c > 255) - { - if (op == OP_CLASS) RRETURN(MATCH_NOMATCH); - } - else - { - if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); - } - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - c = *eptr++; - if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); - } - } - - /* If max == min we can continue with the main loop without the - need to recurse. */ - - if (min == max) continue; - - /* If minimizing, keep testing the rest of the expression and advancing - the pointer while it matches the class. */ - - if (minimize) - { -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINC(c, eptr); - if (c > 255) - { - if (op == OP_CLASS) RRETURN(MATCH_NOMATCH); - } - else - { - if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); - } - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - c = *eptr++; - if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); - } - } - /* Control never gets here */ - } - - /* If maximizing, find the longest possible run, then work backwards. */ - - else - { - pp = eptr; - -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (c > 255) - { - if (op == OP_CLASS) break; - } - else - { - if ((data[c/8] & (1 << (c&7))) == 0) break; - } - eptr += len; - } - for (;;) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ - BACKCHAR(eptr); - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject) break; - c = *eptr; - if ((data[c/8] & (1 << (c&7))) == 0) break; - eptr++; - } - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - eptr--; - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - } - - RRETURN(MATCH_NOMATCH); - } - } - /* Control never gets here */ - - - /* Match an extended character class. This opcode is encountered only - in UTF-8 mode, because that's the only time it is compiled. */ - -#ifdef SUPPORT_UTF8 - case OP_XCLASS: - { - data = ecode + 1 + LINK_SIZE; /* Save for matching */ - ecode += GET(ecode, 1); /* Advance past the item */ - - switch (*ecode) - { - case OP_CRSTAR: - case OP_CRMINSTAR: - case OP_CRPLUS: - case OP_CRMINPLUS: - case OP_CRQUERY: - case OP_CRMINQUERY: - c = *ecode++ - OP_CRSTAR; - minimize = (c & 1) != 0; - min = rep_min[c]; /* Pick up values from tables; */ - max = rep_max[c]; /* zero for max => infinity */ - if (max == 0) max = INT_MAX; - break; - - case OP_CRRANGE: - case OP_CRMINRANGE: - minimize = (*ecode == OP_CRMINRANGE); - min = GET2(ecode, 1); - max = GET2(ecode, 3); - if (max == 0) max = INT_MAX; - ecode += 5; - break; - - default: /* No repeat follows */ - min = max = 1; - break; - } - - /* First, ensure the minimum number of matches are present. */ - - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINC(c, eptr); - if (!match_xclass(c, data)) RRETURN(MATCH_NOMATCH); - } - - /* If max == min we can continue with the main loop without the - need to recurse. */ - - if (min == max) continue; - - /* If minimizing, keep testing the rest of the expression and advancing - the pointer while it matches the class. */ - - if (minimize) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINC(c, eptr); - if (!match_xclass(c, data)) RRETURN(MATCH_NOMATCH); - } - /* Control never gets here */ - } - - /* If maximizing, find the longest possible run, then work backwards. */ - - else - { - pp = eptr; - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (!match_xclass(c, data)) break; - eptr += len; - } - for(;;) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ - BACKCHAR(eptr) - } - RRETURN(MATCH_NOMATCH); - } - - /* Control never gets here */ - } -#endif /* End of XCLASS */ - - /* Match a single character, casefully */ - - case OP_CHAR: -#ifdef SUPPORT_UTF8 - if (md->utf8) - { - length = 1; - ecode++; - GETCHARLEN(fc, ecode, length); - if (length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); - while (length-- > 0) if (*ecode++ != *eptr++) RRETURN(MATCH_NOMATCH); - } - else -#endif - - /* Non-UTF-8 mode */ - { - if (md->end_subject - eptr < 1) RRETURN(MATCH_NOMATCH); - if (ecode[1] != *eptr++) RRETURN(MATCH_NOMATCH); - ecode += 2; - } - break; - - /* Match a single character, caselessly */ - - case OP_CHARNC: -#ifdef SUPPORT_UTF8 - if (md->utf8) - { - length = 1; - ecode++; - GETCHARLEN(fc, ecode, length); - - if (length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); - - /* If the pattern character's value is < 128, we have only one byte, and - can use the fast lookup table. */ - - if (fc < 128) - { - if (md->lcc[*ecode++] != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); - } - - /* Otherwise we must pick up the subject character */ - - else - { - int dc; - GETCHARINC(dc, eptr); - ecode += length; - - /* If we have Unicode property support, we can use it to test the other - case of the character, if there is one. The result of ucp_findchar() is - < 0 if the char isn't found, and othercase is returned as zero if there - isn't one. */ - - if (fc != dc) - { -#ifdef SUPPORT_UCP - int chartype; - int othercase; - if (ucp_findchar(fc, &chartype, &othercase) < 0 || dc != othercase) -#endif - RRETURN(MATCH_NOMATCH); - } - } - } - else -#endif /* SUPPORT_UTF8 */ - - /* Non-UTF-8 mode */ - { - if (md->end_subject - eptr < 1) RRETURN(MATCH_NOMATCH); - if (md->lcc[ecode[1]] != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); - ecode += 2; - } - break; - - /* Match a single character repeatedly; different opcodes share code. */ - - case OP_EXACT: - min = max = GET2(ecode, 1); - ecode += 3; - goto REPEATCHAR; - - case OP_UPTO: - case OP_MINUPTO: - min = 0; - max = GET2(ecode, 1); - minimize = *ecode == OP_MINUPTO; - ecode += 3; - goto REPEATCHAR; - - case OP_STAR: - case OP_MINSTAR: - case OP_PLUS: - case OP_MINPLUS: - case OP_QUERY: - case OP_MINQUERY: - c = *ecode++ - OP_STAR; - minimize = (c & 1) != 0; - min = rep_min[c]; /* Pick up values from tables; */ - max = rep_max[c]; /* zero for max => infinity */ - if (max == 0) max = INT_MAX; - - /* Common code for all repeated single-character matches. We can give - up quickly if there are fewer than the minimum number of characters left in - the subject. */ - - REPEATCHAR: -#ifdef SUPPORT_UTF8 - if (md->utf8) - { - length = 1; - charptr = ecode; - GETCHARLEN(fc, ecode, length); - if (min * length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); - ecode += length; - - /* Handle multibyte character matching specially here. There is - support for caseless matching if UCP support is present. */ - - if (length > 1) - { - int oclength = 0; - uschar occhars[8]; - -#ifdef SUPPORT_UCP - int othercase; - int chartype; - if ((ims & PCRE_CASELESS) != 0 && - ucp_findchar(fc, &chartype, &othercase) >= 0 && - othercase > 0) - oclength = ord2utf8(othercase, occhars); -#endif /* SUPPORT_UCP */ - - for (i = 1; i <= min; i++) - { - if (memcmp(eptr, charptr, length) == 0) eptr += length; - /* Need braces because of following else */ - else if (oclength == 0) { RRETURN(MATCH_NOMATCH); } - else - { - if (memcmp(eptr, occhars, oclength) != 0) RRETURN(MATCH_NOMATCH); - eptr += oclength; - } - } - - if (min == max) continue; - - if (minimize) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - if (memcmp(eptr, charptr, length) == 0) eptr += length; - /* Need braces because of following else */ - else if (oclength == 0) { RRETURN(MATCH_NOMATCH); } - else - { - if (memcmp(eptr, occhars, oclength) != 0) RRETURN(MATCH_NOMATCH); - eptr += oclength; - } - } - /* Control never gets here */ - } - else - { - pp = eptr; - for (i = min; i < max; i++) - { - if (eptr > md->end_subject - length) break; - if (memcmp(eptr, charptr, length) == 0) eptr += length; - else if (oclength == 0) break; - else - { - if (memcmp(eptr, occhars, oclength) != 0) break; - eptr += oclength; - } - } - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - eptr -= length; - } - RRETURN(MATCH_NOMATCH); - } - /* Control never gets here */ - } - - /* If the length of a UTF-8 character is 1, we fall through here, and - obey the code as for non-UTF-8 characters below, though in this case the - value of fc will always be < 128. */ - } - else -#endif /* SUPPORT_UTF8 */ - - /* When not in UTF-8 mode, load a single-byte character. */ - { - if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); - fc = *ecode++; - } - - /* The value of fc at this point is always less than 256, though we may or - may not be in UTF-8 mode. The code is duplicated for the caseless and - caseful cases, for speed, since matching characters is likely to be quite - common. First, ensure the minimum number of matches are present. If min = - max, continue at the same level without recursing. Otherwise, if - minimizing, keep trying the rest of the expression and advancing one - matching character if failing, up to the maximum. Alternatively, if - maximizing, find the maximum number of characters and work backwards. */ - - DPRINTF(("matching %c{%d,%d} against subject %.*s\n", fc, min, max, - max, eptr)); - - if ((ims & PCRE_CASELESS) != 0) - { - fc = md->lcc[fc]; - for (i = 1; i <= min; i++) - if (fc != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); - if (min == max) continue; - if (minimize) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || - fc != md->lcc[*eptr++]) - RRETURN(MATCH_NOMATCH); - } - /* Control never gets here */ - } - else - { - pp = eptr; - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || fc != md->lcc[*eptr]) break; - eptr++; - } - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - eptr--; - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - RRETURN(MATCH_NOMATCH); - } - /* Control never gets here */ - } - - /* Caseful comparisons (includes all multi-byte characters) */ - - else - { - for (i = 1; i <= min; i++) if (fc != *eptr++) RRETURN(MATCH_NOMATCH); - if (min == max) continue; - if (minimize) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || fc != *eptr++) - RRETURN(MATCH_NOMATCH); - } - /* Control never gets here */ - } - else - { - pp = eptr; - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || fc != *eptr) break; - eptr++; - } - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - eptr--; - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - RRETURN(MATCH_NOMATCH); - } - } - /* Control never gets here */ - - /* Match a negated single one-byte character. The character we are - checking can be multibyte. */ - - case OP_NOT: - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - ecode++; - GETCHARINCTEST(c, eptr); - if ((ims & PCRE_CASELESS) != 0) - { -#ifdef SUPPORT_UTF8 - if (c < 256) -#endif - c = md->lcc[c]; - if (md->lcc[*ecode++] == c) RRETURN(MATCH_NOMATCH); - } - else - { - if (*ecode++ == c) RRETURN(MATCH_NOMATCH); - } - break; - - /* Match a negated single one-byte character repeatedly. This is almost a - repeat of the code for a repeated single character, but I haven't found a - nice way of commoning these up that doesn't require a test of the - positive/negative option for each character match. Maybe that wouldn't add - very much to the time taken, but character matching *is* what this is all - about... */ - - case OP_NOTEXACT: - min = max = GET2(ecode, 1); - ecode += 3; - goto REPEATNOTCHAR; - - case OP_NOTUPTO: - case OP_NOTMINUPTO: - min = 0; - max = GET2(ecode, 1); - minimize = *ecode == OP_NOTMINUPTO; - ecode += 3; - goto REPEATNOTCHAR; - - case OP_NOTSTAR: - case OP_NOTMINSTAR: - case OP_NOTPLUS: - case OP_NOTMINPLUS: - case OP_NOTQUERY: - case OP_NOTMINQUERY: - c = *ecode++ - OP_NOTSTAR; - minimize = (c & 1) != 0; - min = rep_min[c]; /* Pick up values from tables; */ - max = rep_max[c]; /* zero for max => infinity */ - if (max == 0) max = INT_MAX; - - /* Common code for all repeated single-byte matches. We can give up quickly - if there are fewer than the minimum number of bytes left in the - subject. */ - - REPEATNOTCHAR: - if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); - fc = *ecode++; - - /* The code is duplicated for the caseless and caseful cases, for speed, - since matching characters is likely to be quite common. First, ensure the - minimum number of matches are present. If min = max, continue at the same - level without recursing. Otherwise, if minimizing, keep trying the rest of - the expression and advancing one matching character if failing, up to the - maximum. Alternatively, if maximizing, find the maximum number of - characters and work backwards. */ - - DPRINTF(("negative matching %c{%d,%d} against subject %.*s\n", fc, min, max, - max, eptr)); - - if ((ims & PCRE_CASELESS) != 0) - { - fc = md->lcc[fc]; - -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - register int d; - for (i = 1; i <= min; i++) - { - GETCHARINC(d, eptr); - if (d < 256) d = md->lcc[d]; - if (fc == d) RRETURN(MATCH_NOMATCH); - } - } - else -#endif - - /* Not UTF-8 mode */ - { - for (i = 1; i <= min; i++) - if (fc == md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); - } - - if (min == max) continue; - - if (minimize) - { -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - register int d; - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - GETCHARINC(d, eptr); - if (d < 256) d = md->lcc[d]; - if (fi >= max || eptr >= md->end_subject || fc == d) - RRETURN(MATCH_NOMATCH); - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || fc == md->lcc[*eptr++]) - RRETURN(MATCH_NOMATCH); - } - } - /* Control never gets here */ - } - - /* Maximize case */ - - else - { - pp = eptr; - -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - register int d; - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(d, eptr, len); - if (d < 256) d = md->lcc[d]; - if (fc == d) break; - eptr += len; - } - for(;;) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ - BACKCHAR(eptr); - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || fc == md->lcc[*eptr]) break; - eptr++; - } - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - eptr--; - } - } - - RRETURN(MATCH_NOMATCH); - } - /* Control never gets here */ - } - - /* Caseful comparisons */ - - else - { -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - register int d; - for (i = 1; i <= min; i++) - { - GETCHARINC(d, eptr); - if (fc == d) RRETURN(MATCH_NOMATCH); - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (i = 1; i <= min; i++) - if (fc == *eptr++) RRETURN(MATCH_NOMATCH); - } - - if (min == max) continue; - - if (minimize) - { -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - register int d; - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - GETCHARINC(d, eptr); - if (fi >= max || eptr >= md->end_subject || fc == d) - RRETURN(MATCH_NOMATCH); - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject || fc == *eptr++) - RRETURN(MATCH_NOMATCH); - } - } - /* Control never gets here */ - } - - /* Maximize case */ - - else - { - pp = eptr; - -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - register int d; - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(d, eptr, len); - if (fc == d) break; - eptr += len; - } - for(;;) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ - BACKCHAR(eptr); - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || fc == *eptr) break; - eptr++; - } - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - eptr--; - } - } - - RRETURN(MATCH_NOMATCH); - } - } - /* Control never gets here */ - - /* Match a single character type repeatedly; several different opcodes - share code. This is very similar to the code for single characters, but we - repeat it in the interests of efficiency. */ - - case OP_TYPEEXACT: - min = max = GET2(ecode, 1); - minimize = TRUE; - ecode += 3; - goto REPEATTYPE; - - case OP_TYPEUPTO: - case OP_TYPEMINUPTO: - min = 0; - max = GET2(ecode, 1); - minimize = *ecode == OP_TYPEMINUPTO; - ecode += 3; - goto REPEATTYPE; - - case OP_TYPESTAR: - case OP_TYPEMINSTAR: - case OP_TYPEPLUS: - case OP_TYPEMINPLUS: - case OP_TYPEQUERY: - case OP_TYPEMINQUERY: - c = *ecode++ - OP_TYPESTAR; - minimize = (c & 1) != 0; - min = rep_min[c]; /* Pick up values from tables; */ - max = rep_max[c]; /* zero for max => infinity */ - if (max == 0) max = INT_MAX; - - /* Common code for all repeated single character type matches. Note that - in UTF-8 mode, '.' matches a character of any length, but for the other - character types, the valid characters are all one-byte long. */ - - REPEATTYPE: - ctype = *ecode++; /* Code for the character type */ - -#ifdef SUPPORT_UCP - if (ctype == OP_PROP || ctype == OP_NOTPROP) - { - prop_fail_result = ctype == OP_NOTPROP; - prop_type = *ecode++; - if (prop_type >= 128) - { - prop_test_against = prop_type - 128; - prop_test_variable = &prop_category; - } - else - { - prop_test_against = prop_type; - prop_test_variable = &prop_chartype; - } - } - else prop_type = -1; -#endif - - /* First, ensure the minimum number of matches are present. Use inline - code for maximizing the speed, and do the type test once at the start - (i.e. keep it out of the loop). Also we can test that there are at least - the minimum number of bytes before we start. This isn't as effective in - UTF-8 mode, but it does no harm. Separate the UTF-8 code completely as that - is tidier. Also separate the UCP code, which can be the same for both UTF-8 - and single-bytes. */ - - if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); - if (min > 0) - { -#ifdef SUPPORT_UCP - if (prop_type > 0) - { - for (i = 1; i <= min; i++) - { - GETCHARINC(c, eptr); - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if ((*prop_test_variable == prop_test_against) == prop_fail_result) - RRETURN(MATCH_NOMATCH); - } - } - - /* Match extended Unicode sequences. We will get here only if the - support is in the binary; otherwise a compile-time error occurs. */ - - else if (ctype == OP_EXTUNI) - { - for (i = 1; i <= min; i++) - { - GETCHARINCTEST(c, eptr); - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if (prop_category == ucp_M) RRETURN(MATCH_NOMATCH); - while (eptr < md->end_subject) - { - int len = 1; - if (!md->utf8) c = *eptr; else - { - GETCHARLEN(c, eptr, len); - } - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if (prop_category != ucp_M) break; - eptr += len; - } - } - } - - else -#endif /* SUPPORT_UCP */ - -/* Handle all other cases when the coding is UTF-8 */ - -#ifdef SUPPORT_UTF8 - if (md->utf8) switch(ctype) - { - case OP_ANY: - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject || - (*eptr++ == NEWLINE && (ims & PCRE_DOTALL) == 0)) - RRETURN(MATCH_NOMATCH); - while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; - } - break; - - case OP_ANYBYTE: - eptr += min; - break; - - case OP_NOT_DIGIT: - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINC(c, eptr); - if (c < 128 && (md->ctypes[c] & ctype_digit) != 0) - RRETURN(MATCH_NOMATCH); - } - break; - - case OP_DIGIT: - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject || - *eptr >= 128 || (md->ctypes[*eptr++] & ctype_digit) == 0) - RRETURN(MATCH_NOMATCH); - /* No need to skip more bytes - we know it's a 1-byte character */ - } - break; - - case OP_NOT_WHITESPACE: - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject || - (*eptr < 128 && (md->ctypes[*eptr++] & ctype_space) != 0)) - RRETURN(MATCH_NOMATCH); - while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; - } - break; - - case OP_WHITESPACE: - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject || - *eptr >= 128 || (md->ctypes[*eptr++] & ctype_space) == 0) - RRETURN(MATCH_NOMATCH); - /* No need to skip more bytes - we know it's a 1-byte character */ - } - break; - - case OP_NOT_WORDCHAR: - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject || - (*eptr < 128 && (md->ctypes[*eptr++] & ctype_word) != 0)) - RRETURN(MATCH_NOMATCH); - while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; - } - break; - - case OP_WORDCHAR: - for (i = 1; i <= min; i++) - { - if (eptr >= md->end_subject || - *eptr >= 128 || (md->ctypes[*eptr++] & ctype_word) == 0) - RRETURN(MATCH_NOMATCH); - /* No need to skip more bytes - we know it's a 1-byte character */ - } - break; - - default: - RRETURN(PCRE_ERROR_INTERNAL); - } /* End switch(ctype) */ - - else -#endif /* SUPPORT_UTF8 */ - - /* Code for the non-UTF-8 case for minimum matching of operators other - than OP_PROP and OP_NOTPROP. */ - - switch(ctype) - { - case OP_ANY: - if ((ims & PCRE_DOTALL) == 0) - { - for (i = 1; i <= min; i++) - if (*eptr++ == NEWLINE) RRETURN(MATCH_NOMATCH); - } - else eptr += min; - break; - - case OP_ANYBYTE: - eptr += min; - break; - - case OP_NOT_DIGIT: - for (i = 1; i <= min; i++) - if ((md->ctypes[*eptr++] & ctype_digit) != 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_DIGIT: - for (i = 1; i <= min; i++) - if ((md->ctypes[*eptr++] & ctype_digit) == 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_NOT_WHITESPACE: - for (i = 1; i <= min; i++) - if ((md->ctypes[*eptr++] & ctype_space) != 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_WHITESPACE: - for (i = 1; i <= min; i++) - if ((md->ctypes[*eptr++] & ctype_space) == 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_NOT_WORDCHAR: - for (i = 1; i <= min; i++) - if ((md->ctypes[*eptr++] & ctype_word) != 0) - RRETURN(MATCH_NOMATCH); - break; - - case OP_WORDCHAR: - for (i = 1; i <= min; i++) - if ((md->ctypes[*eptr++] & ctype_word) == 0) - RRETURN(MATCH_NOMATCH); - break; - - default: - RRETURN(PCRE_ERROR_INTERNAL); - } - } - - /* If min = max, continue at the same level without recursing */ - - if (min == max) continue; - - /* If minimizing, we have to test the rest of the pattern before each - subsequent match. Again, separate the UTF-8 case for speed, and also - separate the UCP cases. */ - - if (minimize) - { -#ifdef SUPPORT_UCP - if (prop_type > 0) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINC(c, eptr); - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if ((*prop_test_variable == prop_test_against) == prop_fail_result) - RRETURN(MATCH_NOMATCH); - } - } - - /* Match extended Unicode sequences. We will get here only if the - support is in the binary; otherwise a compile-time error occurs. */ - - else if (ctype == OP_EXTUNI) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - GETCHARINCTEST(c, eptr); - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if (prop_category == ucp_M) RRETURN(MATCH_NOMATCH); - while (eptr < md->end_subject) - { - int len = 1; - if (!md->utf8) c = *eptr; else - { - GETCHARLEN(c, eptr, len); - } - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if (prop_category != ucp_M) break; - eptr += len; - } - } - } - - else -#endif /* SUPPORT_UCP */ - -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - if (md->utf8) - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - - GETCHARINC(c, eptr); - switch(ctype) - { - case OP_ANY: - if ((ims & PCRE_DOTALL) == 0 && c == NEWLINE) RRETURN(MATCH_NOMATCH); - break; - - case OP_ANYBYTE: - break; - - case OP_NOT_DIGIT: - if (c < 256 && (md->ctypes[c] & ctype_digit) != 0) - RRETURN(MATCH_NOMATCH); - break; - - case OP_DIGIT: - if (c >= 256 || (md->ctypes[c] & ctype_digit) == 0) - RRETURN(MATCH_NOMATCH); - break; - - case OP_NOT_WHITESPACE: - if (c < 256 && (md->ctypes[c] & ctype_space) != 0) - RRETURN(MATCH_NOMATCH); - break; - - case OP_WHITESPACE: - if (c >= 256 || (md->ctypes[c] & ctype_space) == 0) - RRETURN(MATCH_NOMATCH); - break; - - case OP_NOT_WORDCHAR: - if (c < 256 && (md->ctypes[c] & ctype_word) != 0) - RRETURN(MATCH_NOMATCH); - break; - - case OP_WORDCHAR: - if (c >= 256 && (md->ctypes[c] & ctype_word) == 0) - RRETURN(MATCH_NOMATCH); - break; - - default: - RRETURN(PCRE_ERROR_INTERNAL); - } - } - } - else -#endif - /* Not UTF-8 mode */ - { - for (fi = min;; fi++) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); - c = *eptr++; - switch(ctype) - { - case OP_ANY: - if ((ims & PCRE_DOTALL) == 0 && c == NEWLINE) RRETURN(MATCH_NOMATCH); - break; - - case OP_ANYBYTE: - break; - - case OP_NOT_DIGIT: - if ((md->ctypes[c] & ctype_digit) != 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_DIGIT: - if ((md->ctypes[c] & ctype_digit) == 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_NOT_WHITESPACE: - if ((md->ctypes[c] & ctype_space) != 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_WHITESPACE: - if ((md->ctypes[c] & ctype_space) == 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_NOT_WORDCHAR: - if ((md->ctypes[c] & ctype_word) != 0) RRETURN(MATCH_NOMATCH); - break; - - case OP_WORDCHAR: - if ((md->ctypes[c] & ctype_word) == 0) RRETURN(MATCH_NOMATCH); - break; - - default: - RRETURN(PCRE_ERROR_INTERNAL); - } - } - } - /* Control never gets here */ - } - - /* If maximizing it is worth using inline code for speed, doing the type - test once at the start (i.e. keep it out of the loop). Again, keep the - UTF-8 and UCP stuff separate. */ - - else - { - pp = eptr; /* Remember where we started */ - -#ifdef SUPPORT_UCP - if (prop_type > 0) - { - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if ((*prop_test_variable == prop_test_against) == prop_fail_result) - break; - eptr+= len; - } - - /* eptr is now past the end of the maximum run */ - - for(;;) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ - BACKCHAR(eptr); - } - } - - /* Match extended Unicode sequences. We will get here only if the - support is in the binary; otherwise a compile-time error occurs. */ - - else if (ctype == OP_EXTUNI) - { - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject) break; - GETCHARINCTEST(c, eptr); - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if (prop_category == ucp_M) break; - while (eptr < md->end_subject) - { - int len = 1; - if (!md->utf8) c = *eptr; else - { - GETCHARLEN(c, eptr, len); - } - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if (prop_category != ucp_M) break; - eptr += len; - } - } - - /* eptr is now past the end of the maximum run */ - - for(;;) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ - for (;;) /* Move back over one extended */ - { - int len = 1; - BACKCHAR(eptr); - if (!md->utf8) c = *eptr; else - { - GETCHARLEN(c, eptr, len); - } - prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); - if (prop_category != ucp_M) break; - eptr--; - } - } - } - - else -#endif /* SUPPORT_UCP */ - -#ifdef SUPPORT_UTF8 - /* UTF-8 mode */ - - if (md->utf8) - { - switch(ctype) - { - case OP_ANY: - - /* Special code is required for UTF8, but when the maximum is unlimited - we don't need it, so we repeat the non-UTF8 code. This is probably - worth it, because .* is quite a common idiom. */ - - if (max < INT_MAX) - { - if ((ims & PCRE_DOTALL) == 0) - { - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || *eptr == NEWLINE) break; - eptr++; - while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; - } - } - else - { - for (i = min; i < max; i++) - { - eptr++; - while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; - } - } - } - - /* Handle unlimited UTF-8 repeat */ - - else - { - if ((ims & PCRE_DOTALL) == 0) - { - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || *eptr == NEWLINE) break; - eptr++; - } - break; - } - else - { - c = max - min; - if (c > md->end_subject - eptr) c = md->end_subject - eptr; - eptr += c; - } - } - break; - - /* The byte case is the same as non-UTF8 */ - - case OP_ANYBYTE: - c = max - min; - if (c > md->end_subject - eptr) c = md->end_subject - eptr; - eptr += c; - break; - - case OP_NOT_DIGIT: - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (c < 256 && (md->ctypes[c] & ctype_digit) != 0) break; - eptr+= len; - } - break; - - case OP_DIGIT: - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (c >= 256 ||(md->ctypes[c] & ctype_digit) == 0) break; - eptr+= len; - } - break; - - case OP_NOT_WHITESPACE: - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (c < 256 && (md->ctypes[c] & ctype_space) != 0) break; - eptr+= len; - } - break; - - case OP_WHITESPACE: - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (c >= 256 ||(md->ctypes[c] & ctype_space) == 0) break; - eptr+= len; - } - break; - - case OP_NOT_WORDCHAR: - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (c < 256 && (md->ctypes[c] & ctype_word) != 0) break; - eptr+= len; - } - break; - - case OP_WORDCHAR: - for (i = min; i < max; i++) - { - int len = 1; - if (eptr >= md->end_subject) break; - GETCHARLEN(c, eptr, len); - if (c >= 256 || (md->ctypes[c] & ctype_word) == 0) break; - eptr+= len; - } - break; - - default: - RRETURN(PCRE_ERROR_INTERNAL); - } - - /* eptr is now past the end of the maximum run */ - - for(;;) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - if (eptr-- == pp) break; /* Stop if tried at original pos */ - BACKCHAR(eptr); - } - } - else -#endif - - /* Not UTF-8 mode */ - { - switch(ctype) - { - case OP_ANY: - if ((ims & PCRE_DOTALL) == 0) - { - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || *eptr == NEWLINE) break; - eptr++; - } - break; - } - /* For DOTALL case, fall through and treat as \C */ - - case OP_ANYBYTE: - c = max - min; - if (c > md->end_subject - eptr) c = md->end_subject - eptr; - eptr += c; - break; - - case OP_NOT_DIGIT: - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_digit) != 0) - break; - eptr++; - } - break; - - case OP_DIGIT: - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_digit) == 0) - break; - eptr++; - } - break; - - case OP_NOT_WHITESPACE: - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_space) != 0) - break; - eptr++; - } - break; - - case OP_WHITESPACE: - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_space) == 0) - break; - eptr++; - } - break; - - case OP_NOT_WORDCHAR: - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_word) != 0) - break; - eptr++; - } - break; - - case OP_WORDCHAR: - for (i = min; i < max; i++) - { - if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_word) == 0) - break; - eptr++; - } - break; - - default: - RRETURN(PCRE_ERROR_INTERNAL); - } - - /* eptr is now past the end of the maximum run */ - - while (eptr >= pp) - { - RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); - eptr--; - if (rrc != MATCH_NOMATCH) RRETURN(rrc); - } - } - - /* Get here if we can't make it match with any permitted repetitions */ - - RRETURN(MATCH_NOMATCH); - } - /* Control never gets here */ - - /* There's been some horrible disaster. Since all codes > OP_BRA are - for capturing brackets, and there shouldn't be any gaps between 0 and - OP_BRA, arrival here can only mean there is something seriously wrong - in the code above or the OP_xxx definitions. */ - - default: - DPRINTF(("Unknown opcode %d\n", *ecode)); - RRETURN(PCRE_ERROR_UNKNOWN_NODE); - } - - /* Do not stick any code in here without much thought; it is assumed - that "continue" in the code above comes out to here to repeat the main - loop. */ - - } /* End of main loop */ -/* Control never reaches here */ -} - - -/*************************************************************************** -**************************************************************************** - RECURSION IN THE match() FUNCTION - -Undefine all the macros that were defined above to handle this. */ - -#ifdef NO_RECURSE -#undef eptr -#undef ecode -#undef offset_top -#undef ims -#undef eptrb -#undef flags - -#undef callpat -#undef charptr -#undef data -#undef next -#undef pp -#undef prev -#undef saved_eptr - -#undef new_recursive - -#undef cur_is_word -#undef condition -#undef minimize -#undef prev_is_word - -#undef original_ims - -#undef ctype -#undef length -#undef max -#undef min -#undef number -#undef offset -#undef op -#undef save_capture_last -#undef save_offset1 -#undef save_offset2 -#undef save_offset3 -#undef stacksave - -#undef newptrb - -#endif - -/* These two are defined as macros in both cases */ - -#undef fc -#undef fi - -/*************************************************************************** -***************************************************************************/ - - - -/************************************************* -* Execute a Regular Expression * -*************************************************/ - -/* This function applies a compiled re to a subject string and picks out -portions of the string if it matches. Two elements in the vector are set for -each substring: the offsets to the start and end of the substring. - -Arguments: - argument_re points to the compiled expression - extra_data points to extra data or is NULL - subject points to the subject string - length length of subject string (may contain binary zeros) - start_offset where to start in the subject string - options option bits - offsets points to a vector of ints to be filled in with offsets - offsetcount the number of elements in the vector - -Returns: > 0 => success; value is the number of elements filled in - = 0 => success, but offsets is not big enough - -1 => failed to match - < -1 => some kind of unexpected problem -*/ - -EXPORT int -pcre_exec(const pcre *argument_re, const pcre_extra *extra_data, - const char *subject, int length, int start_offset, int options, int *offsets, - int offsetcount) -{ -int rc, resetcount, ocount; -int first_byte = -1; -int req_byte = -1; -int req_byte2 = -1; -unsigned long int ims = 0; -BOOL using_temporary_offsets = FALSE; -BOOL anchored; -BOOL startline; -BOOL first_byte_caseless = FALSE; -BOOL req_byte_caseless = FALSE; -match_data match_block; -const uschar *tables; -const uschar *start_bits = NULL; -const uschar *start_match = (const uschar *)subject + start_offset; -const uschar *end_subject; -const uschar *req_byte_ptr = start_match - 1; - -pcre_study_data internal_study; -const pcre_study_data *study; - -real_pcre internal_re; -const real_pcre *external_re = (const real_pcre *)argument_re; -const real_pcre *re = external_re; - -/* Plausibility checks */ - -if ((options & ~PUBLIC_EXEC_OPTIONS) != 0) return PCRE_ERROR_BADOPTION; -if (re == NULL || subject == NULL || - (offsets == NULL && offsetcount > 0)) return PCRE_ERROR_NULL; -if (offsetcount < 0) return PCRE_ERROR_BADCOUNT; - -/* Fish out the optional data from the extra_data structure, first setting -the default values. */ - -study = NULL; -match_block.match_limit = MATCH_LIMIT; -match_block.callout_data = NULL; - -/* The table pointer is always in native byte order. */ - -tables = external_re->tables; - -if (extra_data != NULL) - { - register unsigned int flags = extra_data->flags; - if ((flags & PCRE_EXTRA_STUDY_DATA) != 0) - study = (const pcre_study_data *)extra_data->study_data; - if ((flags & PCRE_EXTRA_MATCH_LIMIT) != 0) - match_block.match_limit = extra_data->match_limit; - if ((flags & PCRE_EXTRA_CALLOUT_DATA) != 0) - match_block.callout_data = extra_data->callout_data; - if ((flags & PCRE_EXTRA_TABLES) != 0) tables = extra_data->tables; - } - -/* If the exec call supplied NULL for tables, use the inbuilt ones. This -is a feature that makes it possible to save compiled regex and re-use them -in other programs later. */ - -if (tables == NULL) tables = pcre_default_tables; - -/* Check that the first field in the block is the magic number. If it is not, -test for a regex that was compiled on a host of opposite endianness. If this is -the case, flipped values are put in internal_re and internal_study if there was -study data too. */ - -if (re->magic_number != MAGIC_NUMBER) - { - re = try_flipped(re, &internal_re, study, &internal_study); - if (re == NULL) return PCRE_ERROR_BADMAGIC; - if (study != NULL) study = &internal_study; - } - -/* Set up other data */ - -anchored = ((re->options | options) & PCRE_ANCHORED) != 0; -startline = (re->options & PCRE_STARTLINE) != 0; - -/* The code starts after the real_pcre block and the capture name table. */ - -match_block.start_code = (const uschar *)external_re + re->name_table_offset + - re->name_count * re->name_entry_size; - -match_block.start_subject = (const uschar *)subject; -match_block.start_offset = start_offset; -match_block.end_subject = match_block.start_subject + length; -end_subject = match_block.end_subject; - -match_block.endonly = (re->options & PCRE_DOLLAR_ENDONLY) != 0; -match_block.utf8 = (re->options & PCRE_UTF8) != 0; - -match_block.notbol = (options & PCRE_NOTBOL) != 0; -match_block.noteol = (options & PCRE_NOTEOL) != 0; -match_block.notempty = (options & PCRE_NOTEMPTY) != 0; -match_block.partial = (options & PCRE_PARTIAL) != 0; -match_block.hitend = FALSE; - -match_block.recursive = NULL; /* No recursion at top level */ - -match_block.lcc = tables + lcc_offset; -match_block.ctypes = tables + ctypes_offset; - -/* Partial matching is supported only for a restricted set of regexes at the -moment. */ - -if (match_block.partial && (re->options & PCRE_NOPARTIAL) != 0) - return PCRE_ERROR_BADPARTIAL; - -/* Check a UTF-8 string if required. Unfortunately there's no way of passing -back the character offset. */ - -#ifdef SUPPORT_UTF8 -if (match_block.utf8 && (options & PCRE_NO_UTF8_CHECK) == 0) - { - if (valid_utf8((uschar *)subject, length) >= 0) - return PCRE_ERROR_BADUTF8; - if (start_offset > 0 && start_offset < length) - { - int tb = ((uschar *)subject)[start_offset]; - if (tb > 127) - { - tb &= 0xc0; - if (tb != 0 && tb != 0xc0) return PCRE_ERROR_BADUTF8_OFFSET; - } - } - } -#endif - -/* The ims options can vary during the matching as a result of the presence -of (?ims) items in the pattern. They are kept in a local variable so that -restoring at the exit of a group is easy. */ - -ims = re->options & (PCRE_CASELESS|PCRE_MULTILINE|PCRE_DOTALL); - -/* If the expression has got more back references than the offsets supplied can -hold, we get a temporary chunk of working store to use during the matching. -Otherwise, we can use the vector supplied, rounding down its size to a multiple -of 3. */ - -ocount = offsetcount - (offsetcount % 3); - -if (re->top_backref > 0 && re->top_backref >= ocount/3) - { - ocount = re->top_backref * 3 + 3; - match_block.offset_vector = (int *)(pcre_malloc)(ocount * sizeof(int)); - if (match_block.offset_vector == NULL) return PCRE_ERROR_NOMEMORY; - using_temporary_offsets = TRUE; - DPRINTF(("Got memory to hold back references\n")); - } -else match_block.offset_vector = offsets; - -match_block.offset_end = ocount; -match_block.offset_max = (2*ocount)/3; -match_block.offset_overflow = FALSE; -match_block.capture_last = -1; - -/* Compute the minimum number of offsets that we need to reset each time. Doing -this makes a huge difference to execution time when there aren't many brackets -in the pattern. */ - -resetcount = 2 + re->top_bracket * 2; -if (resetcount > offsetcount) resetcount = ocount; - -/* Reset the working variable associated with each extraction. These should -never be used unless previously set, but they get saved and restored, and so we -initialize them to avoid reading uninitialized locations. */ - -if (match_block.offset_vector != NULL) - { - register int *iptr = match_block.offset_vector + ocount; - register int *iend = iptr - resetcount/2 + 1; - while (--iptr >= iend) *iptr = -1; - } - -/* Set up the first character to match, if available. The first_byte value is -never set for an anchored regular expression, but the anchoring may be forced -at run time, so we have to test for anchoring. The first char may be unset for -an unanchored pattern, of course. If there's no first char and the pattern was -studied, there may be a bitmap of possible first characters. */ - -if (!anchored) - { - if ((re->options & PCRE_FIRSTSET) != 0) - { - first_byte = re->first_byte & 255; - if ((first_byte_caseless = ((re->first_byte & REQ_CASELESS) != 0)) == TRUE) - first_byte = match_block.lcc[first_byte]; - } - else - if (!startline && study != NULL && - (study->options & PCRE_STUDY_MAPPED) != 0) - start_bits = study->start_bits; - } - -/* For anchored or unanchored matches, there may be a "last known required -character" set. */ - -if ((re->options & PCRE_REQCHSET) != 0) - { - req_byte = re->req_byte & 255; - req_byte_caseless = (re->req_byte & REQ_CASELESS) != 0; - req_byte2 = (tables + fcc_offset)[req_byte]; /* case flipped */ - } - -/* Loop for handling unanchored repeated matching attempts; for anchored regexs -the loop runs just once. */ - -do - { - /* Reset the maximum number of extractions we might see. */ - - if (match_block.offset_vector != NULL) - { - register int *iptr = match_block.offset_vector; - register int *iend = iptr + resetcount; - while (iptr < iend) *iptr++ = -1; - } - - /* Advance to a unique first char if possible */ - - if (first_byte >= 0) - { - if (first_byte_caseless) - while (start_match < end_subject && - match_block.lcc[*start_match] != first_byte) - start_match++; - else - while (start_match < end_subject && *start_match != first_byte) - start_match++; - } - - /* Or to just after \n for a multiline match if possible */ - - else if (startline) - { - if (start_match > match_block.start_subject + start_offset) - { - while (start_match < end_subject && start_match[-1] != NEWLINE) - start_match++; - } - } - - /* Or to a non-unique first char after study */ - - else if (start_bits != NULL) - { - while (start_match < end_subject) - { - register unsigned int c = *start_match; - if ((start_bits[c/8] & (1 << (c&7))) == 0) start_match++; else break; - } - } - -#ifdef DEBUG /* Sigh. Some compilers never learn. */ - printf(">>>> Match against: "); - pchars(start_match, end_subject - start_match, TRUE, &match_block); - printf("\n"); -#endif - - /* If req_byte is set, we know that that character must appear in the subject - for the match to succeed. If the first character is set, req_byte must be - later in the subject; otherwise the test starts at the match point. This - optimization can save a huge amount of backtracking in patterns with nested - unlimited repeats that aren't going to match. Writing separate code for - cased/caseless versions makes it go faster, as does using an autoincrement - and backing off on a match. - - HOWEVER: when the subject string is very, very long, searching to its end can - take a long time, and give bad performance on quite ordinary patterns. This - showed up when somebody was matching /^C/ on a 32-megabyte string... so we - don't do this when the string is sufficiently long. - - ALSO: this processing is disabled when partial matching is requested. - */ - - if (req_byte >= 0 && - end_subject - start_match < REQ_BYTE_MAX && - !match_block.partial) - { - register const uschar *p = start_match + ((first_byte >= 0)? 1 : 0); - - /* We don't need to repeat the search if we haven't yet reached the - place we found it at last time. */ - - if (p > req_byte_ptr) - { - if (req_byte_caseless) - { - while (p < end_subject) - { - register int pp = *p++; - if (pp == req_byte || pp == req_byte2) { p--; break; } - } - } - else - { - while (p < end_subject) - { - if (*p++ == req_byte) { p--; break; } - } - } - - /* If we can't find the required character, break the matching loop */ - - if (p >= end_subject) break; - - /* If we have found the required character, save the point where we - found it, so that we don't search again next time round the loop if - the start hasn't passed this character yet. */ - - req_byte_ptr = p; - } - } - - /* When a match occurs, substrings will be set for all internal extractions; - we just need to set up the whole thing as substring 0 before returning. If - there were too many extractions, set the return code to zero. In the case - where we had to get some local store to hold offsets for backreferences, copy - those back references that we can. In this case there need not be overflow - if certain parts of the pattern were not used. */ - - match_block.start_match = start_match; - match_block.match_call_count = 0; - - rc = match(start_match, match_block.start_code, 2, &match_block, ims, NULL, - match_isgroup); - - if (rc == MATCH_NOMATCH) - { - start_match++; -#ifdef SUPPORT_UTF8 - if (match_block.utf8) - while(start_match < end_subject && (*start_match & 0xc0) == 0x80) - start_match++; -#endif - continue; - } - - if (rc != MATCH_MATCH) - { - DPRINTF((">>>> error: returning %d\n", rc)); - return rc; - } - - /* We have a match! Copy the offset information from temporary store if - necessary */ - - if (using_temporary_offsets) - { - if (offsetcount >= 4) - { - memcpy(offsets + 2, match_block.offset_vector + 2, - (offsetcount - 2) * sizeof(int)); - DPRINTF(("Copied offsets from temporary memory\n")); - } - if (match_block.end_offset_top > offsetcount) - match_block.offset_overflow = TRUE; - - DPRINTF(("Freeing temporary memory\n")); - (pcre_free)(match_block.offset_vector); - } - - rc = match_block.offset_overflow? 0 : match_block.end_offset_top/2; - - if (offsetcount < 2) rc = 0; else - { - offsets[0] = start_match - match_block.start_subject; - offsets[1] = match_block.end_match_ptr - match_block.start_subject; - } - - DPRINTF((">>>> returning %d\n", rc)); - return rc; - } - -/* This "while" is the end of the "do" above */ - -while (!anchored && start_match <= end_subject); - -if (using_temporary_offsets) - { - DPRINTF(("Freeing temporary memory\n")); - (pcre_free)(match_block.offset_vector); - } - -if (match_block.partial && match_block.hitend) - { - DPRINTF((">>>> returning PCRE_ERROR_PARTIAL\n")); - return PCRE_ERROR_PARTIAL; - } -else - { - DPRINTF((">>>> returning PCRE_ERROR_NOMATCH\n")); - return PCRE_ERROR_NOMATCH; - } -} - -/* End of pcre.c */ +/* End of pcre_compile.c */ diff --git a/pcre_config.c b/pcre_config.c new file mode 100644 index 0000000..04029a9 --- /dev/null +++ b/pcre_config.c @@ -0,0 +1,112 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains the external function pcre_config(). */ + + +#include "pcre_internal.h" + + +/************************************************* +* Return info about what features are configured * +*************************************************/ + +/* This function has an extensible interface so that additional items can be +added compatibly. + +Arguments: + what what information is required + where where to put the information + +Returns: 0 if data returned, negative on error +*/ + +EXPORT int +pcre_config(int what, void *where) +{ +switch (what) + { + case PCRE_CONFIG_UTF8: +#ifdef SUPPORT_UTF8 + *((int *)where) = 1; +#else + *((int *)where) = 0; +#endif + break; + + case PCRE_CONFIG_UNICODE_PROPERTIES: +#ifdef SUPPORT_UCP + *((int *)where) = 1; +#else + *((int *)where) = 0; +#endif + break; + + case PCRE_CONFIG_NEWLINE: + *((int *)where) = NEWLINE; + break; + + case PCRE_CONFIG_LINK_SIZE: + *((int *)where) = LINK_SIZE; + break; + + case PCRE_CONFIG_POSIX_MALLOC_THRESHOLD: + *((int *)where) = POSIX_MALLOC_THRESHOLD; + break; + + case PCRE_CONFIG_MATCH_LIMIT: + *((unsigned int *)where) = MATCH_LIMIT; + break; + + case PCRE_CONFIG_STACKRECURSE: +#ifdef NO_RECURSE + *((int *)where) = 0; +#else + *((int *)where) = 1; +#endif + break; + + default: return PCRE_ERROR_BADOPTION; + } + +return 0; +} + +/* End of pcre_config.c */ diff --git a/pcre_dfa_exec.c b/pcre_dfa_exec.c new file mode 100644 index 0000000..7101570 --- /dev/null +++ b/pcre_dfa_exec.c @@ -0,0 +1,1922 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains the external function pcre_dfa_exec(), which is an +alternative matching function that uses a DFA algorithm. This is NOT Perl- +compatible, but it has advantages in certain applications. */ + + +#include "pcre_internal.h" + + +/* For use to indent debugging output */ + +#define SP " " + + + +/************************************************* +* Code parameters and static tables * +*************************************************/ + +/* These are offsets that are used to turn the OP_TYPESTAR and friends opcodes +into others, under special conditions. A gap of 10 between the blocks should be +enough. */ + +#define OP_PROP_EXTRA (EXTRACT_BASIC_MAX+1) +#define OP_EXTUNI_EXTRA (EXTRACT_BASIC_MAX+11) + + +/* This table identifies those opcodes that are followed immediately by a +character that is to be tested in some way. This makes is possible to +centralize the loading of these characters. In the case of Type * etc, the +"character" is the opcode for \D, \d, \S, \s, \W, or \w, which will always be a +small value. */ + +static uschar coptable[] = { + 0, /* End */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* \A, \G, \B, \b, \D, \d, \S, \s, \W, \w */ + 0, 0, /* Any, Anybyte */ + 0, 0, 0, /* NOTPROP, PROP, EXTUNI */ + 0, 0, 0, 0, 0, /* \Z, \z, Opt, ^, $ */ + 1, /* Char */ + 1, /* Charnc */ + 1, /* not */ + /* Positive single-char repeats */ + 1, 1, 1, 1, 1, 1, /* *, *?, +, +?, ?, ?? */ + 3, 3, 3, /* upto, minupto, exact */ + /* Negative single-char repeats - only for chars < 256 */ + 1, 1, 1, 1, 1, 1, /* NOT *, *?, +, +?, ?, ?? */ + 3, 3, 3, /* NOT upto, minupto, exact */ + /* Positive type repeats */ + 1, 1, 1, 1, 1, 1, /* Type *, *?, +, +?, ?, ?? */ + 3, 3, 3, /* Type upto, minupto, exact */ + /* Character class & ref repeats */ + 0, 0, 0, 0, 0, 0, /* *, *?, +, +?, ?, ?? */ + 0, 0, /* CRRANGE, CRMINRANGE */ + 0, /* CLASS */ + 0, /* NCLASS */ + 0, /* XCLASS - variable length */ + 0, /* REF */ + 0, /* RECURSE */ + 0, /* CALLOUT */ + 0, /* Alt */ + 0, /* Ket */ + 0, /* KetRmax */ + 0, /* KetRmin */ + 0, /* Assert */ + 0, /* Assert not */ + 0, /* Assert behind */ + 0, /* Assert behind not */ + 0, /* Reverse */ + 0, /* Once */ + 0, /* COND */ + 0, /* CREF */ + 0, 0, /* BRAZERO, BRAMINZERO */ + 0, /* BRANUMBER */ + 0 /* BRA */ +}; + +/* These 2 tables allow for compact code for testing for \D, \d, \S, \s, \W, +and \w */ + +static uschar toptable1[] = { + 0, 0, 0, 0, 0, + ctype_digit, ctype_digit, + ctype_space, ctype_space, + ctype_word, ctype_word, + 0 /* OP_ANY */ +}; + +static uschar toptable2[] = { + 0, 0, 0, 0, 0, + ctype_digit, 0, + ctype_space, 0, + ctype_word, 0, + 1 /* OP_ANY */ +}; + + +/* Structure for holding data about a particular state, which is in effect the +current data for an active path through the match tree. It must consist +entirely of ints because the working vector we are passed, and which we put +these structures in, is a vector of ints. */ + +typedef struct stateblock { + int offset; /* Offset to opcode */ + int count; /* Count for repeats */ + int ims; /* ims flag bits */ + int data; /* Some use extra data */ +} stateblock; + +#define INTS_PER_STATEBLOCK (sizeof(stateblock)/sizeof(int)) + + +#ifdef DEBUG +/************************************************* +* Print character string * +*************************************************/ + +/* Character string printing function for debugging. + +Arguments: + p points to string + length number of bytes + f where to print + +Returns: nothing +*/ + +static void +pchars(unsigned char *p, int length, FILE *f) +{ +int c; +while (length-- > 0) + { + if (isprint(c = *(p++))) + fprintf(f, "%c", c); + else + fprintf(f, "\\x%02x", c); + } +} +#endif + + + +/************************************************* +* Execute a Regular Expression - DFA engine * +*************************************************/ + +/* This internal function applies a compiled pattern to a subject string, +starting at a given point, using a DFA engine. This function is called from the +external one, possibly multiple times if the pattern is not anchored. The +function calls itself recursively for some kinds of subpattern. + +Arguments: + md the match_data block with fixed information + this_start_code the opening bracket of this subexpression's code + current_subject where we currently are in the subject string + start_offset start offset in the subject string + offsets vector to contain the matching string offsets + offsetcount size of same + workspace vector of workspace + wscount size of same + ims the current ims flags + rlevel function call recursion level + recursing regex recursive call level + +Returns: > 0 => + = 0 => + -1 => failed to match + < -1 => some kind of unexpected problem + +The following macros are used for adding states to the two state vectors (one +for the current character, one for the following character). */ + +#define ADD_ACTIVE(x,y) \ + if (active_count++ < wscount) \ + { \ + next_active_state->offset = (x); \ + next_active_state->count = (y); \ + next_active_state->ims = ims; \ + next_active_state++; \ + DPRINTF(("%.*sADD_ACTIVE(%d,%d)\n", rlevel*2-2, SP, (x), (y))); \ + } \ + else return PCRE_ERROR_DFA_WSSIZE + +#define ADD_ACTIVE_DATA(x,y,z) \ + if (active_count++ < wscount) \ + { \ + next_active_state->offset = (x); \ + next_active_state->count = (y); \ + next_active_state->ims = ims; \ + next_active_state->data = (z); \ + next_active_state++; \ + DPRINTF(("%.*sADD_ACTIVE_DATA(%d,%d,%d)\n", rlevel*2-2, SP, (x), (y), (z))); \ + } \ + else return PCRE_ERROR_DFA_WSSIZE + +#define ADD_NEW(x,y) \ + if (new_count++ < wscount) \ + { \ + next_new_state->offset = (x); \ + next_new_state->count = (y); \ + next_new_state->ims = ims; \ + next_new_state++; \ + DPRINTF(("%.*sADD_NEW(%d,%d)\n", rlevel*2-2, SP, (x), (y))); \ + } \ + else return PCRE_ERROR_DFA_WSSIZE + +#define ADD_NEW_DATA(x,y,z) \ + if (new_count++ < wscount) \ + { \ + next_new_state->offset = (x); \ + next_new_state->count = (y); \ + next_new_state->ims = ims; \ + next_new_state->data = (z); \ + next_new_state++; \ + DPRINTF(("%.*sADD_NEW_DATA(%d,%d,%d)\n", rlevel*2-2, SP, (x), (y), (z))); \ + } \ + else return PCRE_ERROR_DFA_WSSIZE + +/* And now, here is the code */ + +static int +internal_dfa_exec( + dfa_match_data *md, + const uschar *this_start_code, + const uschar *current_subject, + int start_offset, + int *offsets, + int offsetcount, + int *workspace, + int wscount, + int ims, + int rlevel, + int recursing) +{ +stateblock *active_states, *new_states, *temp_states; +stateblock *next_active_state, *next_new_state; + +const uschar *ctypes, *lcc, *fcc; +const uschar *ptr; +const uschar *end_code; + +int active_count, new_count, match_count; + +/* Some fields in the md block are frequently referenced, so we load them into +independent variables in the hope that this will perform better. */ + +const uschar *start_subject = md->start_subject; +const uschar *end_subject = md->end_subject; +const uschar *start_code = md->start_code; + +BOOL utf8 = (md->poptions & PCRE_UTF8) != 0; + +rlevel++; +offsetcount &= (-2); + +wscount -= 2; +wscount = (wscount - (wscount % (INTS_PER_STATEBLOCK * 2))) / + (2 * INTS_PER_STATEBLOCK); + +DPRINTF(("\n%.*s---------------------\n" + "%.*sCall to internal_dfa_exec f=%d r=%d\n", + rlevel*2-2, SP, rlevel*2-2, SP, rlevel, recursing)); + +ctypes = md->tables + ctypes_offset; +lcc = md->tables + lcc_offset; +fcc = md->tables + fcc_offset; + +match_count = PCRE_ERROR_NOMATCH; /* A negative number */ + +active_states = (stateblock *)(workspace + 2); +next_new_state = new_states = active_states + wscount; +new_count = 0; + +/* The first thing in any (sub) pattern is a bracket of some sort. Push all +the alternative states onto the list, and find out where the end is. This +makes is possible to use this function recursively, when we want to stop at a +matching internal ket rather than at the end. + +If the first opcode in the first alternative is OP_REVERSE, we are dealing with +a backward assertion. In that case, we have to find out the maximum amount to +move back, and set up each alternative appropriately. */ + +if (this_start_code[1+LINK_SIZE] == OP_REVERSE) + { + int max_back = 0; + int gone_back; + + end_code = this_start_code; + do + { + int back = GET(end_code, 2+LINK_SIZE); + if (back > max_back) max_back = back; + end_code += GET(end_code, 1); + } + while (*end_code == OP_ALT); + + /* If we can't go back the amount required for the longest lookbehind + pattern, go back as far as we can; some alternatives may still be viable. */ + +#ifdef SUPPORT_UTF8 + /* In character mode we have to step back character by character */ + + if (utf8) + { + for (gone_back = 0; gone_back < max_back; gone_back++) + { + if (current_subject <= start_subject) break; + current_subject--; + while (current_subject > start_subject && + (*current_subject & 0xc0) == 0x80) + current_subject--; + } + } + else +#endif + + /* In byte-mode we can do this quickly. */ + + { + gone_back = (current_subject - max_back < start_subject)? + current_subject - start_subject : max_back; + current_subject -= gone_back; + } + + /* Now we can process the individual branches. */ + + end_code = this_start_code; + do + { + int back = GET(end_code, 2+LINK_SIZE); + if (back <= gone_back) + { + int bstate = end_code - start_code + 2 + 2*LINK_SIZE; + ADD_NEW_DATA(-bstate, 0, gone_back - back); + } + end_code += GET(end_code, 1); + } + while (*end_code == OP_ALT); + } + +/* This is the code for a "normal" subpattern (not a backward assertion). The +start of a whole pattern is always one of these. If we are at the top level, +we may be asked to restart matching from the same point that we reached for a +previous partial match. We still have to scan through the top-level branches to +find the end state. */ + +else + { + end_code = this_start_code; + + /* Restarting */ + + if (rlevel == 1 && (md->moptions & PCRE_DFA_RESTART) != 0) + { + do { end_code += GET(end_code, 1); } while (*end_code == OP_ALT); + new_count = workspace[1]; + if (!workspace[0]) + memcpy(new_states, active_states, new_count * sizeof(stateblock)); + } + + /* Not restarting */ + + else + { + do + { + ADD_NEW(end_code - start_code + 1 + LINK_SIZE, 0); + end_code += GET(end_code, 1); + } + while (*end_code == OP_ALT); + } + } + +workspace[0] = 0; /* Bit indicating which vector is current */ + +DPRINTF(("%.*sEnd state = %d\n", rlevel*2-2, SP, end_code - start_code)); + +/* Loop for scanning the subject */ + +ptr = current_subject; +for (;;) + { + int i, j; + int c, d, clen, dlen; + + /* Make the new state list into the active state list and empty the + new state list. */ + + temp_states = active_states; + active_states = new_states; + new_states = temp_states; + active_count = new_count; + new_count = 0; + + workspace[0] ^= 1; /* Remember for the restarting feature */ + workspace[1] = active_count; + +#ifdef DEBUG + printf("%.*sNext character: rest of subject = \"", rlevel*2-2, SP); + pchars((uschar *)ptr, strlen((char *)ptr), stdout); + printf("\"\n"); + + printf("%.*sActive states: ", rlevel*2-2, SP); + for (i = 0; i < active_count; i++) + printf("%d/%d ", active_states[i].offset, active_states[i].count); + printf("\n"); +#endif + + /* Set the pointers for adding new states */ + + next_active_state = active_states + active_count; + next_new_state = new_states; + + /* Load the current character from the subject outside the loop, as many + different states may want to look at it, and we assume that at least one + will. */ + + if (ptr < end_subject) + { + clen = 1; +#ifdef SUPPORT_UTF8 + if (utf8) { GETCHARLEN(c, ptr, clen); } else +#endif /* SUPPORT_UTF8 */ + c = *ptr; + } + else + { + clen = 0; /* At end subject */ + c = -1; + } + + /* Scan up the active states and act on each one. The result of an action + may be to add more states to the currently active list (e.g. on hitting a + parenthesis) or it may be to put states on the new list, for considering + when we move the character pointer on. */ + + for (i = 0; i < active_count; i++) + { + stateblock *current_state = active_states + i; + const uschar *code; + int state_offset = current_state->offset; + int count, codevalue; + int chartype, othercase; + +#ifdef DEBUG + printf ("%.*sProcessing state %d c=", rlevel*2-2, SP, state_offset); + if (c < 0) printf("-1\n"); + else if (c > 32 && c < 127) printf("'%c'\n", c); + else printf("0x%02x\n", c); +#endif + + /* This variable is referred to implicity in the ADD_xxx macros. */ + + ims = current_state->ims; + + /* A negative offset is a special case meaning "hold off going to this + (negated) state until the number of characters in the data field have + been skipped". */ + + if (state_offset < 0) + { + if (current_state->data > 0) + { + DPRINTF(("%.*sSkipping this character\n", rlevel*2-2, SP)); + ADD_NEW_DATA(state_offset, current_state->count, + current_state->data - 1); + continue; + } + else + { + current_state->offset = state_offset = -state_offset; + } + } + + /* Check for a duplicate state with the same count, and skip if found. */ + + for (j = 0; j < i; j++) + { + if (active_states[j].offset == state_offset && + active_states[j].count == current_state->count) + { + DPRINTF(("%.*sDuplicate state: skipped\n", rlevel*2-2, SP)); + goto NEXT_ACTIVE_STATE; + } + } + + /* The state offset is the offset to the opcode */ + + code = start_code + state_offset; + codevalue = *code; + if (codevalue >= OP_BRA) codevalue = OP_BRA; /* All brackets are equal */ + + /* If this opcode is followed by an inline character, load it. It is + tempting to test for the presence of a subject character here, but that + is wrong, because sometimes zero repetitions of the subject are + permitted. + + We also use this mechanism for opcodes such as OP_TYPEPLUS that take an + argument that is not a data character - but is always one byte long. + Unfortunately, we have to take special action to deal with \P, \p, and + \X in this case. To keep the other cases fast, convert these ones to new + opcodes. */ + + if (coptable[codevalue] > 0) + { + dlen = 1; +#ifdef SUPPORT_UTF8 + if (utf8) { GETCHARLEN(d, (code + coptable[codevalue]), dlen); } else +#endif /* SUPPORT_UTF8 */ + d = code[coptable[codevalue]]; + if (codevalue >= OP_TYPESTAR) + { + if (d == OP_ANYBYTE) return PCRE_ERROR_DFA_UITEM; + if (d >= OP_NOTPROP) + codevalue += (d == OP_EXTUNI)? OP_EXTUNI_EXTRA : OP_PROP_EXTRA; + } + } + else + { + dlen = 0; /* Not strictly necessary, but compilers moan */ + d = -1; /* if these variables are not set. */ + } + + + /* Now process the individual opcodes */ + + switch (codevalue) + { + +/* ========================================================================== */ + /* Reached a closing bracket. If not at the end of the pattern, carry + on with the next opcode. Otherwise, unless we have an empty string and + PCRE_NOTEMPTY is set, save the match data, shifting up all previous + matches so we always have the longest first. */ + + case OP_KET: + case OP_KETRMIN: + case OP_KETRMAX: + if (code != end_code) + { + ADD_ACTIVE(state_offset + 1 + LINK_SIZE, 0); + if (codevalue != OP_KET) + { + ADD_ACTIVE(state_offset - GET(code, 1), 0); + } + } + else if (ptr > current_subject || (md->moptions & PCRE_NOTEMPTY) == 0) + { + if (match_count < 0) match_count = (offsetcount >= 2)? 1 : 0; + else if (match_count > 0 && ++match_count * 2 >= offsetcount) + match_count = 0; + count = ((match_count == 0)? offsetcount : match_count * 2) - 2; + if (count > 0) memmove(offsets + 2, offsets, count * sizeof(int)); + if (offsetcount >= 2) + { + offsets[0] = current_subject - start_subject; + offsets[1] = ptr - start_subject; + DPRINTF(("%.*sSet matched string = \"%.*s\"\n", rlevel*2-2, SP, + offsets[1] - offsets[0], current_subject)); + } + if ((md->moptions & PCRE_DFA_SHORTEST) != 0) + { + DPRINTF(("%.*sEnd of internal_dfa_exec %d: returning %d\n" + "%.*s---------------------\n\n", rlevel*2-2, SP, rlevel, + match_count, rlevel*2-2, SP)); + return match_count; + } + } + break; + +/* ========================================================================== */ + /* These opcodes add to the current list of states without looking + at the current character. */ + + /*-----------------------------------------------------------------*/ + case OP_ALT: + do { code += GET(code, 1); } while (*code == OP_ALT); + ADD_ACTIVE(code - start_code, 0); + break; + + /*-----------------------------------------------------------------*/ + case OP_BRA: + do + { + ADD_ACTIVE(code - start_code + 1 + LINK_SIZE, 0); + code += GET(code, 1); + } + while (*code == OP_ALT); + break; + + /*-----------------------------------------------------------------*/ + case OP_BRAZERO: + case OP_BRAMINZERO: + ADD_ACTIVE(state_offset + 1, 0); + code += 1 + GET(code, 2); + while (*code == OP_ALT) code += GET(code, 1); + ADD_ACTIVE(code - start_code + 1 + LINK_SIZE, 0); + break; + + /*-----------------------------------------------------------------*/ + case OP_BRANUMBER: + ADD_ACTIVE(state_offset + 1 + LINK_SIZE, 0); + break; + + /*-----------------------------------------------------------------*/ + case OP_CIRC: + if ((ptr == start_subject && (md->moptions & PCRE_NOTBOL) == 0) || + ((ims & PCRE_MULTILINE) != 0 && ptr[-1] == NEWLINE)) + { ADD_ACTIVE(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_EOD: + if (ptr >= end_subject) { ADD_ACTIVE(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_OPT: + ims = code[1]; + ADD_ACTIVE(state_offset + 2, 0); + break; + + /*-----------------------------------------------------------------*/ + case OP_SOD: + if (ptr == start_subject) { ADD_ACTIVE(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_SOM: + if (ptr == start_subject + start_offset) { ADD_ACTIVE(state_offset + 1, 0); } + break; + + +/* ========================================================================== */ + /* These opcodes inspect the next subject character, and sometimes + the previous one as well, but do not have an argument. The variable + clen contains the length of the current character and is zero if we are + at the end of the subject. */ + + /*-----------------------------------------------------------------*/ + case OP_ANY: + if (clen > 0 && (c != NEWLINE || (ims & PCRE_DOTALL) != 0)) + { ADD_NEW(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_EODN: + if (clen == 0 || (c == NEWLINE && ptr + 1 == end_subject)) + { ADD_ACTIVE(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_DOLL: + if ((md->moptions & PCRE_NOTEOL) == 0) + { + if (clen == 0 || (c == NEWLINE && (ptr + 1 == end_subject || + (ims & PCRE_MULTILINE) != 0))) + { ADD_ACTIVE(state_offset + 1, 0); } + } + else if (c == NEWLINE && (ims & PCRE_MULTILINE) != 0) + { ADD_ACTIVE(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + + case OP_DIGIT: + case OP_WHITESPACE: + case OP_WORDCHAR: + if (clen > 0 && c < 256 && + ((ctypes[c] & toptable1[codevalue]) ^ toptable2[codevalue]) != 0) + { ADD_NEW(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_NOT_DIGIT: + case OP_NOT_WHITESPACE: + case OP_NOT_WORDCHAR: + if (clen > 0 && (c >= 256 || + ((ctypes[c] & toptable1[codevalue]) ^ toptable2[codevalue]) != 0)) + { ADD_NEW(state_offset + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_WORD_BOUNDARY: + case OP_NOT_WORD_BOUNDARY: + { + int left_word, right_word; + + if (ptr > start_subject) + { + const uschar *temp = ptr - 1; +#ifdef SUPPORT_UTF8 + if (utf8) BACKCHAR(temp); +#endif + GETCHARTEST(d, temp); + left_word = d < 256 && (ctypes[d] & ctype_word) != 0; + } + else left_word = 0; + + if (clen > 0) right_word = c < 256 && (ctypes[c] & ctype_word) != 0; + else right_word = 0; + + if ((left_word == right_word) == (codevalue == OP_NOT_WORD_BOUNDARY)) + { ADD_ACTIVE(state_offset + 1, 0); } + } + break; + + +#ifdef SUPPORT_UCP + + /*-----------------------------------------------------------------*/ + /* Check the next character by Unicode property. We will get here only + if the support is in the binary; otherwise a compile-time error occurs. + */ + + case OP_PROP: + case OP_NOTPROP: + if (clen > 0) + { + int rqdtype, category; + category = ucp_findchar(c, &chartype, &othercase); + rqdtype = code[1]; + if (rqdtype >= 128) + { + if ((rqdtype - 128 == category) == (codevalue == OP_PROP)) + { ADD_NEW(state_offset + 2, 0); } + } + else + { + if ((rqdtype == chartype) == (codevalue == OP_PROP)) + { ADD_NEW(state_offset + 2, 0); } + } + } + break; +#endif + + + +/* ========================================================================== */ + /* These opcodes likewise inspect the subject character, but have an + argument that is not a data character. It is one of these opcodes: + OP_ANY, OP_DIGIT, OP_NOT_DIGIT, OP_WHITESPACE, OP_NOT_SPACE, OP_WORDCHAR, + OP_NOT_WORDCHAR. The value is loaded into d. */ + + case OP_TYPEPLUS: + case OP_TYPEMINPLUS: + count = current_state->count; /* Already matched */ + if (count > 0) { ADD_ACTIVE(state_offset + 2, 0); } + if (clen > 0) + { + if ((c >= 256 && d != OP_DIGIT && d != OP_WHITESPACE && d != OP_WORDCHAR) || + (c < 256 && + (d != OP_ANY || c != '\n' || (ims & PCRE_DOTALL) != 0) && + ((ctypes[c] & toptable1[d]) ^ toptable2[d]) != 0)) + { + count++; + ADD_NEW(state_offset, count); + } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_TYPEQUERY: + case OP_TYPEMINQUERY: + ADD_ACTIVE(state_offset + 2, 0); + if (clen > 0) + { + if ((c >= 256 && d != OP_DIGIT && d != OP_WHITESPACE && d != OP_WORDCHAR) || + (c < 256 && + (d != OP_ANY || c != '\n' || (ims & PCRE_DOTALL) != 0) && + ((ctypes[c] & toptable1[d]) ^ toptable2[d]) != 0)) + { + ADD_NEW(state_offset + 2, 0); + } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_TYPESTAR: + case OP_TYPEMINSTAR: + ADD_ACTIVE(state_offset + 2, 0); + if (clen > 0) + { + if ((c >= 256 && d != OP_DIGIT && d != OP_WHITESPACE && d != OP_WORDCHAR) || + (c < 256 && + (d != OP_ANY || c != '\n' || (ims & PCRE_DOTALL) != 0) && + ((ctypes[c] & toptable1[d]) ^ toptable2[d]) != 0)) + { + ADD_NEW(state_offset, 0); + } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_TYPEEXACT: + case OP_TYPEUPTO: + case OP_TYPEMINUPTO: + if (codevalue != OP_TYPEEXACT) + { ADD_ACTIVE(state_offset + 4, 0); } + count = current_state->count; /* Number already matched */ + if (clen > 0) + { + if ((c >= 256 && d != OP_DIGIT && d != OP_WHITESPACE && d != OP_WORDCHAR) || + (c < 256 && + (d != OP_ANY || c != '\n' || (ims & PCRE_DOTALL) != 0) && + ((ctypes[c] & toptable1[d]) ^ toptable2[d]) != 0)) + { + if (++count >= GET2(code, 1)) + { ADD_NEW(state_offset + 4, 0); } + else + { ADD_NEW(state_offset, count); } + } + } + break; + +/* ========================================================================== */ + /* These are virtual opcodes that are used when something like + OP_TYPEPLUS has OP_PROP, OP_NOTPROP, or OP_EXTUNI as its argument. It + keeps the code above fast for the other cases. The argument is in the + d variable. */ + + case OP_PROP_EXTRA + OP_TYPEPLUS: + case OP_PROP_EXTRA + OP_TYPEMINPLUS: + count = current_state->count; /* Already matched */ + if (count > 0) { ADD_ACTIVE(state_offset + 3, 0); } + if (clen > 0) + { + int category = ucp_findchar(c, &chartype, &othercase); + int rqdtype = code[2]; + if ((d == OP_PROP) == + (rqdtype == ((rqdtype >= 128)? (category + 128) : chartype))) + { count++; ADD_NEW(state_offset, count); } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_EXTUNI_EXTRA + OP_TYPEPLUS: + case OP_EXTUNI_EXTRA + OP_TYPEMINPLUS: + count = current_state->count; /* Already matched */ + if (count > 0) { ADD_ACTIVE(state_offset + 2, 0); } + if (clen > 0 && ucp_findchar(c, &chartype, &othercase) != ucp_M) + { + const uschar *nptr = ptr + clen; + int ncount = 0; + while (nptr < end_subject) + { + int nd; + int ndlen = 1; + GETCHARLEN(nd, nptr, ndlen); + if (ucp_findchar(nd, &chartype, &othercase) != ucp_M) break; + ncount++; + nptr += ndlen; + } + count++; + ADD_NEW_DATA(-state_offset, count, ncount); + } + break; + + /*-----------------------------------------------------------------*/ + case OP_PROP_EXTRA + OP_TYPEQUERY: + case OP_PROP_EXTRA + OP_TYPEMINQUERY: + count = 3; + goto QS1; + + case OP_PROP_EXTRA + OP_TYPESTAR: + case OP_PROP_EXTRA + OP_TYPEMINSTAR: + count = 0; + + QS1: + + ADD_ACTIVE(state_offset + 3, 0); + if (clen > 0) + { + int category = ucp_findchar(c, &chartype, &othercase); + int rqdtype = code[2]; + if ((d == OP_PROP) == + (rqdtype == ((rqdtype >= 128)? (category + 128) : chartype))) + { ADD_NEW(state_offset + count, 0); } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_EXTUNI_EXTRA + OP_TYPEQUERY: + case OP_EXTUNI_EXTRA + OP_TYPEMINQUERY: + count = 2; + goto QS2; + + case OP_EXTUNI_EXTRA + OP_TYPESTAR: + case OP_EXTUNI_EXTRA + OP_TYPEMINSTAR: + count = 0; + + QS2: + + ADD_ACTIVE(state_offset + 2, 0); + if (clen > 0 && ucp_findchar(c, &chartype, &othercase) != ucp_M) + { + const uschar *nptr = ptr + clen; + int ncount = 0; + while (nptr < end_subject) + { + int nd; + int ndlen = 1; + GETCHARLEN(nd, nptr, ndlen); + if (ucp_findchar(nd, &chartype, &othercase) != ucp_M) break; + ncount++; + nptr += ndlen; + } + ADD_NEW_DATA(-(state_offset + count), 0, ncount); + } + break; + + /*-----------------------------------------------------------------*/ + case OP_PROP_EXTRA + OP_TYPEEXACT: + case OP_PROP_EXTRA + OP_TYPEUPTO: + case OP_PROP_EXTRA + OP_TYPEMINUPTO: + if (codevalue != OP_PROP_EXTRA + OP_TYPEEXACT) + { ADD_ACTIVE(state_offset + 5, 0); } + count = current_state->count; /* Number already matched */ + if (clen > 0) + { + int category = ucp_findchar(c, &chartype, &othercase); + int rqdtype = code[4]; + if ((d == OP_PROP) == + (rqdtype == ((rqdtype >= 128)? (category + 128) : chartype))) + { + if (++count >= GET2(code, 1)) + { ADD_NEW(state_offset + 5, 0); } + else + { ADD_NEW(state_offset, count); } + } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_EXTUNI_EXTRA + OP_TYPEEXACT: + case OP_EXTUNI_EXTRA + OP_TYPEUPTO: + case OP_EXTUNI_EXTRA + OP_TYPEMINUPTO: + if (codevalue != OP_EXTUNI_EXTRA + OP_TYPEEXACT) + { ADD_ACTIVE(state_offset + 4, 0); } + count = current_state->count; /* Number already matched */ + if (clen > 0 && ucp_findchar(c, &chartype, &othercase) != ucp_M) + { + const uschar *nptr = ptr + clen; + int ncount = 0; + while (nptr < end_subject) + { + int nd; + int ndlen = 1; + GETCHARLEN(nd, nptr, ndlen); + if (ucp_findchar(nd, &chartype, &othercase) != ucp_M) break; + ncount++; + nptr += ndlen; + } + if (++count >= GET2(code, 1)) + { ADD_NEW_DATA(-(state_offset + 4), 0, ncount); } + else + { ADD_NEW_DATA(-state_offset, count, ncount); } + } + break; + +/* ========================================================================== */ + /* These opcodes are followed by a character that is usually compared + to the current subject character; it is loaded into d. We still get + here even if there is no subject character, because in some cases zero + repetitions are permitted. */ + + /*-----------------------------------------------------------------*/ + case OP_CHAR: + if (clen > 0 && c == d) { ADD_NEW(state_offset + dlen + 1, 0); } + break; + + /*-----------------------------------------------------------------*/ + case OP_CHARNC: + if (clen == 0) break; + +#ifdef SUPPORT_UTF8 + if (utf8) + { + if (c == d) { ADD_NEW(state_offset + dlen + 1, 0); } else + { + if (c < 128) othercase = fcc[c]; else + + /* If we have Unicode property support, we can use it to test the + other case of the character, if there is one. The result of + ucp_findchar() is < 0 if the char isn't found, and othercase is + returned as zero if there isn't another case. */ + +#ifdef SUPPORT_UCP + if (ucp_findchar(c, &chartype, &othercase) < 0) +#endif + othercase = -1; + + if (d == othercase) { ADD_NEW(state_offset + dlen + 1, 0); } + } + } + else +#endif /* SUPPORT_UTF8 */ + + /* Non-UTF-8 mode */ + { + if (lcc[c] == lcc[d]) { ADD_NEW(state_offset + 2, 0); } + } + break; + + +#ifdef SUPPORT_UCP + /*-----------------------------------------------------------------*/ + /* This is a tricky one because it can match more than one character. + Find out how many characters to skip, and then set up a negative state + to wait for them to pass before continuing. */ + + case OP_EXTUNI: + if (clen > 0 && ucp_findchar(c, &chartype, &othercase) != ucp_M) + { + const uschar *nptr = ptr + clen; + int ncount = 0; + while (nptr < end_subject) + { + int nclen = 1; + GETCHARLEN(c, nptr, nclen); + if (ucp_findchar(c, &chartype, &othercase) != ucp_M) break; + ncount++; + nptr += nclen; + } + ADD_NEW_DATA(-(state_offset + 1), 0, ncount); + } + break; +#endif + + /*-----------------------------------------------------------------*/ + /* Match a negated single character. This is only used for one-byte + characters, that is, we know that d < 256. The character we are + checking (c) can be multibyte. */ + + case OP_NOT: + if (clen > 0) + { + int otherd = ((ims & PCRE_CASELESS) != 0)? fcc[d] : d; + if (c != d && c != otherd) { ADD_NEW(state_offset + dlen + 1, 0); } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_PLUS: + case OP_MINPLUS: + case OP_NOTPLUS: + case OP_NOTMINPLUS: + count = current_state->count; /* Already matched */ + if (count > 0) { ADD_ACTIVE(state_offset + dlen + 1, 0); } + if (clen > 0) + { + int otherd = -1; + if ((ims & PCRE_CASELESS) != 0) + { +#ifdef SUPPORT_UTF8 + if (utf8 && c >= 128) + { +#ifdef SUPPORT_UCP + if (ucp_findchar(d, &chartype, &otherd) < 0) otherd = -1; +#endif /* SUPPORT_UCP */ + } + else +#endif /* SUPPORT_UTF8 */ + otherd = fcc[d]; + } + if ((c == d || c == otherd) == (codevalue < OP_NOTSTAR)) + { count++; ADD_NEW(state_offset, count); } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_QUERY: + case OP_MINQUERY: + case OP_NOTQUERY: + case OP_NOTMINQUERY: + ADD_ACTIVE(state_offset + dlen + 1, 0); + if (clen > 0) + { + int otherd = -1; + if ((ims && PCRE_CASELESS) != 0) + { +#ifdef SUPPORT_UTF8 + if (utf8 && c >= 128) + { +#ifdef SUPPORT_UCP + if (ucp_findchar(c, &chartype, &otherd) < 0) otherd = -1; +#endif /* SUPPORT_UCP */ + } + else +#endif /* SUPPORT_UTF8 */ + otherd = fcc[d]; + } + if ((c == d || c == otherd) == (codevalue < OP_NOTSTAR)) + { ADD_NEW(state_offset + dlen + 1, 0); } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_STAR: + case OP_MINSTAR: + case OP_NOTSTAR: + case OP_NOTMINSTAR: + ADD_ACTIVE(state_offset + dlen + 1, 0); + if (clen > 0) + { + int otherd = -1; + if ((ims && PCRE_CASELESS) != 0) + { +#ifdef SUPPORT_UTF8 + if (utf8 && c >= 128) + { +#ifdef SUPPORT_UCP + if (ucp_findchar(c, &chartype, &otherd) < 0) otherd = -1; +#endif /* SUPPORT_UCP */ + } + else +#endif /* SUPPORT_UTF8 */ + otherd = fcc[d]; + } + if ((c == d || c == otherd) == (codevalue < OP_NOTSTAR)) + { ADD_NEW(state_offset, 0); } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_EXACT: + case OP_UPTO: + case OP_MINUPTO: + case OP_NOTEXACT: + case OP_NOTUPTO: + case OP_NOTMINUPTO: + if (codevalue != OP_EXACT && codevalue != OP_NOTEXACT) + { ADD_ACTIVE(state_offset + dlen + 3, 0); } + count = current_state->count; /* Number already matched */ + if (clen > 0) + { + int otherd = -1; + if ((ims & PCRE_CASELESS) != 0) + { +#ifdef SUPPORT_UTF8 + if (utf8 && c >= 128) + { +#ifdef SUPPORT_UCP + if (ucp_findchar(d, &chartype, &otherd) < 0) otherd = -1; +#endif /* SUPPORT_UCP */ + } + else +#endif /* SUPPORT_UTF8 */ + otherd = fcc[d]; + } + if ((c == d || c == otherd) == (codevalue < OP_NOTSTAR)) + { + if (++count >= GET2(code, 1)) + { ADD_NEW(state_offset + dlen + 3, 0); } + else + { ADD_NEW(state_offset, count); } + } + } + break; + + +/* ========================================================================== */ + /* These are the class-handling opcodes */ + + case OP_CLASS: + case OP_NCLASS: + case OP_XCLASS: + { + BOOL isinclass = FALSE; + int next_state_offset; + const uschar *ecode; + + /* For a simple class, there is always just a 32-byte table, and we + can set isinclass from it. */ + + if (codevalue != OP_XCLASS) + { + ecode = code + 33; + if (clen > 0) + { + isinclass = (c > 255)? (codevalue == OP_NCLASS) : + ((code[1 + c/8] & (1 << (c&7))) != 0); + } + } + + /* An extended class may have a table or a list of single characters, + ranges, or both, and it may be positive or negative. There's a + function that sorts all this out. */ + + else + { + ecode = code + GET(code, 1); + if (clen > 0) isinclass = _pcre_xclass(c, code + 1 + LINK_SIZE); + } + + /* At this point, isinclass is set for all kinds of class, and ecode + points to the byte after the end of the class. If there is a + quantifier, this is where it will be. */ + + next_state_offset = ecode - start_code; + + switch (*ecode) + { + case OP_CRSTAR: + case OP_CRMINSTAR: + ADD_ACTIVE(next_state_offset + 1, 0); + if (isinclass) { ADD_NEW(state_offset, 0); } + break; + + case OP_CRPLUS: + case OP_CRMINPLUS: + count = current_state->count; /* Already matched */ + if (count > 0) { ADD_ACTIVE(next_state_offset + 1, 0); } + if (isinclass) { count++; ADD_NEW(state_offset, count); } + break; + + case OP_CRQUERY: + case OP_CRMINQUERY: + ADD_ACTIVE(next_state_offset + 1, 0); + if (isinclass) { ADD_NEW(next_state_offset + 1, 0); } + break; + + case OP_CRRANGE: + case OP_CRMINRANGE: + count = current_state->count; /* Already matched */ + if (count >= GET2(ecode, 1)) + { ADD_ACTIVE(next_state_offset + 5, 0); } + if (isinclass) + { + if (++count >= GET2(ecode, 3)) + { ADD_NEW(next_state_offset + 5, 0); } + else + { ADD_NEW(state_offset, count); } + } + break; + + default: + if (isinclass) { ADD_NEW(next_state_offset, 0); } + break; + } + } + break; + +/* ========================================================================== */ + /* These are the opcodes for fancy brackets of various kinds. We have + to use recursion in order to handle them. */ + + case OP_ASSERT: + case OP_ASSERT_NOT: + case OP_ASSERTBACK: + case OP_ASSERTBACK_NOT: + { + int rc; + int local_offsets[2]; + int local_workspace[1000]; + const uschar *endasscode = code + GET(code, 1); + + while (*endasscode == OP_ALT) endasscode += GET(endasscode, 1); + + rc = internal_dfa_exec( + md, /* static match data */ + code, /* this subexpression's code */ + ptr, /* where we currently are */ + ptr - start_subject, /* start offset */ + local_offsets, /* offset vector */ + sizeof(local_offsets)/sizeof(int), /* size of same */ + local_workspace, /* workspace vector */ + sizeof(local_workspace)/sizeof(int), /* size of same */ + ims, /* the current ims flags */ + rlevel, /* function recursion level */ + recursing); /* pass on regex recursion */ + + if ((rc >= 0) == (codevalue == OP_ASSERT || codevalue == OP_ASSERTBACK)) + { ADD_ACTIVE(endasscode + LINK_SIZE + 1 - start_code, 0); } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_COND: + { + int local_offsets[1000]; + int local_workspace[1000]; + int condcode = code[LINK_SIZE+1]; + + /* The only supported version of OP_CREF is for the value 0xffff, which + means "test if in a recursion". */ + + if (condcode == OP_CREF) + { + int value = GET2(code, LINK_SIZE+2); + if (value != 0xffff) return PCRE_ERROR_DFA_UCOND; + if (recursing > 0) { ADD_ACTIVE(state_offset + LINK_SIZE + 4, 0); } + else { ADD_ACTIVE(state_offset + GET(code, 1) + LINK_SIZE + 1, 0); } + } + + /* Otherwise, the condition is an assertion */ + + else + { + int rc; + const uschar *asscode = code + LINK_SIZE + 1; + const uschar *endasscode = asscode + GET(asscode, 1); + + while (*endasscode == OP_ALT) endasscode += GET(endasscode, 1); + + rc = internal_dfa_exec( + md, /* fixed match data */ + asscode, /* this subexpression's code */ + ptr, /* where we currently are */ + ptr - start_subject, /* start offset */ + local_offsets, /* offset vector */ + sizeof(local_offsets)/sizeof(int), /* size of same */ + local_workspace, /* workspace vector */ + sizeof(local_workspace)/sizeof(int), /* size of same */ + ims, /* the current ims flags */ + rlevel, /* function recursion level */ + recursing); /* pass on regex recursion */ + + if ((rc >= 0) == + (condcode == OP_ASSERT || condcode == OP_ASSERTBACK)) + { ADD_ACTIVE(endasscode + LINK_SIZE + 1 - start_code, 0); } + else + { ADD_ACTIVE(state_offset + GET(code, 1) + LINK_SIZE + 1, 0); } + } + } + break; + + /*-----------------------------------------------------------------*/ + case OP_RECURSE: + { + int local_offsets[1000]; + int local_workspace[1000]; + int rc; + + DPRINTF(("%.*sStarting regex recursion %d\n", rlevel*2-2, SP, + recursing + 1)); + + rc = internal_dfa_exec( + md, /* fixed match data */ + start_code + GET(code, 1), /* this subexpression's code */ + ptr, /* where we currently are */ + ptr - start_subject, /* start offset */ + local_offsets, /* offset vector */ + sizeof(local_offsets)/sizeof(int), /* size of same */ + local_workspace, /* workspace vector */ + sizeof(local_workspace)/sizeof(int), /* size of same */ + ims, /* the current ims flags */ + rlevel, /* function recursion level */ + recursing + 1); /* regex recurse level */ + + DPRINTF(("%.*sReturn from regex recursion %d: rc=%d\n", rlevel*2-2, SP, + recursing + 1, rc)); + + /* Ran out of internal offsets */ + + if (rc == 0) return PCRE_ERROR_DFA_RECURSE; + + /* For each successful matched substring, set up the next state with a + count of characters to skip before trying it. Note that the count is in + characters, not bytes. */ + + if (rc > 0) + { + for (rc = rc*2 - 2; rc >= 0; rc -= 2) + { + const uschar *p = start_subject + local_offsets[rc]; + const uschar *pp = start_subject + local_offsets[rc+1]; + int charcount = local_offsets[rc+1] - local_offsets[rc]; + while (p < pp) if ((*p++ & 0xc0) == 0x80) charcount--; + if (charcount > 0) + { + ADD_NEW_DATA(-(state_offset + LINK_SIZE + 1), 0, (charcount - 1)); + } + else + { + ADD_ACTIVE(state_offset + LINK_SIZE + 1, 0); + } + } + } + else if (rc != PCRE_ERROR_NOMATCH) return rc; + } + break; + + /*-----------------------------------------------------------------*/ + case OP_ONCE: + { + const uschar *endcode; + int local_offsets[2]; + int local_workspace[1000]; + + int rc = internal_dfa_exec( + md, /* fixed match data */ + code, /* this subexpression's code */ + ptr, /* where we currently are */ + ptr - start_subject, /* start offset */ + local_offsets, /* offset vector */ + sizeof(local_offsets)/sizeof(int), /* size of same */ + local_workspace, /* workspace vector */ + sizeof(local_workspace)/sizeof(int), /* size of same */ + ims, /* the current ims flags */ + rlevel, /* function recursion level */ + recursing); /* pass on regex recursion */ + + if (rc >= 0) + { + const uschar *end_subpattern = code; + int charcount = local_offsets[1] - local_offsets[0]; + int next_state_offset, repeat_state_offset; + BOOL is_repeated; + + do { end_subpattern += GET(end_subpattern, 1); } + while (*end_subpattern == OP_ALT); + next_state_offset = end_subpattern - start_code + LINK_SIZE + 1; + + /* If the end of this subpattern is KETRMAX or KETRMIN, we must + arrange for the repeat state also to be added to the relevant list. + Calculate the offset, or set -1 for no repeat. */ + + repeat_state_offset = (*end_subpattern == OP_KETRMAX || + *end_subpattern == OP_KETRMIN)? + end_subpattern - start_code - GET(end_subpattern, 1) : -1; + + /* If we have matched an empty string, add the next state at the + current character pointer. This is important so that the duplicate + checking kicks in, which is what breaks infinite loops that match an + empty string. */ + + if (charcount == 0) + { + ADD_ACTIVE(next_state_offset, 0); + } + + /* Optimization: if there are no more active states, and there + are no new states yet set up, then skip over the subject string + right here, to save looping. Otherwise, set up the new state to swing + into action when the end of the substring is reached. */ + + else if (i + 1 >= active_count && new_count == 0) + { + ptr += charcount; + clen = 0; + ADD_NEW(next_state_offset, 0); + + /* If we are adding a repeat state at the new character position, + we must fudge things so that it is the only current state. + Otherwise, it might be a duplicate of one we processed before, and + that would cause it to be skipped. */ + + if (repeat_state_offset >= 0) + { + next_active_state = active_states; + active_count = 0; + i = -1; + ADD_ACTIVE(repeat_state_offset, 0); + } + } + else + { + const uschar *p = start_subject + local_offsets[0]; + const uschar *pp = start_subject + local_offsets[1]; + while (p < pp) if ((*p++ & 0xc0) == 0x80) charcount--; + ADD_NEW_DATA(-next_state_offset, 0, (charcount - 1)); + if (repeat_state_offset >= 0) + { ADD_NEW_DATA(-repeat_state_offset, 0, (charcount - 1)); } + } + + } + else if (rc != PCRE_ERROR_NOMATCH) return rc; + } + break; + + +/* ========================================================================== */ + /* Handle callouts */ + + case OP_CALLOUT: + if (pcre_callout != NULL) + { + int rrc; + pcre_callout_block cb; + cb.version = 1; /* Version 1 of the callout block */ + cb.callout_number = code[1]; + cb.offset_vector = offsets; + cb.subject = (char *)start_subject; + cb.subject_length = end_subject - start_subject; + cb.start_match = current_subject - start_subject; + cb.current_position = ptr - start_subject; + cb.pattern_position = GET(code, 2); + cb.next_item_length = GET(code, 2 + LINK_SIZE); + cb.capture_top = 1; + cb.capture_last = -1; + cb.callout_data = md->callout_data; + if ((rrc = (*pcre_callout)(&cb)) < 0) return rrc; /* Abandon */ + if (rrc == 0) { ADD_ACTIVE(state_offset + 2 + 2*LINK_SIZE, 0); } + } + break; + + +/* ========================================================================== */ + default: /* Unsupported opcode */ + return PCRE_ERROR_DFA_UITEM; + } + + NEXT_ACTIVE_STATE: continue; + + } /* End of loop scanning active states */ + + /* We have finished the processing at the current subject character. If no + new states have been set for the next character, we have found all the + matches that we are going to find. If we are at the top level and partial + matching has been requested, check for appropriate conditions. */ + + if (new_count <= 0) + { + if (match_count < 0 && /* No matches found */ + rlevel == 1 && /* Top level match function */ + (md->moptions & PCRE_PARTIAL) != 0 && /* Want partial matching */ + ptr >= end_subject && /* Reached end of subject */ + ptr > current_subject) /* Matched non-empty string */ + { + if (offsetcount >= 2) + { + offsets[0] = current_subject - start_subject; + offsets[1] = end_subject - start_subject; + } + match_count = PCRE_ERROR_PARTIAL; + } + + DPRINTF(("%.*sEnd of internal_dfa_exec %d: returning %d\n" + "%.*s---------------------\n\n", rlevel*2-2, SP, rlevel, match_count, + rlevel*2-2, SP)); + return match_count; + } + + /* One or more states are active for the next character. */ + + ptr += clen; /* Advance to next subject character */ + } /* Loop to move along the subject string */ + +/* Control never gets here, but we must keep the compiler happy. */ + +DPRINTF(("%.*s+++ Unexpected end of internal_dfa_exec %d +++\n" + "%.*s---------------------\n\n", rlevel*2-2, SP, rlevel, rlevel*2-2, SP)); +return PCRE_ERROR_NOMATCH; +} + + + + +/************************************************* +* Execute a Regular Expression - DFA engine * +*************************************************/ + +/* This external function applies a compiled re to a subject string using a DFA +engine. This function calls the internal function multiple times if the pattern +is not anchored. + +Arguments: + argument_re points to the compiled expression + extra_data points to extra data or is NULL (not currently used) + subject points to the subject string + length length of subject string (may contain binary zeros) + start_offset where to start in the subject string + options option bits + offsets vector of match offsets + offsetcount size of same + workspace workspace vector + wscount size of same + +Returns: > 0 => number of match offset pairs placed in offsets + = 0 => offsets overflowed; longest matches are present + -1 => failed to match + < -1 => some kind of unexpected problem +*/ + +EXPORT int +pcre_dfa_exec(const pcre *argument_re, const pcre_extra *extra_data, + const char *subject, int length, int start_offset, int options, int *offsets, + int offsetcount, int *workspace, int wscount) +{ +real_pcre *re = (real_pcre *)argument_re; +dfa_match_data match_block; +BOOL utf8, anchored, startline, firstline; +const uschar *current_subject, *end_subject, *lcc; + +pcre_study_data internal_study; +const pcre_study_data *study = NULL; +real_pcre internal_re; + +const uschar *req_byte_ptr; +const uschar *start_bits = NULL; +BOOL first_byte_caseless = FALSE; +BOOL req_byte_caseless = FALSE; +int first_byte = -1; +int req_byte = -1; +int req_byte2 = -1; + +/* Plausibility checks */ + +if ((options & ~PUBLIC_DFA_EXEC_OPTIONS) != 0) return PCRE_ERROR_BADOPTION; +if (re == NULL || subject == NULL || workspace == NULL || + (offsets == NULL && offsetcount > 0)) return PCRE_ERROR_NULL; +if (offsetcount < 0) return PCRE_ERROR_BADCOUNT; +if (wscount < 20) return PCRE_ERROR_DFA_WSSIZE; + +/* We need to find the pointer to any study data before we test for byte +flipping, so we scan the extra_data block first. This may set two fields in the +match block, so we must initialize them beforehand. However, the other fields +in the match block must not be set until after the byte flipping. */ + +match_block.tables = re->tables; +match_block.callout_data = NULL; + +if (extra_data != NULL) + { + unsigned int flags = extra_data->flags; + if ((flags & PCRE_EXTRA_STUDY_DATA) != 0) + study = (const pcre_study_data *)extra_data->study_data; + if ((flags & PCRE_EXTRA_MATCH_LIMIT) != 0) return PCRE_ERROR_DFA_UMLIMIT; + if ((flags & PCRE_EXTRA_CALLOUT_DATA) != 0) + match_block.callout_data = extra_data->callout_data; + if ((flags & PCRE_EXTRA_TABLES) != 0) + match_block.tables = extra_data->tables; + } + +/* Check that the first field in the block is the magic number. If it is not, +test for a regex that was compiled on a host of opposite endianness. If this is +the case, flipped values are put in internal_re and internal_study if there was +study data too. */ + +if (re->magic_number != MAGIC_NUMBER) + { + re = _pcre_try_flipped(re, &internal_re, study, &internal_study); + if (re == NULL) return PCRE_ERROR_BADMAGIC; + if (study != NULL) study = &internal_study; + } + +/* Set some local values */ + +current_subject = (const unsigned char *)subject + start_offset; +end_subject = (const unsigned char *)subject + length; +req_byte_ptr = current_subject - 1; + +utf8 = (re->options & PCRE_UTF8) != 0; +anchored = (options & PCRE_ANCHORED) != 0 || (re->options & PCRE_ANCHORED) != 0; + +/* The remaining fixed data for passing around. */ + +match_block.start_code = (const uschar *)argument_re + + re->name_table_offset + re->name_count * re->name_entry_size; +match_block.start_subject = (const unsigned char *)subject; +match_block.end_subject = end_subject; +match_block.moptions = options; +match_block.poptions = re->options; + +/* Check a UTF-8 string if required. Unfortunately there's no way of passing +back the character offset. */ + +#ifdef SUPPORT_UTF8 +if (utf8 && (options & PCRE_NO_UTF8_CHECK) == 0) + { + if (_pcre_valid_utf8((uschar *)subject, length) >= 0) + return PCRE_ERROR_BADUTF8; + if (start_offset > 0 && start_offset < length) + { + int tb = ((uschar *)subject)[start_offset]; + if (tb > 127) + { + tb &= 0xc0; + if (tb != 0 && tb != 0xc0) return PCRE_ERROR_BADUTF8_OFFSET; + } + } + } +#endif + +/* If the exec call supplied NULL for tables, use the inbuilt ones. This +is a feature that makes it possible to save compiled regex and re-use them +in other programs later. */ + +if (match_block.tables == NULL) match_block.tables = _pcre_default_tables; + +/* The lower casing table and the "must be at the start of a line" flag are +used in a loop when finding where to start. */ + +lcc = match_block.tables + lcc_offset; +startline = (re->options & PCRE_STARTLINE) != 0; +firstline = (re->options & PCRE_FIRSTLINE) != 0; + +/* Set up the first character to match, if available. The first_byte value is +never set for an anchored regular expression, but the anchoring may be forced +at run time, so we have to test for anchoring. The first char may be unset for +an unanchored pattern, of course. If there's no first char and the pattern was +studied, there may be a bitmap of possible first characters. */ + +if (!anchored) + { + if ((re->options & PCRE_FIRSTSET) != 0) + { + first_byte = re->first_byte & 255; + if ((first_byte_caseless = ((re->first_byte & REQ_CASELESS) != 0)) == TRUE) + first_byte = lcc[first_byte]; + } + else + { + if (startline && study != NULL && + (study->options & PCRE_STUDY_MAPPED) != 0) + start_bits = study->start_bits; + } + } + +/* For anchored or unanchored matches, there may be a "last known required +character" set. */ + +if ((re->options & PCRE_REQCHSET) != 0) + { + req_byte = re->req_byte & 255; + req_byte_caseless = (re->req_byte & REQ_CASELESS) != 0; + req_byte2 = (match_block.tables + fcc_offset)[req_byte]; /* case flipped */ + } + +/* Call the main matching function, looping for a non-anchored regex after a +failed match. Unless restarting, optimize by moving to the first match +character if possible, when not anchored. Then unless wanting a partial match, +check for a required later character. */ + +for (;;) + { + int rc; + + if ((options & PCRE_DFA_RESTART) == 0) + { + const uschar *save_end_subject = end_subject; + + /* Advance to a unique first char if possible. If firstline is TRUE, the + start of the match is constrained to the first line of a multiline string. + Implement this by temporarily adjusting end_subject so that we stop scanning + at a newline. If the match fails at the newline, later code breaks this loop. + */ + + if (firstline) + { + const uschar *t = current_subject; + while (t < save_end_subject && *t != '\n') t++; + end_subject = t; + } + + if (first_byte >= 0) + { + if (first_byte_caseless) + while (current_subject < end_subject && + lcc[*current_subject] != first_byte) + current_subject++; + else + while (current_subject < end_subject && *current_subject != first_byte) + current_subject++; + } + + /* Or to just after \n for a multiline match if possible */ + + else if (startline) + { + if (current_subject > match_block.start_subject + start_offset) + { + while (current_subject < end_subject && current_subject[-1] != NEWLINE) + current_subject++; + } + } + + /* Or to a non-unique first char after study */ + + else if (start_bits != NULL) + { + while (current_subject < end_subject) + { + register unsigned int c = *current_subject; + if ((start_bits[c/8] & (1 << (c&7))) == 0) current_subject++; + else break; + } + } + + /* Restore fudged end_subject */ + + end_subject = save_end_subject; + } + + /* If req_byte is set, we know that that character must appear in the subject + for the match to succeed. If the first character is set, req_byte must be + later in the subject; otherwise the test starts at the match point. This + optimization can save a huge amount of work in patterns with nested unlimited + repeats that aren't going to match. Writing separate code for cased/caseless + versions makes it go faster, as does using an autoincrement and backing off + on a match. + + HOWEVER: when the subject string is very, very long, searching to its end can + take a long time, and give bad performance on quite ordinary patterns. This + showed up when somebody was matching /^C/ on a 32-megabyte string... so we + don't do this when the string is sufficiently long. + + ALSO: this processing is disabled when partial matching is requested. + */ + + if (req_byte >= 0 && + end_subject - current_subject < REQ_BYTE_MAX && + (options & PCRE_PARTIAL) == 0) + { + register const uschar *p = current_subject + ((first_byte >= 0)? 1 : 0); + + /* We don't need to repeat the search if we haven't yet reached the + place we found it at last time. */ + + if (p > req_byte_ptr) + { + if (req_byte_caseless) + { + while (p < end_subject) + { + register int pp = *p++; + if (pp == req_byte || pp == req_byte2) { p--; break; } + } + } + else + { + while (p < end_subject) + { + if (*p++ == req_byte) { p--; break; } + } + } + + /* If we can't find the required character, break the matching loop, + which will cause a return or PCRE_ERROR_NOMATCH. */ + + if (p >= end_subject) break; + + /* If we have found the required character, save the point where we + found it, so that we don't search again next time round the loop if + the start hasn't passed this character yet. */ + + req_byte_ptr = p; + } + } + + /* OK, now we can do the business */ + + rc = internal_dfa_exec( + &match_block, /* fixed match data */ + match_block.start_code, /* this subexpression's code */ + current_subject, /* where we currently are */ + start_offset, /* start offset in subject */ + offsets, /* offset vector */ + offsetcount, /* size of same */ + workspace, /* workspace vector */ + wscount, /* size of same */ + re->options & (PCRE_CASELESS|PCRE_MULTILINE|PCRE_DOTALL), /* ims flags */ + 0, /* function recurse level */ + 0); /* regex recurse level */ + + /* Anything other than "no match" means we are done, always; otherwise, carry + on only if not anchored. */ + + if (rc != PCRE_ERROR_NOMATCH || anchored) return rc; + + /* Advance to the next subject character unless we are at the end of a line + and firstline is set. */ + + if (firstline && *current_subject == NEWLINE) break; + current_subject++; + +#ifdef SUPPORT_UTF8 + if (utf8) + { + while (current_subject < end_subject && (*current_subject & 0xc0) == 0x80) + current_subject++; + } +#endif + + if (current_subject > end_subject) break; + } + +return PCRE_ERROR_NOMATCH; +} + +/* End of pcre_dfa_exec.c */ diff --git a/pcre_exec.c b/pcre_exec.c new file mode 100644 index 0000000..65173e2 --- /dev/null +++ b/pcre_exec.c @@ -0,0 +1,3632 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains pcre_exec(), the externally visible function that does +pattern matching using an NFA algorithm, trying to mimic Perl as closely as +possible. There are also some static supporting functions. */ + + +#include "pcre_internal.h" + + +/* Structure for building a chain of data that actually lives on the +stack, for holding the values of the subject pointer at the start of each +subpattern, so as to detect when an empty string has been matched by a +subpattern - to break infinite loops. When NO_RECURSE is set, these blocks +are on the heap, not on the stack. */ + +typedef struct eptrblock { + struct eptrblock *epb_prev; + const uschar *epb_saved_eptr; +} eptrblock; + +/* Flag bits for the match() function */ + +#define match_condassert 0x01 /* Called to check a condition assertion */ +#define match_isgroup 0x02 /* Set if start of bracketed group */ + +/* Non-error returns from the match() function. Error returns are externally +defined PCRE_ERROR_xxx codes, which are all negative. */ + +#define MATCH_MATCH 1 +#define MATCH_NOMATCH 0 + +/* Maximum number of ints of offset to save on the stack for recursive calls. +If the offset vector is bigger, malloc is used. This should be a multiple of 3, +because the offset vector is always a multiple of 3 long. */ + +#define REC_STACK_SAVE_MAX 30 + +/* Min and max values for the common repeats; for the maxima, 0 => infinity */ + +static const char rep_min[] = { 0, 0, 1, 1, 0, 0 }; +static const char rep_max[] = { 0, 0, 0, 0, 1, 1 }; + + + +#ifdef DEBUG +/************************************************* +* Debugging function to print chars * +*************************************************/ + +/* Print a sequence of chars in printable format, stopping at the end of the +subject if the requested. + +Arguments: + p points to characters + length number to print + is_subject TRUE if printing from within md->start_subject + md pointer to matching data block, if is_subject is TRUE + +Returns: nothing +*/ + +static void +pchars(const uschar *p, int length, BOOL is_subject, match_data *md) +{ +int c; +if (is_subject && length > md->end_subject - p) length = md->end_subject - p; +while (length-- > 0) + if (isprint(c = *(p++))) printf("%c", c); else printf("\\x%02x", c); +} +#endif + + + +/************************************************* +* Match a back-reference * +*************************************************/ + +/* If a back reference hasn't been set, the length that is passed is greater +than the number of characters left in the string, so the match fails. + +Arguments: + offset index into the offset vector + eptr points into the subject + length length to be matched + md points to match data block + ims the ims flags + +Returns: TRUE if matched +*/ + +static BOOL +match_ref(int offset, register const uschar *eptr, int length, match_data *md, + unsigned long int ims) +{ +const uschar *p = md->start_subject + md->offset_vector[offset]; + +#ifdef DEBUG +if (eptr >= md->end_subject) + printf("matching subject <null>"); +else + { + printf("matching subject "); + pchars(eptr, length, TRUE, md); + } +printf(" against backref "); +pchars(p, length, FALSE, md); +printf("\n"); +#endif + +/* Always fail if not enough characters left */ + +if (length > md->end_subject - eptr) return FALSE; + +/* Separate the caselesss case for speed */ + +if ((ims & PCRE_CASELESS) != 0) + { + while (length-- > 0) + if (md->lcc[*p++] != md->lcc[*eptr++]) return FALSE; + } +else + { while (length-- > 0) if (*p++ != *eptr++) return FALSE; } + +return TRUE; +} + + + +/*************************************************************************** +**************************************************************************** + RECURSION IN THE match() FUNCTION + +The match() function is highly recursive. Some regular expressions can cause +it to recurse thousands of times. I was writing for Unix, so I just let it +call itself recursively. This uses the stack for saving everything that has +to be saved for a recursive call. On Unix, the stack can be large, and this +works fine. + +It turns out that on non-Unix systems there are problems with programs that +use a lot of stack. (This despite the fact that every last chip has oodles +of memory these days, and techniques for extending the stack have been known +for decades.) So.... + +There is a fudge, triggered by defining NO_RECURSE, which avoids recursive +calls by keeping local variables that need to be preserved in blocks of memory +obtained from malloc instead instead of on the stack. Macros are used to +achieve this so that the actual code doesn't look very different to what it +always used to. +**************************************************************************** +***************************************************************************/ + + +/* These versions of the macros use the stack, as normal */ + +#ifndef NO_RECURSE +#define REGISTER register +#define RMATCH(rx,ra,rb,rc,rd,re,rf,rg) rx = match(ra,rb,rc,rd,re,rf,rg) +#define RRETURN(ra) return ra +#else + + +/* These versions of the macros manage a private stack on the heap. Note +that the rd argument of RMATCH isn't actually used. It's the md argument of +match(), which never changes. */ + +#define REGISTER + +#define RMATCH(rx,ra,rb,rc,rd,re,rf,rg)\ + {\ + heapframe *newframe = (pcre_stack_malloc)(sizeof(heapframe));\ + if (setjmp(frame->Xwhere) == 0)\ + {\ + newframe->Xeptr = ra;\ + newframe->Xecode = rb;\ + newframe->Xoffset_top = rc;\ + newframe->Xims = re;\ + newframe->Xeptrb = rf;\ + newframe->Xflags = rg;\ + newframe->Xprevframe = frame;\ + frame = newframe;\ + DPRINTF(("restarting from line %d\n", __LINE__));\ + goto HEAP_RECURSE;\ + }\ + else\ + {\ + DPRINTF(("longjumped back to line %d\n", __LINE__));\ + frame = md->thisframe;\ + rx = frame->Xresult;\ + }\ + } + +#define RRETURN(ra)\ + {\ + heapframe *newframe = frame;\ + frame = newframe->Xprevframe;\ + (pcre_stack_free)(newframe);\ + if (frame != NULL)\ + {\ + frame->Xresult = ra;\ + md->thisframe = frame;\ + longjmp(frame->Xwhere, 1);\ + }\ + return ra;\ + } + + +/* Structure for remembering the local variables in a private frame */ + +typedef struct heapframe { + struct heapframe *Xprevframe; + + /* Function arguments that may change */ + + const uschar *Xeptr; + const uschar *Xecode; + int Xoffset_top; + long int Xims; + eptrblock *Xeptrb; + int Xflags; + + /* Function local variables */ + + const uschar *Xcallpat; + const uschar *Xcharptr; + const uschar *Xdata; + const uschar *Xnext; + const uschar *Xpp; + const uschar *Xprev; + const uschar *Xsaved_eptr; + + recursion_info Xnew_recursive; + + BOOL Xcur_is_word; + BOOL Xcondition; + BOOL Xminimize; + BOOL Xprev_is_word; + + unsigned long int Xoriginal_ims; + +#ifdef SUPPORT_UCP + int Xprop_type; + int Xprop_fail_result; + int Xprop_category; + int Xprop_chartype; + int Xprop_othercase; + int Xprop_test_against; + int *Xprop_test_variable; +#endif + + int Xctype; + int Xfc; + int Xfi; + int Xlength; + int Xmax; + int Xmin; + int Xnumber; + int Xoffset; + int Xop; + int Xsave_capture_last; + int Xsave_offset1, Xsave_offset2, Xsave_offset3; + int Xstacksave[REC_STACK_SAVE_MAX]; + + eptrblock Xnewptrb; + + /* Place to pass back result, and where to jump back to */ + + int Xresult; + jmp_buf Xwhere; + +} heapframe; + +#endif + + +/*************************************************************************** +***************************************************************************/ + + + +/************************************************* +* Match from current position * +*************************************************/ + +/* On entry ecode points to the first opcode, and eptr to the first character +in the subject string, while eptrb holds the value of eptr at the start of the +last bracketed group - used for breaking infinite loops matching zero-length +strings. This function is called recursively in many circumstances. Whenever it +returns a negative (error) response, the outer incarnation must also return the +same response. + +Performance note: It might be tempting to extract commonly used fields from the +md structure (e.g. utf8, end_subject) into individual variables to improve +performance. Tests using gcc on a SPARC disproved this; in the first case, it +made performance worse. + +Arguments: + eptr pointer in subject + ecode position in code + offset_top current top pointer + md pointer to "static" info for the match + ims current /i, /m, and /s options + eptrb pointer to chain of blocks containing eptr at start of + brackets - for testing for empty matches + flags can contain + match_condassert - this is an assertion condition + match_isgroup - this is the start of a bracketed group + +Returns: MATCH_MATCH if matched ) these values are >= 0 + MATCH_NOMATCH if failed to match ) + a negative PCRE_ERROR_xxx value if aborted by an error condition + (e.g. stopped by recursion limit) +*/ + +static int +match(REGISTER const uschar *eptr, REGISTER const uschar *ecode, + int offset_top, match_data *md, unsigned long int ims, eptrblock *eptrb, + int flags) +{ +/* These variables do not need to be preserved over recursion in this function, +so they can be ordinary variables in all cases. Mark them with "register" +because they are used a lot in loops. */ + +register int rrc; /* Returns from recursive calls */ +register int i; /* Used for loops not involving calls to RMATCH() */ +register int c; /* Character values not kept over RMATCH() calls */ +register BOOL utf8; /* Local copy of UTF-8 flag for speed */ + +/* When recursion is not being used, all "local" variables that have to be +preserved over calls to RMATCH() are part of a "frame" which is obtained from +heap storage. Set up the top-level frame here; others are obtained from the +heap whenever RMATCH() does a "recursion". See the macro definitions above. */ + +#ifdef NO_RECURSE +heapframe *frame = (pcre_stack_malloc)(sizeof(heapframe)); +frame->Xprevframe = NULL; /* Marks the top level */ + +/* Copy in the original argument variables */ + +frame->Xeptr = eptr; +frame->Xecode = ecode; +frame->Xoffset_top = offset_top; +frame->Xims = ims; +frame->Xeptrb = eptrb; +frame->Xflags = flags; + +/* This is where control jumps back to to effect "recursion" */ + +HEAP_RECURSE: + +/* Macros make the argument variables come from the current frame */ + +#define eptr frame->Xeptr +#define ecode frame->Xecode +#define offset_top frame->Xoffset_top +#define ims frame->Xims +#define eptrb frame->Xeptrb +#define flags frame->Xflags + +/* Ditto for the local variables */ + +#ifdef SUPPORT_UTF8 +#define charptr frame->Xcharptr +#endif +#define callpat frame->Xcallpat +#define data frame->Xdata +#define next frame->Xnext +#define pp frame->Xpp +#define prev frame->Xprev +#define saved_eptr frame->Xsaved_eptr + +#define new_recursive frame->Xnew_recursive + +#define cur_is_word frame->Xcur_is_word +#define condition frame->Xcondition +#define minimize frame->Xminimize +#define prev_is_word frame->Xprev_is_word + +#define original_ims frame->Xoriginal_ims + +#ifdef SUPPORT_UCP +#define prop_type frame->Xprop_type +#define prop_fail_result frame->Xprop_fail_result +#define prop_category frame->Xprop_category +#define prop_chartype frame->Xprop_chartype +#define prop_othercase frame->Xprop_othercase +#define prop_test_against frame->Xprop_test_against +#define prop_test_variable frame->Xprop_test_variable +#endif + +#define ctype frame->Xctype +#define fc frame->Xfc +#define fi frame->Xfi +#define length frame->Xlength +#define max frame->Xmax +#define min frame->Xmin +#define number frame->Xnumber +#define offset frame->Xoffset +#define op frame->Xop +#define save_capture_last frame->Xsave_capture_last +#define save_offset1 frame->Xsave_offset1 +#define save_offset2 frame->Xsave_offset2 +#define save_offset3 frame->Xsave_offset3 +#define stacksave frame->Xstacksave + +#define newptrb frame->Xnewptrb + +/* When recursion is being used, local variables are allocated on the stack and +get preserved during recursion in the normal way. In this environment, fi and +i, and fc and c, can be the same variables. */ + +#else +#define fi i +#define fc c + + +#ifdef SUPPORT_UTF8 /* Many of these variables are used ony */ +const uschar *charptr; /* small blocks of the code. My normal */ +#endif /* style of coding would have declared */ +const uschar *callpat; /* them within each of those blocks. */ +const uschar *data; /* However, in order to accommodate the */ +const uschar *next; /* version of this code that uses an */ +const uschar *pp; /* external "stack" implemented on the */ +const uschar *prev; /* heap, it is easier to declare them */ +const uschar *saved_eptr; /* all here, so the declarations can */ + /* be cut out in a block. The only */ +recursion_info new_recursive; /* declarations within blocks below are */ + /* for variables that do not have to */ +BOOL cur_is_word; /* be preserved over a recursive call */ +BOOL condition; /* to RMATCH(). */ +BOOL minimize; +BOOL prev_is_word; + +unsigned long int original_ims; + +#ifdef SUPPORT_UCP +int prop_type; +int prop_fail_result; +int prop_category; +int prop_chartype; +int prop_othercase; +int prop_test_against; +int *prop_test_variable; +#endif + +int ctype; +int length; +int max; +int min; +int number; +int offset; +int op; +int save_capture_last; +int save_offset1, save_offset2, save_offset3; +int stacksave[REC_STACK_SAVE_MAX]; + +eptrblock newptrb; +#endif + +/* These statements are here to stop the compiler complaining about unitialized +variables. */ + +#ifdef SUPPORT_UCP +prop_fail_result = 0; +prop_test_against = 0; +prop_test_variable = NULL; +#endif + +/* OK, now we can get on with the real code of the function. Recursion is +specified by the macros RMATCH and RRETURN. When NO_RECURSE is *not* defined, +these just turn into a recursive call to match() and a "return", respectively. +However, RMATCH isn't like a function call because it's quite a complicated +macro. It has to be used in one particular way. This shouldn't, however, impact +performance when true recursion is being used. */ + +if (md->match_call_count++ >= md->match_limit) RRETURN(PCRE_ERROR_MATCHLIMIT); + +original_ims = ims; /* Save for resetting on ')' */ +utf8 = md->utf8; /* Local copy of the flag */ + +/* At the start of a bracketed group, add the current subject pointer to the +stack of such pointers, to be re-instated at the end of the group when we hit +the closing ket. When match() is called in other circumstances, we don't add to +this stack. */ + +if ((flags & match_isgroup) != 0) + { + newptrb.epb_prev = eptrb; + newptrb.epb_saved_eptr = eptr; + eptrb = &newptrb; + } + +/* Now start processing the operations. */ + +for (;;) + { + op = *ecode; + minimize = FALSE; + + /* For partial matching, remember if we ever hit the end of the subject after + matching at least one subject character. */ + + if (md->partial && + eptr >= md->end_subject && + eptr > md->start_match) + md->hitend = TRUE; + + /* Opening capturing bracket. If there is space in the offset vector, save + the current subject position in the working slot at the top of the vector. We + mustn't change the current values of the data slot, because they may be set + from a previous iteration of this group, and be referred to by a reference + inside the group. + + If the bracket fails to match, we need to restore this value and also the + values of the final offsets, in case they were set by a previous iteration of + the same bracket. + + If there isn't enough space in the offset vector, treat this as if it were a + non-capturing bracket. Don't worry about setting the flag for the error case + here; that is handled in the code for KET. */ + + if (op > OP_BRA) + { + number = op - OP_BRA; + + /* For extended extraction brackets (large number), we have to fish out the + number from a dummy opcode at the start. */ + + if (number > EXTRACT_BASIC_MAX) + number = GET2(ecode, 2+LINK_SIZE); + offset = number << 1; + +#ifdef DEBUG + printf("start bracket %d subject=", number); + pchars(eptr, 16, TRUE, md); + printf("\n"); +#endif + + if (offset < md->offset_max) + { + save_offset1 = md->offset_vector[offset]; + save_offset2 = md->offset_vector[offset+1]; + save_offset3 = md->offset_vector[md->offset_end - number]; + save_capture_last = md->capture_last; + + DPRINTF(("saving %d %d %d\n", save_offset1, save_offset2, save_offset3)); + md->offset_vector[md->offset_end - number] = eptr - md->start_subject; + + do + { + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, + match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + md->capture_last = save_capture_last; + ecode += GET(ecode, 1); + } + while (*ecode == OP_ALT); + + DPRINTF(("bracket %d failed\n", number)); + + md->offset_vector[offset] = save_offset1; + md->offset_vector[offset+1] = save_offset2; + md->offset_vector[md->offset_end - number] = save_offset3; + + RRETURN(MATCH_NOMATCH); + } + + /* Insufficient room for saving captured contents */ + + else op = OP_BRA; + } + + /* Other types of node can be handled by a switch */ + + switch(op) + { + case OP_BRA: /* Non-capturing bracket: optimized */ + DPRINTF(("start bracket 0\n")); + do + { + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, + match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + ecode += GET(ecode, 1); + } + while (*ecode == OP_ALT); + DPRINTF(("bracket 0 failed\n")); + RRETURN(MATCH_NOMATCH); + + /* Conditional group: compilation checked that there are no more than + two branches. If the condition is false, skipping the first branch takes us + past the end if there is only one branch, but that's OK because that is + exactly what going to the ket would do. */ + + case OP_COND: + if (ecode[LINK_SIZE+1] == OP_CREF) /* Condition extract or recurse test */ + { + offset = GET2(ecode, LINK_SIZE+2) << 1; /* Doubled ref number */ + condition = (offset == CREF_RECURSE * 2)? + (md->recursive != NULL) : + (offset < offset_top && md->offset_vector[offset] >= 0); + RMATCH(rrc, eptr, ecode + (condition? + (LINK_SIZE + 4) : (LINK_SIZE + 1 + GET(ecode, 1))), + offset_top, md, ims, eptrb, match_isgroup); + RRETURN(rrc); + } + + /* The condition is an assertion. Call match() to evaluate it - setting + the final argument TRUE causes it to stop at the end of an assertion. */ + + else + { + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, NULL, + match_condassert | match_isgroup); + if (rrc == MATCH_MATCH) + { + ecode += 1 + LINK_SIZE + GET(ecode, LINK_SIZE+2); + while (*ecode == OP_ALT) ecode += GET(ecode, 1); + } + else if (rrc != MATCH_NOMATCH) + { + RRETURN(rrc); /* Need braces because of following else */ + } + else ecode += GET(ecode, 1); + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, + match_isgroup); + RRETURN(rrc); + } + /* Control never reaches here */ + + /* Skip over conditional reference or large extraction number data if + encountered. */ + + case OP_CREF: + case OP_BRANUMBER: + ecode += 3; + break; + + /* End of the pattern. If we are in a recursion, we should restore the + offsets appropriately and continue from after the call. */ + + case OP_END: + if (md->recursive != NULL && md->recursive->group_num == 0) + { + recursion_info *rec = md->recursive; + DPRINTF(("Hit the end in a (?0) recursion\n")); + md->recursive = rec->prevrec; + memmove(md->offset_vector, rec->offset_save, + rec->saved_max * sizeof(int)); + md->start_match = rec->save_start; + ims = original_ims; + ecode = rec->after_call; + break; + } + + /* Otherwise, if PCRE_NOTEMPTY is set, fail if we have matched an empty + string - backtracking will then try other alternatives, if any. */ + + if (md->notempty && eptr == md->start_match) RRETURN(MATCH_NOMATCH); + md->end_match_ptr = eptr; /* Record where we ended */ + md->end_offset_top = offset_top; /* and how many extracts were taken */ + RRETURN(MATCH_MATCH); + + /* Change option settings */ + + case OP_OPT: + ims = ecode[1]; + ecode += 2; + DPRINTF(("ims set to %02lx\n", ims)); + break; + + /* Assertion brackets. Check the alternative branches in turn - the + matching won't pass the KET for an assertion. If any one branch matches, + the assertion is true. Lookbehind assertions have an OP_REVERSE item at the + start of each branch to move the current point backwards, so the code at + this level is identical to the lookahead case. */ + + case OP_ASSERT: + case OP_ASSERTBACK: + do + { + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, NULL, + match_isgroup); + if (rrc == MATCH_MATCH) break; + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + ecode += GET(ecode, 1); + } + while (*ecode == OP_ALT); + if (*ecode == OP_KET) RRETURN(MATCH_NOMATCH); + + /* If checking an assertion for a condition, return MATCH_MATCH. */ + + if ((flags & match_condassert) != 0) RRETURN(MATCH_MATCH); + + /* Continue from after the assertion, updating the offsets high water + mark, since extracts may have been taken during the assertion. */ + + do ecode += GET(ecode,1); while (*ecode == OP_ALT); + ecode += 1 + LINK_SIZE; + offset_top = md->end_offset_top; + continue; + + /* Negative assertion: all branches must fail to match */ + + case OP_ASSERT_NOT: + case OP_ASSERTBACK_NOT: + do + { + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, NULL, + match_isgroup); + if (rrc == MATCH_MATCH) RRETURN(MATCH_NOMATCH); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + ecode += GET(ecode,1); + } + while (*ecode == OP_ALT); + + if ((flags & match_condassert) != 0) RRETURN(MATCH_MATCH); + + ecode += 1 + LINK_SIZE; + continue; + + /* Move the subject pointer back. This occurs only at the start of + each branch of a lookbehind assertion. If we are too close to the start to + move back, this match function fails. When working with UTF-8 we move + back a number of characters, not bytes. */ + + case OP_REVERSE: +#ifdef SUPPORT_UTF8 + if (utf8) + { + c = GET(ecode,1); + for (i = 0; i < c; i++) + { + eptr--; + if (eptr < md->start_subject) RRETURN(MATCH_NOMATCH); + BACKCHAR(eptr) + } + } + else +#endif + + /* No UTF-8 support, or not in UTF-8 mode: count is byte count */ + + { + eptr -= GET(ecode,1); + if (eptr < md->start_subject) RRETURN(MATCH_NOMATCH); + } + + /* Skip to next op code */ + + ecode += 1 + LINK_SIZE; + break; + + /* The callout item calls an external function, if one is provided, passing + details of the match so far. This is mainly for debugging, though the + function is able to force a failure. */ + + case OP_CALLOUT: + if (pcre_callout != NULL) + { + pcre_callout_block cb; + cb.version = 1; /* Version 1 of the callout block */ + cb.callout_number = ecode[1]; + cb.offset_vector = md->offset_vector; + cb.subject = (const char *)md->start_subject; + cb.subject_length = md->end_subject - md->start_subject; + cb.start_match = md->start_match - md->start_subject; + cb.current_position = eptr - md->start_subject; + cb.pattern_position = GET(ecode, 2); + cb.next_item_length = GET(ecode, 2 + LINK_SIZE); + cb.capture_top = offset_top/2; + cb.capture_last = md->capture_last; + cb.callout_data = md->callout_data; + if ((rrc = (*pcre_callout)(&cb)) > 0) RRETURN(MATCH_NOMATCH); + if (rrc < 0) RRETURN(rrc); + } + ecode += 2 + 2*LINK_SIZE; + break; + + /* Recursion either matches the current regex, or some subexpression. The + offset data is the offset to the starting bracket from the start of the + whole pattern. (This is so that it works from duplicated subpatterns.) + + If there are any capturing brackets started but not finished, we have to + save their starting points and reinstate them after the recursion. However, + we don't know how many such there are (offset_top records the completed + total) so we just have to save all the potential data. There may be up to + 65535 such values, which is too large to put on the stack, but using malloc + for small numbers seems expensive. As a compromise, the stack is used when + there are no more than REC_STACK_SAVE_MAX values to store; otherwise malloc + is used. A problem is what to do if the malloc fails ... there is no way of + returning to the top level with an error. Save the top REC_STACK_SAVE_MAX + values on the stack, and accept that the rest may be wrong. + + There are also other values that have to be saved. We use a chained + sequence of blocks that actually live on the stack. Thanks to Robin Houston + for the original version of this logic. */ + + case OP_RECURSE: + { + callpat = md->start_code + GET(ecode, 1); + new_recursive.group_num = *callpat - OP_BRA; + + /* For extended extraction brackets (large number), we have to fish out + the number from a dummy opcode at the start. */ + + if (new_recursive.group_num > EXTRACT_BASIC_MAX) + new_recursive.group_num = GET2(callpat, 2+LINK_SIZE); + + /* Add to "recursing stack" */ + + new_recursive.prevrec = md->recursive; + md->recursive = &new_recursive; + + /* Find where to continue from afterwards */ + + ecode += 1 + LINK_SIZE; + new_recursive.after_call = ecode; + + /* Now save the offset data. */ + + new_recursive.saved_max = md->offset_end; + if (new_recursive.saved_max <= REC_STACK_SAVE_MAX) + new_recursive.offset_save = stacksave; + else + { + new_recursive.offset_save = + (int *)(pcre_malloc)(new_recursive.saved_max * sizeof(int)); + if (new_recursive.offset_save == NULL) RRETURN(PCRE_ERROR_NOMEMORY); + } + + memcpy(new_recursive.offset_save, md->offset_vector, + new_recursive.saved_max * sizeof(int)); + new_recursive.save_start = md->start_match; + md->start_match = eptr; + + /* OK, now we can do the recursion. For each top-level alternative we + restore the offset and recursion data. */ + + DPRINTF(("Recursing into group %d\n", new_recursive.group_num)); + do + { + RMATCH(rrc, eptr, callpat + 1 + LINK_SIZE, offset_top, md, ims, + eptrb, match_isgroup); + if (rrc == MATCH_MATCH) + { + md->recursive = new_recursive.prevrec; + if (new_recursive.offset_save != stacksave) + (pcre_free)(new_recursive.offset_save); + RRETURN(MATCH_MATCH); + } + else if (rrc != MATCH_NOMATCH) RRETURN(rrc); + + md->recursive = &new_recursive; + memcpy(md->offset_vector, new_recursive.offset_save, + new_recursive.saved_max * sizeof(int)); + callpat += GET(callpat, 1); + } + while (*callpat == OP_ALT); + + DPRINTF(("Recursion didn't match\n")); + md->recursive = new_recursive.prevrec; + if (new_recursive.offset_save != stacksave) + (pcre_free)(new_recursive.offset_save); + RRETURN(MATCH_NOMATCH); + } + /* Control never reaches here */ + + /* "Once" brackets are like assertion brackets except that after a match, + the point in the subject string is not moved back. Thus there can never be + a move back into the brackets. Friedl calls these "atomic" subpatterns. + Check the alternative branches in turn - the matching won't pass the KET + for this kind of subpattern. If any one branch matches, we carry on as at + the end of a normal bracket, leaving the subject pointer. */ + + case OP_ONCE: + { + prev = ecode; + saved_eptr = eptr; + + do + { + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, + eptrb, match_isgroup); + if (rrc == MATCH_MATCH) break; + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + ecode += GET(ecode,1); + } + while (*ecode == OP_ALT); + + /* If hit the end of the group (which could be repeated), fail */ + + if (*ecode != OP_ONCE && *ecode != OP_ALT) RRETURN(MATCH_NOMATCH); + + /* Continue as from after the assertion, updating the offsets high water + mark, since extracts may have been taken. */ + + do ecode += GET(ecode,1); while (*ecode == OP_ALT); + + offset_top = md->end_offset_top; + eptr = md->end_match_ptr; + + /* For a non-repeating ket, just continue at this level. This also + happens for a repeating ket if no characters were matched in the group. + This is the forcible breaking of infinite loops as implemented in Perl + 5.005. If there is an options reset, it will get obeyed in the normal + course of events. */ + + if (*ecode == OP_KET || eptr == saved_eptr) + { + ecode += 1+LINK_SIZE; + break; + } + + /* The repeating kets try the rest of the pattern or restart from the + preceding bracket, in the appropriate order. We need to reset any options + that changed within the bracket before re-running it, so check the next + opcode. */ + + if (ecode[1+LINK_SIZE] == OP_OPT) + { + ims = (ims & ~PCRE_IMS) | ecode[4]; + DPRINTF(("ims set to %02lx at group repeat\n", ims)); + } + + if (*ecode == OP_KETRMIN) + { + RMATCH(rrc, eptr, ecode + 1 + LINK_SIZE, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + else /* OP_KETRMAX */ + { + RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + RMATCH(rrc, eptr, ecode + 1+LINK_SIZE, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + } + RRETURN(MATCH_NOMATCH); + + /* An alternation is the end of a branch; scan along to find the end of the + bracketed group and go to there. */ + + case OP_ALT: + do ecode += GET(ecode,1); while (*ecode == OP_ALT); + break; + + /* BRAZERO and BRAMINZERO occur just before a bracket group, indicating + that it may occur zero times. It may repeat infinitely, or not at all - + i.e. it could be ()* or ()? in the pattern. Brackets with fixed upper + repeat limits are compiled as a number of copies, with the optional ones + preceded by BRAZERO or BRAMINZERO. */ + + case OP_BRAZERO: + { + next = ecode+1; + RMATCH(rrc, eptr, next, offset_top, md, ims, eptrb, match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + do next += GET(next,1); while (*next == OP_ALT); + ecode = next + 1+LINK_SIZE; + } + break; + + case OP_BRAMINZERO: + { + next = ecode+1; + do next += GET(next,1); while (*next == OP_ALT); + RMATCH(rrc, eptr, next + 1+LINK_SIZE, offset_top, md, ims, eptrb, + match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + ecode++; + } + break; + + /* End of a group, repeated or non-repeating. If we are at the end of + an assertion "group", stop matching and return MATCH_MATCH, but record the + current high water mark for use by positive assertions. Do this also + for the "once" (not-backup up) groups. */ + + case OP_KET: + case OP_KETRMIN: + case OP_KETRMAX: + { + prev = ecode - GET(ecode, 1); + saved_eptr = eptrb->epb_saved_eptr; + + /* Back up the stack of bracket start pointers. */ + + eptrb = eptrb->epb_prev; + + if (*prev == OP_ASSERT || *prev == OP_ASSERT_NOT || + *prev == OP_ASSERTBACK || *prev == OP_ASSERTBACK_NOT || + *prev == OP_ONCE) + { + md->end_match_ptr = eptr; /* For ONCE */ + md->end_offset_top = offset_top; + RRETURN(MATCH_MATCH); + } + + /* In all other cases except a conditional group we have to check the + group number back at the start and if necessary complete handling an + extraction by setting the offsets and bumping the high water mark. */ + + if (*prev != OP_COND) + { + number = *prev - OP_BRA; + + /* For extended extraction brackets (large number), we have to fish out + the number from a dummy opcode at the start. */ + + if (number > EXTRACT_BASIC_MAX) number = GET2(prev, 2+LINK_SIZE); + offset = number << 1; + +#ifdef DEBUG + printf("end bracket %d", number); + printf("\n"); +#endif + + /* Test for a numbered group. This includes groups called as a result + of recursion. Note that whole-pattern recursion is coded as a recurse + into group 0, so it won't be picked up here. Instead, we catch it when + the OP_END is reached. */ + + if (number > 0) + { + md->capture_last = number; + if (offset >= md->offset_max) md->offset_overflow = TRUE; else + { + md->offset_vector[offset] = + md->offset_vector[md->offset_end - number]; + md->offset_vector[offset+1] = eptr - md->start_subject; + if (offset_top <= offset) offset_top = offset + 2; + } + + /* Handle a recursively called group. Restore the offsets + appropriately and continue from after the call. */ + + if (md->recursive != NULL && md->recursive->group_num == number) + { + recursion_info *rec = md->recursive; + DPRINTF(("Recursion (%d) succeeded - continuing\n", number)); + md->recursive = rec->prevrec; + md->start_match = rec->save_start; + memcpy(md->offset_vector, rec->offset_save, + rec->saved_max * sizeof(int)); + ecode = rec->after_call; + ims = original_ims; + break; + } + } + } + + /* Reset the value of the ims flags, in case they got changed during + the group. */ + + ims = original_ims; + DPRINTF(("ims reset to %02lx\n", ims)); + + /* For a non-repeating ket, just continue at this level. This also + happens for a repeating ket if no characters were matched in the group. + This is the forcible breaking of infinite loops as implemented in Perl + 5.005. If there is an options reset, it will get obeyed in the normal + course of events. */ + + if (*ecode == OP_KET || eptr == saved_eptr) + { + ecode += 1 + LINK_SIZE; + break; + } + + /* The repeating kets try the rest of the pattern or restart from the + preceding bracket, in the appropriate order. */ + + if (*ecode == OP_KETRMIN) + { + RMATCH(rrc, eptr, ecode + 1+LINK_SIZE, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + else /* OP_KETRMAX */ + { + RMATCH(rrc, eptr, prev, offset_top, md, ims, eptrb, match_isgroup); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + RMATCH(rrc, eptr, ecode + 1+LINK_SIZE, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + } + + RRETURN(MATCH_NOMATCH); + + /* Start of subject unless notbol, or after internal newline if multiline */ + + case OP_CIRC: + if (md->notbol && eptr == md->start_subject) RRETURN(MATCH_NOMATCH); + if ((ims & PCRE_MULTILINE) != 0) + { + if (eptr != md->start_subject && eptr[-1] != NEWLINE) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + } + /* ... else fall through */ + + /* Start of subject assertion */ + + case OP_SOD: + if (eptr != md->start_subject) RRETURN(MATCH_NOMATCH); + ecode++; + break; + + /* Start of match assertion */ + + case OP_SOM: + if (eptr != md->start_subject + md->start_offset) RRETURN(MATCH_NOMATCH); + ecode++; + break; + + /* Assert before internal newline if multiline, or before a terminating + newline unless endonly is set, else end of subject unless noteol is set. */ + + case OP_DOLL: + if ((ims & PCRE_MULTILINE) != 0) + { + if (eptr < md->end_subject) + { if (*eptr != NEWLINE) RRETURN(MATCH_NOMATCH); } + else + { if (md->noteol) RRETURN(MATCH_NOMATCH); } + ecode++; + break; + } + else + { + if (md->noteol) RRETURN(MATCH_NOMATCH); + if (!md->endonly) + { + if (eptr < md->end_subject - 1 || + (eptr == md->end_subject - 1 && *eptr != NEWLINE)) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + } + } + /* ... else fall through */ + + /* End of subject assertion (\z) */ + + case OP_EOD: + if (eptr < md->end_subject) RRETURN(MATCH_NOMATCH); + ecode++; + break; + + /* End of subject or ending \n assertion (\Z) */ + + case OP_EODN: + if (eptr < md->end_subject - 1 || + (eptr == md->end_subject - 1 && *eptr != NEWLINE)) RRETURN(MATCH_NOMATCH); + ecode++; + break; + + /* Word boundary assertions */ + + case OP_NOT_WORD_BOUNDARY: + case OP_WORD_BOUNDARY: + { + + /* Find out if the previous and current characters are "word" characters. + It takes a bit more work in UTF-8 mode. Characters > 255 are assumed to + be "non-word" characters. */ + +#ifdef SUPPORT_UTF8 + if (utf8) + { + if (eptr == md->start_subject) prev_is_word = FALSE; else + { + const uschar *lastptr = eptr - 1; + while((*lastptr & 0xc0) == 0x80) lastptr--; + GETCHAR(c, lastptr); + prev_is_word = c < 256 && (md->ctypes[c] & ctype_word) != 0; + } + if (eptr >= md->end_subject) cur_is_word = FALSE; else + { + GETCHAR(c, eptr); + cur_is_word = c < 256 && (md->ctypes[c] & ctype_word) != 0; + } + } + else +#endif + + /* More streamlined when not in UTF-8 mode */ + + { + prev_is_word = (eptr != md->start_subject) && + ((md->ctypes[eptr[-1]] & ctype_word) != 0); + cur_is_word = (eptr < md->end_subject) && + ((md->ctypes[*eptr] & ctype_word) != 0); + } + + /* Now see if the situation is what we want */ + + if ((*ecode++ == OP_WORD_BOUNDARY)? + cur_is_word == prev_is_word : cur_is_word != prev_is_word) + RRETURN(MATCH_NOMATCH); + } + break; + + /* Match a single character type; inline for speed */ + + case OP_ANY: + if ((ims & PCRE_DOTALL) == 0 && eptr < md->end_subject && *eptr == NEWLINE) + RRETURN(MATCH_NOMATCH); + if (eptr++ >= md->end_subject) RRETURN(MATCH_NOMATCH); +#ifdef SUPPORT_UTF8 + if (utf8) + while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; +#endif + ecode++; + break; + + /* Match a single byte, even in UTF-8 mode. This opcode really does match + any byte, even newline, independent of the setting of PCRE_DOTALL. */ + + case OP_ANYBYTE: + if (eptr++ >= md->end_subject) RRETURN(MATCH_NOMATCH); + ecode++; + break; + + case OP_NOT_DIGIT: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + if ( +#ifdef SUPPORT_UTF8 + c < 256 && +#endif + (md->ctypes[c] & ctype_digit) != 0 + ) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + + case OP_DIGIT: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + if ( +#ifdef SUPPORT_UTF8 + c >= 256 || +#endif + (md->ctypes[c] & ctype_digit) == 0 + ) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + + case OP_NOT_WHITESPACE: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + if ( +#ifdef SUPPORT_UTF8 + c < 256 && +#endif + (md->ctypes[c] & ctype_space) != 0 + ) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + + case OP_WHITESPACE: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + if ( +#ifdef SUPPORT_UTF8 + c >= 256 || +#endif + (md->ctypes[c] & ctype_space) == 0 + ) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + + case OP_NOT_WORDCHAR: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + if ( +#ifdef SUPPORT_UTF8 + c < 256 && +#endif + (md->ctypes[c] & ctype_word) != 0 + ) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + + case OP_WORDCHAR: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + if ( +#ifdef SUPPORT_UTF8 + c >= 256 || +#endif + (md->ctypes[c] & ctype_word) == 0 + ) + RRETURN(MATCH_NOMATCH); + ecode++; + break; + +#ifdef SUPPORT_UCP + /* Check the next character by Unicode property. We will get here only + if the support is in the binary; otherwise a compile-time error occurs. */ + + case OP_PROP: + case OP_NOTPROP: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + { + int chartype, rqdtype; + int othercase; + int category = ucp_findchar(c, &chartype, &othercase); + + rqdtype = *(++ecode); + ecode++; + + if (rqdtype >= 128) + { + if ((rqdtype - 128 != category) == (op == OP_PROP)) + RRETURN(MATCH_NOMATCH); + } + else + { + if ((rqdtype != chartype) == (op == OP_PROP)) + RRETURN(MATCH_NOMATCH); + } + } + break; + + /* Match an extended Unicode sequence. We will get here only if the support + is in the binary; otherwise a compile-time error occurs. */ + + case OP_EXTUNI: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + { + int chartype; + int othercase; + int category = ucp_findchar(c, &chartype, &othercase); + if (category == ucp_M) RRETURN(MATCH_NOMATCH); + while (eptr < md->end_subject) + { + int len = 1; + if (!utf8) c = *eptr; else + { + GETCHARLEN(c, eptr, len); + } + category = ucp_findchar(c, &chartype, &othercase); + if (category != ucp_M) break; + eptr += len; + } + } + ecode++; + break; +#endif + + + /* Match a back reference, possibly repeatedly. Look past the end of the + item to see if there is repeat information following. The code is similar + to that for character classes, but repeated for efficiency. Then obey + similar code to character type repeats - written out again for speed. + However, if the referenced string is the empty string, always treat + it as matched, any number of times (otherwise there could be infinite + loops). */ + + case OP_REF: + { + offset = GET2(ecode, 1) << 1; /* Doubled ref number */ + ecode += 3; /* Advance past item */ + + /* If the reference is unset, set the length to be longer than the amount + of subject left; this ensures that every attempt at a match fails. We + can't just fail here, because of the possibility of quantifiers with zero + minima. */ + + length = (offset >= offset_top || md->offset_vector[offset] < 0)? + md->end_subject - eptr + 1 : + md->offset_vector[offset+1] - md->offset_vector[offset]; + + /* Set up for repetition, or handle the non-repeated case */ + + switch (*ecode) + { + case OP_CRSTAR: + case OP_CRMINSTAR: + case OP_CRPLUS: + case OP_CRMINPLUS: + case OP_CRQUERY: + case OP_CRMINQUERY: + c = *ecode++ - OP_CRSTAR; + minimize = (c & 1) != 0; + min = rep_min[c]; /* Pick up values from tables; */ + max = rep_max[c]; /* zero for max => infinity */ + if (max == 0) max = INT_MAX; + break; + + case OP_CRRANGE: + case OP_CRMINRANGE: + minimize = (*ecode == OP_CRMINRANGE); + min = GET2(ecode, 1); + max = GET2(ecode, 3); + if (max == 0) max = INT_MAX; + ecode += 5; + break; + + default: /* No repeat follows */ + if (!match_ref(offset, eptr, length, md, ims)) RRETURN(MATCH_NOMATCH); + eptr += length; + continue; /* With the main loop */ + } + + /* If the length of the reference is zero, just continue with the + main loop. */ + + if (length == 0) continue; + + /* First, ensure the minimum number of matches are present. We get back + the length of the reference string explicitly rather than passing the + address of eptr, so that eptr can be a register variable. */ + + for (i = 1; i <= min; i++) + { + if (!match_ref(offset, eptr, length, md, ims)) RRETURN(MATCH_NOMATCH); + eptr += length; + } + + /* If min = max, continue at the same level without recursion. + They are not both allowed to be zero. */ + + if (min == max) continue; + + /* If minimizing, keep trying and advancing the pointer */ + + if (minimize) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || !match_ref(offset, eptr, length, md, ims)) + RRETURN(MATCH_NOMATCH); + eptr += length; + } + /* Control never gets here */ + } + + /* If maximizing, find the longest string and work backwards */ + + else + { + pp = eptr; + for (i = min; i < max; i++) + { + if (!match_ref(offset, eptr, length, md, ims)) break; + eptr += length; + } + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + eptr -= length; + } + RRETURN(MATCH_NOMATCH); + } + } + /* Control never gets here */ + + + + /* Match a bit-mapped character class, possibly repeatedly. This op code is + used when all the characters in the class have values in the range 0-255, + and either the matching is caseful, or the characters are in the range + 0-127 when UTF-8 processing is enabled. The only difference between + OP_CLASS and OP_NCLASS occurs when a data character outside the range is + encountered. + + First, look past the end of the item to see if there is repeat information + following. Then obey similar code to character type repeats - written out + again for speed. */ + + case OP_NCLASS: + case OP_CLASS: + { + data = ecode + 1; /* Save for matching */ + ecode += 33; /* Advance past the item */ + + switch (*ecode) + { + case OP_CRSTAR: + case OP_CRMINSTAR: + case OP_CRPLUS: + case OP_CRMINPLUS: + case OP_CRQUERY: + case OP_CRMINQUERY: + c = *ecode++ - OP_CRSTAR; + minimize = (c & 1) != 0; + min = rep_min[c]; /* Pick up values from tables; */ + max = rep_max[c]; /* zero for max => infinity */ + if (max == 0) max = INT_MAX; + break; + + case OP_CRRANGE: + case OP_CRMINRANGE: + minimize = (*ecode == OP_CRMINRANGE); + min = GET2(ecode, 1); + max = GET2(ecode, 3); + if (max == 0) max = INT_MAX; + ecode += 5; + break; + + default: /* No repeat follows */ + min = max = 1; + break; + } + + /* First, ensure the minimum number of matches are present. */ + +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINC(c, eptr); + if (c > 255) + { + if (op == OP_CLASS) RRETURN(MATCH_NOMATCH); + } + else + { + if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); + } + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + c = *eptr++; + if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); + } + } + + /* If max == min we can continue with the main loop without the + need to recurse. */ + + if (min == max) continue; + + /* If minimizing, keep testing the rest of the expression and advancing + the pointer while it matches the class. */ + + if (minimize) + { +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINC(c, eptr); + if (c > 255) + { + if (op == OP_CLASS) RRETURN(MATCH_NOMATCH); + } + else + { + if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); + } + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + c = *eptr++; + if ((data[c/8] & (1 << (c&7))) == 0) RRETURN(MATCH_NOMATCH); + } + } + /* Control never gets here */ + } + + /* If maximizing, find the longest possible run, then work backwards. */ + + else + { + pp = eptr; + +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (c > 255) + { + if (op == OP_CLASS) break; + } + else + { + if ((data[c/8] & (1 << (c&7))) == 0) break; + } + eptr += len; + } + for (;;) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr-- == pp) break; /* Stop if tried at original pos */ + BACKCHAR(eptr); + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject) break; + c = *eptr; + if ((data[c/8] & (1 << (c&7))) == 0) break; + eptr++; + } + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + eptr--; + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + } + + RRETURN(MATCH_NOMATCH); + } + } + /* Control never gets here */ + + + /* Match an extended character class. This opcode is encountered only + in UTF-8 mode, because that's the only time it is compiled. */ + +#ifdef SUPPORT_UTF8 + case OP_XCLASS: + { + data = ecode + 1 + LINK_SIZE; /* Save for matching */ + ecode += GET(ecode, 1); /* Advance past the item */ + + switch (*ecode) + { + case OP_CRSTAR: + case OP_CRMINSTAR: + case OP_CRPLUS: + case OP_CRMINPLUS: + case OP_CRQUERY: + case OP_CRMINQUERY: + c = *ecode++ - OP_CRSTAR; + minimize = (c & 1) != 0; + min = rep_min[c]; /* Pick up values from tables; */ + max = rep_max[c]; /* zero for max => infinity */ + if (max == 0) max = INT_MAX; + break; + + case OP_CRRANGE: + case OP_CRMINRANGE: + minimize = (*ecode == OP_CRMINRANGE); + min = GET2(ecode, 1); + max = GET2(ecode, 3); + if (max == 0) max = INT_MAX; + ecode += 5; + break; + + default: /* No repeat follows */ + min = max = 1; + break; + } + + /* First, ensure the minimum number of matches are present. */ + + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINC(c, eptr); + if (!_pcre_xclass(c, data)) RRETURN(MATCH_NOMATCH); + } + + /* If max == min we can continue with the main loop without the + need to recurse. */ + + if (min == max) continue; + + /* If minimizing, keep testing the rest of the expression and advancing + the pointer while it matches the class. */ + + if (minimize) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINC(c, eptr); + if (!_pcre_xclass(c, data)) RRETURN(MATCH_NOMATCH); + } + /* Control never gets here */ + } + + /* If maximizing, find the longest possible run, then work backwards. */ + + else + { + pp = eptr; + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (!_pcre_xclass(c, data)) break; + eptr += len; + } + for(;;) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr-- == pp) break; /* Stop if tried at original pos */ + BACKCHAR(eptr) + } + RRETURN(MATCH_NOMATCH); + } + + /* Control never gets here */ + } +#endif /* End of XCLASS */ + + /* Match a single character, casefully */ + + case OP_CHAR: +#ifdef SUPPORT_UTF8 + if (utf8) + { + length = 1; + ecode++; + GETCHARLEN(fc, ecode, length); + if (length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + while (length-- > 0) if (*ecode++ != *eptr++) RRETURN(MATCH_NOMATCH); + } + else +#endif + + /* Non-UTF-8 mode */ + { + if (md->end_subject - eptr < 1) RRETURN(MATCH_NOMATCH); + if (ecode[1] != *eptr++) RRETURN(MATCH_NOMATCH); + ecode += 2; + } + break; + + /* Match a single character, caselessly */ + + case OP_CHARNC: +#ifdef SUPPORT_UTF8 + if (utf8) + { + length = 1; + ecode++; + GETCHARLEN(fc, ecode, length); + + if (length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + + /* If the pattern character's value is < 128, we have only one byte, and + can use the fast lookup table. */ + + if (fc < 128) + { + if (md->lcc[*ecode++] != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); + } + + /* Otherwise we must pick up the subject character */ + + else + { + int dc; + GETCHARINC(dc, eptr); + ecode += length; + + /* If we have Unicode property support, we can use it to test the other + case of the character, if there is one. The result of ucp_findchar() is + < 0 if the char isn't found, and othercase is returned as zero if there + isn't one. */ + + if (fc != dc) + { +#ifdef SUPPORT_UCP + int chartype; + int othercase; + if (ucp_findchar(fc, &chartype, &othercase) < 0 || dc != othercase) +#endif + RRETURN(MATCH_NOMATCH); + } + } + } + else +#endif /* SUPPORT_UTF8 */ + + /* Non-UTF-8 mode */ + { + if (md->end_subject - eptr < 1) RRETURN(MATCH_NOMATCH); + if (md->lcc[ecode[1]] != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); + ecode += 2; + } + break; + + /* Match a single character repeatedly; different opcodes share code. */ + + case OP_EXACT: + min = max = GET2(ecode, 1); + ecode += 3; + goto REPEATCHAR; + + case OP_UPTO: + case OP_MINUPTO: + min = 0; + max = GET2(ecode, 1); + minimize = *ecode == OP_MINUPTO; + ecode += 3; + goto REPEATCHAR; + + case OP_STAR: + case OP_MINSTAR: + case OP_PLUS: + case OP_MINPLUS: + case OP_QUERY: + case OP_MINQUERY: + c = *ecode++ - OP_STAR; + minimize = (c & 1) != 0; + min = rep_min[c]; /* Pick up values from tables; */ + max = rep_max[c]; /* zero for max => infinity */ + if (max == 0) max = INT_MAX; + + /* Common code for all repeated single-character matches. We can give + up quickly if there are fewer than the minimum number of characters left in + the subject. */ + + REPEATCHAR: +#ifdef SUPPORT_UTF8 + if (utf8) + { + length = 1; + charptr = ecode; + GETCHARLEN(fc, ecode, length); + if (min * length > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + ecode += length; + + /* Handle multibyte character matching specially here. There is + support for caseless matching if UCP support is present. */ + + if (length > 1) + { + int oclength = 0; + uschar occhars[8]; + +#ifdef SUPPORT_UCP + int othercase; + int chartype; + if ((ims & PCRE_CASELESS) != 0 && + ucp_findchar(fc, &chartype, &othercase) >= 0 && + othercase > 0) + oclength = _pcre_ord2utf8(othercase, occhars); +#endif /* SUPPORT_UCP */ + + for (i = 1; i <= min; i++) + { + if (memcmp(eptr, charptr, length) == 0) eptr += length; + /* Need braces because of following else */ + else if (oclength == 0) { RRETURN(MATCH_NOMATCH); } + else + { + if (memcmp(eptr, occhars, oclength) != 0) RRETURN(MATCH_NOMATCH); + eptr += oclength; + } + } + + if (min == max) continue; + + if (minimize) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + if (memcmp(eptr, charptr, length) == 0) eptr += length; + /* Need braces because of following else */ + else if (oclength == 0) { RRETURN(MATCH_NOMATCH); } + else + { + if (memcmp(eptr, occhars, oclength) != 0) RRETURN(MATCH_NOMATCH); + eptr += oclength; + } + } + /* Control never gets here */ + } + else + { + pp = eptr; + for (i = min; i < max; i++) + { + if (eptr > md->end_subject - length) break; + if (memcmp(eptr, charptr, length) == 0) eptr += length; + else if (oclength == 0) break; + else + { + if (memcmp(eptr, occhars, oclength) != 0) break; + eptr += oclength; + } + } + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + eptr -= length; + } + RRETURN(MATCH_NOMATCH); + } + /* Control never gets here */ + } + + /* If the length of a UTF-8 character is 1, we fall through here, and + obey the code as for non-UTF-8 characters below, though in this case the + value of fc will always be < 128. */ + } + else +#endif /* SUPPORT_UTF8 */ + + /* When not in UTF-8 mode, load a single-byte character. */ + { + if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + fc = *ecode++; + } + + /* The value of fc at this point is always less than 256, though we may or + may not be in UTF-8 mode. The code is duplicated for the caseless and + caseful cases, for speed, since matching characters is likely to be quite + common. First, ensure the minimum number of matches are present. If min = + max, continue at the same level without recursing. Otherwise, if + minimizing, keep trying the rest of the expression and advancing one + matching character if failing, up to the maximum. Alternatively, if + maximizing, find the maximum number of characters and work backwards. */ + + DPRINTF(("matching %c{%d,%d} against subject %.*s\n", fc, min, max, + max, eptr)); + + if ((ims & PCRE_CASELESS) != 0) + { + fc = md->lcc[fc]; + for (i = 1; i <= min; i++) + if (fc != md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); + if (min == max) continue; + if (minimize) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject || + fc != md->lcc[*eptr++]) + RRETURN(MATCH_NOMATCH); + } + /* Control never gets here */ + } + else + { + pp = eptr; + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || fc != md->lcc[*eptr]) break; + eptr++; + } + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + eptr--; + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + RRETURN(MATCH_NOMATCH); + } + /* Control never gets here */ + } + + /* Caseful comparisons (includes all multi-byte characters) */ + + else + { + for (i = 1; i <= min; i++) if (fc != *eptr++) RRETURN(MATCH_NOMATCH); + if (min == max) continue; + if (minimize) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject || fc != *eptr++) + RRETURN(MATCH_NOMATCH); + } + /* Control never gets here */ + } + else + { + pp = eptr; + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || fc != *eptr) break; + eptr++; + } + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + eptr--; + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + RRETURN(MATCH_NOMATCH); + } + } + /* Control never gets here */ + + /* Match a negated single one-byte character. The character we are + checking can be multibyte. */ + + case OP_NOT: + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + ecode++; + GETCHARINCTEST(c, eptr); + if ((ims & PCRE_CASELESS) != 0) + { +#ifdef SUPPORT_UTF8 + if (c < 256) +#endif + c = md->lcc[c]; + if (md->lcc[*ecode++] == c) RRETURN(MATCH_NOMATCH); + } + else + { + if (*ecode++ == c) RRETURN(MATCH_NOMATCH); + } + break; + + /* Match a negated single one-byte character repeatedly. This is almost a + repeat of the code for a repeated single character, but I haven't found a + nice way of commoning these up that doesn't require a test of the + positive/negative option for each character match. Maybe that wouldn't add + very much to the time taken, but character matching *is* what this is all + about... */ + + case OP_NOTEXACT: + min = max = GET2(ecode, 1); + ecode += 3; + goto REPEATNOTCHAR; + + case OP_NOTUPTO: + case OP_NOTMINUPTO: + min = 0; + max = GET2(ecode, 1); + minimize = *ecode == OP_NOTMINUPTO; + ecode += 3; + goto REPEATNOTCHAR; + + case OP_NOTSTAR: + case OP_NOTMINSTAR: + case OP_NOTPLUS: + case OP_NOTMINPLUS: + case OP_NOTQUERY: + case OP_NOTMINQUERY: + c = *ecode++ - OP_NOTSTAR; + minimize = (c & 1) != 0; + min = rep_min[c]; /* Pick up values from tables; */ + max = rep_max[c]; /* zero for max => infinity */ + if (max == 0) max = INT_MAX; + + /* Common code for all repeated single-byte matches. We can give up quickly + if there are fewer than the minimum number of bytes left in the + subject. */ + + REPEATNOTCHAR: + if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + fc = *ecode++; + + /* The code is duplicated for the caseless and caseful cases, for speed, + since matching characters is likely to be quite common. First, ensure the + minimum number of matches are present. If min = max, continue at the same + level without recursing. Otherwise, if minimizing, keep trying the rest of + the expression and advancing one matching character if failing, up to the + maximum. Alternatively, if maximizing, find the maximum number of + characters and work backwards. */ + + DPRINTF(("negative matching %c{%d,%d} against subject %.*s\n", fc, min, max, + max, eptr)); + + if ((ims & PCRE_CASELESS) != 0) + { + fc = md->lcc[fc]; + +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + register int d; + for (i = 1; i <= min; i++) + { + GETCHARINC(d, eptr); + if (d < 256) d = md->lcc[d]; + if (fc == d) RRETURN(MATCH_NOMATCH); + } + } + else +#endif + + /* Not UTF-8 mode */ + { + for (i = 1; i <= min; i++) + if (fc == md->lcc[*eptr++]) RRETURN(MATCH_NOMATCH); + } + + if (min == max) continue; + + if (minimize) + { +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + register int d; + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + GETCHARINC(d, eptr); + if (d < 256) d = md->lcc[d]; + if (fi >= max || eptr >= md->end_subject || fc == d) + RRETURN(MATCH_NOMATCH); + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject || fc == md->lcc[*eptr++]) + RRETURN(MATCH_NOMATCH); + } + } + /* Control never gets here */ + } + + /* Maximize case */ + + else + { + pp = eptr; + +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + register int d; + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(d, eptr, len); + if (d < 256) d = md->lcc[d]; + if (fc == d) break; + eptr += len; + } + for(;;) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr-- == pp) break; /* Stop if tried at original pos */ + BACKCHAR(eptr); + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || fc == md->lcc[*eptr]) break; + eptr++; + } + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + eptr--; + } + } + + RRETURN(MATCH_NOMATCH); + } + /* Control never gets here */ + } + + /* Caseful comparisons */ + + else + { +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + register int d; + for (i = 1; i <= min; i++) + { + GETCHARINC(d, eptr); + if (fc == d) RRETURN(MATCH_NOMATCH); + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (i = 1; i <= min; i++) + if (fc == *eptr++) RRETURN(MATCH_NOMATCH); + } + + if (min == max) continue; + + if (minimize) + { +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + register int d; + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + GETCHARINC(d, eptr); + if (fi >= max || eptr >= md->end_subject || fc == d) + RRETURN(MATCH_NOMATCH); + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject || fc == *eptr++) + RRETURN(MATCH_NOMATCH); + } + } + /* Control never gets here */ + } + + /* Maximize case */ + + else + { + pp = eptr; + +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + register int d; + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(d, eptr, len); + if (fc == d) break; + eptr += len; + } + for(;;) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr-- == pp) break; /* Stop if tried at original pos */ + BACKCHAR(eptr); + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || fc == *eptr) break; + eptr++; + } + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + eptr--; + } + } + + RRETURN(MATCH_NOMATCH); + } + } + /* Control never gets here */ + + /* Match a single character type repeatedly; several different opcodes + share code. This is very similar to the code for single characters, but we + repeat it in the interests of efficiency. */ + + case OP_TYPEEXACT: + min = max = GET2(ecode, 1); + minimize = TRUE; + ecode += 3; + goto REPEATTYPE; + + case OP_TYPEUPTO: + case OP_TYPEMINUPTO: + min = 0; + max = GET2(ecode, 1); + minimize = *ecode == OP_TYPEMINUPTO; + ecode += 3; + goto REPEATTYPE; + + case OP_TYPESTAR: + case OP_TYPEMINSTAR: + case OP_TYPEPLUS: + case OP_TYPEMINPLUS: + case OP_TYPEQUERY: + case OP_TYPEMINQUERY: + c = *ecode++ - OP_TYPESTAR; + minimize = (c & 1) != 0; + min = rep_min[c]; /* Pick up values from tables; */ + max = rep_max[c]; /* zero for max => infinity */ + if (max == 0) max = INT_MAX; + + /* Common code for all repeated single character type matches. Note that + in UTF-8 mode, '.' matches a character of any length, but for the other + character types, the valid characters are all one-byte long. */ + + REPEATTYPE: + ctype = *ecode++; /* Code for the character type */ + +#ifdef SUPPORT_UCP + if (ctype == OP_PROP || ctype == OP_NOTPROP) + { + prop_fail_result = ctype == OP_NOTPROP; + prop_type = *ecode++; + if (prop_type >= 128) + { + prop_test_against = prop_type - 128; + prop_test_variable = &prop_category; + } + else + { + prop_test_against = prop_type; + prop_test_variable = &prop_chartype; + } + } + else prop_type = -1; +#endif + + /* First, ensure the minimum number of matches are present. Use inline + code for maximizing the speed, and do the type test once at the start + (i.e. keep it out of the loop). Also we can test that there are at least + the minimum number of bytes before we start. This isn't as effective in + UTF-8 mode, but it does no harm. Separate the UTF-8 code completely as that + is tidier. Also separate the UCP code, which can be the same for both UTF-8 + and single-bytes. */ + + if (min > md->end_subject - eptr) RRETURN(MATCH_NOMATCH); + if (min > 0) + { +#ifdef SUPPORT_UCP + if (prop_type > 0) + { + for (i = 1; i <= min; i++) + { + GETCHARINC(c, eptr); + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if ((*prop_test_variable == prop_test_against) == prop_fail_result) + RRETURN(MATCH_NOMATCH); + } + } + + /* Match extended Unicode sequences. We will get here only if the + support is in the binary; otherwise a compile-time error occurs. */ + + else if (ctype == OP_EXTUNI) + { + for (i = 1; i <= min; i++) + { + GETCHARINCTEST(c, eptr); + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if (prop_category == ucp_M) RRETURN(MATCH_NOMATCH); + while (eptr < md->end_subject) + { + int len = 1; + if (!utf8) c = *eptr; else + { + GETCHARLEN(c, eptr, len); + } + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if (prop_category != ucp_M) break; + eptr += len; + } + } + } + + else +#endif /* SUPPORT_UCP */ + +/* Handle all other cases when the coding is UTF-8 */ + +#ifdef SUPPORT_UTF8 + if (utf8) switch(ctype) + { + case OP_ANY: + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject || + (*eptr++ == NEWLINE && (ims & PCRE_DOTALL) == 0)) + RRETURN(MATCH_NOMATCH); + while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; + } + break; + + case OP_ANYBYTE: + eptr += min; + break; + + case OP_NOT_DIGIT: + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINC(c, eptr); + if (c < 128 && (md->ctypes[c] & ctype_digit) != 0) + RRETURN(MATCH_NOMATCH); + } + break; + + case OP_DIGIT: + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject || + *eptr >= 128 || (md->ctypes[*eptr++] & ctype_digit) == 0) + RRETURN(MATCH_NOMATCH); + /* No need to skip more bytes - we know it's a 1-byte character */ + } + break; + + case OP_NOT_WHITESPACE: + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject || + (*eptr < 128 && (md->ctypes[*eptr++] & ctype_space) != 0)) + RRETURN(MATCH_NOMATCH); + while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; + } + break; + + case OP_WHITESPACE: + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject || + *eptr >= 128 || (md->ctypes[*eptr++] & ctype_space) == 0) + RRETURN(MATCH_NOMATCH); + /* No need to skip more bytes - we know it's a 1-byte character */ + } + break; + + case OP_NOT_WORDCHAR: + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject || + (*eptr < 128 && (md->ctypes[*eptr++] & ctype_word) != 0)) + RRETURN(MATCH_NOMATCH); + while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; + } + break; + + case OP_WORDCHAR: + for (i = 1; i <= min; i++) + { + if (eptr >= md->end_subject || + *eptr >= 128 || (md->ctypes[*eptr++] & ctype_word) == 0) + RRETURN(MATCH_NOMATCH); + /* No need to skip more bytes - we know it's a 1-byte character */ + } + break; + + default: + RRETURN(PCRE_ERROR_INTERNAL); + } /* End switch(ctype) */ + + else +#endif /* SUPPORT_UTF8 */ + + /* Code for the non-UTF-8 case for minimum matching of operators other + than OP_PROP and OP_NOTPROP. */ + + switch(ctype) + { + case OP_ANY: + if ((ims & PCRE_DOTALL) == 0) + { + for (i = 1; i <= min; i++) + if (*eptr++ == NEWLINE) RRETURN(MATCH_NOMATCH); + } + else eptr += min; + break; + + case OP_ANYBYTE: + eptr += min; + break; + + case OP_NOT_DIGIT: + for (i = 1; i <= min; i++) + if ((md->ctypes[*eptr++] & ctype_digit) != 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_DIGIT: + for (i = 1; i <= min; i++) + if ((md->ctypes[*eptr++] & ctype_digit) == 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_NOT_WHITESPACE: + for (i = 1; i <= min; i++) + if ((md->ctypes[*eptr++] & ctype_space) != 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_WHITESPACE: + for (i = 1; i <= min; i++) + if ((md->ctypes[*eptr++] & ctype_space) == 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_NOT_WORDCHAR: + for (i = 1; i <= min; i++) + if ((md->ctypes[*eptr++] & ctype_word) != 0) + RRETURN(MATCH_NOMATCH); + break; + + case OP_WORDCHAR: + for (i = 1; i <= min; i++) + if ((md->ctypes[*eptr++] & ctype_word) == 0) + RRETURN(MATCH_NOMATCH); + break; + + default: + RRETURN(PCRE_ERROR_INTERNAL); + } + } + + /* If min = max, continue at the same level without recursing */ + + if (min == max) continue; + + /* If minimizing, we have to test the rest of the pattern before each + subsequent match. Again, separate the UTF-8 case for speed, and also + separate the UCP cases. */ + + if (minimize) + { +#ifdef SUPPORT_UCP + if (prop_type > 0) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINC(c, eptr); + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if ((*prop_test_variable == prop_test_against) == prop_fail_result) + RRETURN(MATCH_NOMATCH); + } + } + + /* Match extended Unicode sequences. We will get here only if the + support is in the binary; otherwise a compile-time error occurs. */ + + else if (ctype == OP_EXTUNI) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + GETCHARINCTEST(c, eptr); + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if (prop_category == ucp_M) RRETURN(MATCH_NOMATCH); + while (eptr < md->end_subject) + { + int len = 1; + if (!utf8) c = *eptr; else + { + GETCHARLEN(c, eptr, len); + } + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if (prop_category != ucp_M) break; + eptr += len; + } + } + } + + else +#endif /* SUPPORT_UCP */ + +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + if (utf8) + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + + GETCHARINC(c, eptr); + switch(ctype) + { + case OP_ANY: + if ((ims & PCRE_DOTALL) == 0 && c == NEWLINE) RRETURN(MATCH_NOMATCH); + break; + + case OP_ANYBYTE: + break; + + case OP_NOT_DIGIT: + if (c < 256 && (md->ctypes[c] & ctype_digit) != 0) + RRETURN(MATCH_NOMATCH); + break; + + case OP_DIGIT: + if (c >= 256 || (md->ctypes[c] & ctype_digit) == 0) + RRETURN(MATCH_NOMATCH); + break; + + case OP_NOT_WHITESPACE: + if (c < 256 && (md->ctypes[c] & ctype_space) != 0) + RRETURN(MATCH_NOMATCH); + break; + + case OP_WHITESPACE: + if (c >= 256 || (md->ctypes[c] & ctype_space) == 0) + RRETURN(MATCH_NOMATCH); + break; + + case OP_NOT_WORDCHAR: + if (c < 256 && (md->ctypes[c] & ctype_word) != 0) + RRETURN(MATCH_NOMATCH); + break; + + case OP_WORDCHAR: + if (c >= 256 || (md->ctypes[c] & ctype_word) == 0) + RRETURN(MATCH_NOMATCH); + break; + + default: + RRETURN(PCRE_ERROR_INTERNAL); + } + } + } + else +#endif + /* Not UTF-8 mode */ + { + for (fi = min;; fi++) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (fi >= max || eptr >= md->end_subject) RRETURN(MATCH_NOMATCH); + c = *eptr++; + switch(ctype) + { + case OP_ANY: + if ((ims & PCRE_DOTALL) == 0 && c == NEWLINE) RRETURN(MATCH_NOMATCH); + break; + + case OP_ANYBYTE: + break; + + case OP_NOT_DIGIT: + if ((md->ctypes[c] & ctype_digit) != 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_DIGIT: + if ((md->ctypes[c] & ctype_digit) == 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_NOT_WHITESPACE: + if ((md->ctypes[c] & ctype_space) != 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_WHITESPACE: + if ((md->ctypes[c] & ctype_space) == 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_NOT_WORDCHAR: + if ((md->ctypes[c] & ctype_word) != 0) RRETURN(MATCH_NOMATCH); + break; + + case OP_WORDCHAR: + if ((md->ctypes[c] & ctype_word) == 0) RRETURN(MATCH_NOMATCH); + break; + + default: + RRETURN(PCRE_ERROR_INTERNAL); + } + } + } + /* Control never gets here */ + } + + /* If maximizing it is worth using inline code for speed, doing the type + test once at the start (i.e. keep it out of the loop). Again, keep the + UTF-8 and UCP stuff separate. */ + + else + { + pp = eptr; /* Remember where we started */ + +#ifdef SUPPORT_UCP + if (prop_type > 0) + { + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if ((*prop_test_variable == prop_test_against) == prop_fail_result) + break; + eptr+= len; + } + + /* eptr is now past the end of the maximum run */ + + for(;;) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr-- == pp) break; /* Stop if tried at original pos */ + BACKCHAR(eptr); + } + } + + /* Match extended Unicode sequences. We will get here only if the + support is in the binary; otherwise a compile-time error occurs. */ + + else if (ctype == OP_EXTUNI) + { + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject) break; + GETCHARINCTEST(c, eptr); + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if (prop_category == ucp_M) break; + while (eptr < md->end_subject) + { + int len = 1; + if (!utf8) c = *eptr; else + { + GETCHARLEN(c, eptr, len); + } + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if (prop_category != ucp_M) break; + eptr += len; + } + } + + /* eptr is now past the end of the maximum run */ + + for(;;) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr-- == pp) break; /* Stop if tried at original pos */ + for (;;) /* Move back over one extended */ + { + int len = 1; + BACKCHAR(eptr); + if (!utf8) c = *eptr; else + { + GETCHARLEN(c, eptr, len); + } + prop_category = ucp_findchar(c, &prop_chartype, &prop_othercase); + if (prop_category != ucp_M) break; + eptr--; + } + } + } + + else +#endif /* SUPPORT_UCP */ + +#ifdef SUPPORT_UTF8 + /* UTF-8 mode */ + + if (utf8) + { + switch(ctype) + { + case OP_ANY: + + /* Special code is required for UTF8, but when the maximum is unlimited + we don't need it, so we repeat the non-UTF8 code. This is probably + worth it, because .* is quite a common idiom. */ + + if (max < INT_MAX) + { + if ((ims & PCRE_DOTALL) == 0) + { + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || *eptr == NEWLINE) break; + eptr++; + while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; + } + } + else + { + for (i = min; i < max; i++) + { + eptr++; + while (eptr < md->end_subject && (*eptr & 0xc0) == 0x80) eptr++; + } + } + } + + /* Handle unlimited UTF-8 repeat */ + + else + { + if ((ims & PCRE_DOTALL) == 0) + { + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || *eptr == NEWLINE) break; + eptr++; + } + break; + } + else + { + c = max - min; + if (c > md->end_subject - eptr) c = md->end_subject - eptr; + eptr += c; + } + } + break; + + /* The byte case is the same as non-UTF8 */ + + case OP_ANYBYTE: + c = max - min; + if (c > md->end_subject - eptr) c = md->end_subject - eptr; + eptr += c; + break; + + case OP_NOT_DIGIT: + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (c < 256 && (md->ctypes[c] & ctype_digit) != 0) break; + eptr+= len; + } + break; + + case OP_DIGIT: + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (c >= 256 ||(md->ctypes[c] & ctype_digit) == 0) break; + eptr+= len; + } + break; + + case OP_NOT_WHITESPACE: + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (c < 256 && (md->ctypes[c] & ctype_space) != 0) break; + eptr+= len; + } + break; + + case OP_WHITESPACE: + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (c >= 256 ||(md->ctypes[c] & ctype_space) == 0) break; + eptr+= len; + } + break; + + case OP_NOT_WORDCHAR: + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (c < 256 && (md->ctypes[c] & ctype_word) != 0) break; + eptr+= len; + } + break; + + case OP_WORDCHAR: + for (i = min; i < max; i++) + { + int len = 1; + if (eptr >= md->end_subject) break; + GETCHARLEN(c, eptr, len); + if (c >= 256 || (md->ctypes[c] & ctype_word) == 0) break; + eptr+= len; + } + break; + + default: + RRETURN(PCRE_ERROR_INTERNAL); + } + + /* eptr is now past the end of the maximum run */ + + for(;;) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + if (eptr-- == pp) break; /* Stop if tried at original pos */ + BACKCHAR(eptr); + } + } + else +#endif + + /* Not UTF-8 mode */ + { + switch(ctype) + { + case OP_ANY: + if ((ims & PCRE_DOTALL) == 0) + { + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || *eptr == NEWLINE) break; + eptr++; + } + break; + } + /* For DOTALL case, fall through and treat as \C */ + + case OP_ANYBYTE: + c = max - min; + if (c > md->end_subject - eptr) c = md->end_subject - eptr; + eptr += c; + break; + + case OP_NOT_DIGIT: + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_digit) != 0) + break; + eptr++; + } + break; + + case OP_DIGIT: + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_digit) == 0) + break; + eptr++; + } + break; + + case OP_NOT_WHITESPACE: + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_space) != 0) + break; + eptr++; + } + break; + + case OP_WHITESPACE: + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_space) == 0) + break; + eptr++; + } + break; + + case OP_NOT_WORDCHAR: + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_word) != 0) + break; + eptr++; + } + break; + + case OP_WORDCHAR: + for (i = min; i < max; i++) + { + if (eptr >= md->end_subject || (md->ctypes[*eptr] & ctype_word) == 0) + break; + eptr++; + } + break; + + default: + RRETURN(PCRE_ERROR_INTERNAL); + } + + /* eptr is now past the end of the maximum run */ + + while (eptr >= pp) + { + RMATCH(rrc, eptr, ecode, offset_top, md, ims, eptrb, 0); + eptr--; + if (rrc != MATCH_NOMATCH) RRETURN(rrc); + } + } + + /* Get here if we can't make it match with any permitted repetitions */ + + RRETURN(MATCH_NOMATCH); + } + /* Control never gets here */ + + /* There's been some horrible disaster. Since all codes > OP_BRA are + for capturing brackets, and there shouldn't be any gaps between 0 and + OP_BRA, arrival here can only mean there is something seriously wrong + in the code above or the OP_xxx definitions. */ + + default: + DPRINTF(("Unknown opcode %d\n", *ecode)); + RRETURN(PCRE_ERROR_UNKNOWN_NODE); + } + + /* Do not stick any code in here without much thought; it is assumed + that "continue" in the code above comes out to here to repeat the main + loop. */ + + } /* End of main loop */ +/* Control never reaches here */ +} + + +/*************************************************************************** +**************************************************************************** + RECURSION IN THE match() FUNCTION + +Undefine all the macros that were defined above to handle this. */ + +#ifdef NO_RECURSE +#undef eptr +#undef ecode +#undef offset_top +#undef ims +#undef eptrb +#undef flags + +#undef callpat +#undef charptr +#undef data +#undef next +#undef pp +#undef prev +#undef saved_eptr + +#undef new_recursive + +#undef cur_is_word +#undef condition +#undef minimize +#undef prev_is_word + +#undef original_ims + +#undef ctype +#undef length +#undef max +#undef min +#undef number +#undef offset +#undef op +#undef save_capture_last +#undef save_offset1 +#undef save_offset2 +#undef save_offset3 +#undef stacksave + +#undef newptrb + +#endif + +/* These two are defined as macros in both cases */ + +#undef fc +#undef fi + +/*************************************************************************** +***************************************************************************/ + + + +/************************************************* +* Execute a Regular Expression * +*************************************************/ + +/* This function applies a compiled re to a subject string and picks out +portions of the string if it matches. Two elements in the vector are set for +each substring: the offsets to the start and end of the substring. + +Arguments: + argument_re points to the compiled expression + extra_data points to extra data or is NULL + subject points to the subject string + length length of subject string (may contain binary zeros) + start_offset where to start in the subject string + options option bits + offsets points to a vector of ints to be filled in with offsets + offsetcount the number of elements in the vector + +Returns: > 0 => success; value is the number of elements filled in + = 0 => success, but offsets is not big enough + -1 => failed to match + < -1 => some kind of unexpected problem +*/ + +EXPORT int +pcre_exec(const pcre *argument_re, const pcre_extra *extra_data, + const char *subject, int length, int start_offset, int options, int *offsets, + int offsetcount) +{ +int rc, resetcount, ocount; +int first_byte = -1; +int req_byte = -1; +int req_byte2 = -1; +unsigned long int ims = 0; +BOOL using_temporary_offsets = FALSE; +BOOL anchored; +BOOL startline; +BOOL firstline; +BOOL first_byte_caseless = FALSE; +BOOL req_byte_caseless = FALSE; +match_data match_block; +const uschar *tables; +const uschar *start_bits = NULL; +const uschar *start_match = (const uschar *)subject + start_offset; +const uschar *end_subject; +const uschar *req_byte_ptr = start_match - 1; + +pcre_study_data internal_study; +const pcre_study_data *study; + +real_pcre internal_re; +const real_pcre *external_re = (const real_pcre *)argument_re; +const real_pcre *re = external_re; + +/* Plausibility checks */ + +if ((options & ~PUBLIC_EXEC_OPTIONS) != 0) return PCRE_ERROR_BADOPTION; +if (re == NULL || subject == NULL || + (offsets == NULL && offsetcount > 0)) return PCRE_ERROR_NULL; +if (offsetcount < 0) return PCRE_ERROR_BADCOUNT; + +/* Fish out the optional data from the extra_data structure, first setting +the default values. */ + +study = NULL; +match_block.match_limit = MATCH_LIMIT; +match_block.callout_data = NULL; + +/* The table pointer is always in native byte order. */ + +tables = external_re->tables; + +if (extra_data != NULL) + { + register unsigned int flags = extra_data->flags; + if ((flags & PCRE_EXTRA_STUDY_DATA) != 0) + study = (const pcre_study_data *)extra_data->study_data; + if ((flags & PCRE_EXTRA_MATCH_LIMIT) != 0) + match_block.match_limit = extra_data->match_limit; + if ((flags & PCRE_EXTRA_CALLOUT_DATA) != 0) + match_block.callout_data = extra_data->callout_data; + if ((flags & PCRE_EXTRA_TABLES) != 0) tables = extra_data->tables; + } + +/* If the exec call supplied NULL for tables, use the inbuilt ones. This +is a feature that makes it possible to save compiled regex and re-use them +in other programs later. */ + +if (tables == NULL) tables = _pcre_default_tables; + +/* Check that the first field in the block is the magic number. If it is not, +test for a regex that was compiled on a host of opposite endianness. If this is +the case, flipped values are put in internal_re and internal_study if there was +study data too. */ + +if (re->magic_number != MAGIC_NUMBER) + { + re = _pcre_try_flipped(re, &internal_re, study, &internal_study); + if (re == NULL) return PCRE_ERROR_BADMAGIC; + if (study != NULL) study = &internal_study; + } + +/* Set up other data */ + +anchored = ((re->options | options) & PCRE_ANCHORED) != 0; +startline = (re->options & PCRE_STARTLINE) != 0; +firstline = (re->options & PCRE_FIRSTLINE) != 0; + +/* The code starts after the real_pcre block and the capture name table. */ + +match_block.start_code = (const uschar *)external_re + re->name_table_offset + + re->name_count * re->name_entry_size; + +match_block.start_subject = (const uschar *)subject; +match_block.start_offset = start_offset; +match_block.end_subject = match_block.start_subject + length; +end_subject = match_block.end_subject; + +match_block.endonly = (re->options & PCRE_DOLLAR_ENDONLY) != 0; +match_block.utf8 = (re->options & PCRE_UTF8) != 0; + +match_block.notbol = (options & PCRE_NOTBOL) != 0; +match_block.noteol = (options & PCRE_NOTEOL) != 0; +match_block.notempty = (options & PCRE_NOTEMPTY) != 0; +match_block.partial = (options & PCRE_PARTIAL) != 0; +match_block.hitend = FALSE; + +match_block.recursive = NULL; /* No recursion at top level */ + +match_block.lcc = tables + lcc_offset; +match_block.ctypes = tables + ctypes_offset; + +/* Partial matching is supported only for a restricted set of regexes at the +moment. */ + +if (match_block.partial && (re->options & PCRE_NOPARTIAL) != 0) + return PCRE_ERROR_BADPARTIAL; + +/* Check a UTF-8 string if required. Unfortunately there's no way of passing +back the character offset. */ + +#ifdef SUPPORT_UTF8 +if (match_block.utf8 && (options & PCRE_NO_UTF8_CHECK) == 0) + { + if (_pcre_valid_utf8((uschar *)subject, length) >= 0) + return PCRE_ERROR_BADUTF8; + if (start_offset > 0 && start_offset < length) + { + int tb = ((uschar *)subject)[start_offset]; + if (tb > 127) + { + tb &= 0xc0; + if (tb != 0 && tb != 0xc0) return PCRE_ERROR_BADUTF8_OFFSET; + } + } + } +#endif + +/* The ims options can vary during the matching as a result of the presence +of (?ims) items in the pattern. They are kept in a local variable so that +restoring at the exit of a group is easy. */ + +ims = re->options & (PCRE_CASELESS|PCRE_MULTILINE|PCRE_DOTALL); + +/* If the expression has got more back references than the offsets supplied can +hold, we get a temporary chunk of working store to use during the matching. +Otherwise, we can use the vector supplied, rounding down its size to a multiple +of 3. */ + +ocount = offsetcount - (offsetcount % 3); + +if (re->top_backref > 0 && re->top_backref >= ocount/3) + { + ocount = re->top_backref * 3 + 3; + match_block.offset_vector = (int *)(pcre_malloc)(ocount * sizeof(int)); + if (match_block.offset_vector == NULL) return PCRE_ERROR_NOMEMORY; + using_temporary_offsets = TRUE; + DPRINTF(("Got memory to hold back references\n")); + } +else match_block.offset_vector = offsets; + +match_block.offset_end = ocount; +match_block.offset_max = (2*ocount)/3; +match_block.offset_overflow = FALSE; +match_block.capture_last = -1; + +/* Compute the minimum number of offsets that we need to reset each time. Doing +this makes a huge difference to execution time when there aren't many brackets +in the pattern. */ + +resetcount = 2 + re->top_bracket * 2; +if (resetcount > offsetcount) resetcount = ocount; + +/* Reset the working variable associated with each extraction. These should +never be used unless previously set, but they get saved and restored, and so we +initialize them to avoid reading uninitialized locations. */ + +if (match_block.offset_vector != NULL) + { + register int *iptr = match_block.offset_vector + ocount; + register int *iend = iptr - resetcount/2 + 1; + while (--iptr >= iend) *iptr = -1; + } + +/* Set up the first character to match, if available. The first_byte value is +never set for an anchored regular expression, but the anchoring may be forced +at run time, so we have to test for anchoring. The first char may be unset for +an unanchored pattern, of course. If there's no first char and the pattern was +studied, there may be a bitmap of possible first characters. */ + +if (!anchored) + { + if ((re->options & PCRE_FIRSTSET) != 0) + { + first_byte = re->first_byte & 255; + if ((first_byte_caseless = ((re->first_byte & REQ_CASELESS) != 0)) == TRUE) + first_byte = match_block.lcc[first_byte]; + } + else + if (!startline && study != NULL && + (study->options & PCRE_STUDY_MAPPED) != 0) + start_bits = study->start_bits; + } + +/* For anchored or unanchored matches, there may be a "last known required +character" set. */ + +if ((re->options & PCRE_REQCHSET) != 0) + { + req_byte = re->req_byte & 255; + req_byte_caseless = (re->req_byte & REQ_CASELESS) != 0; + req_byte2 = (tables + fcc_offset)[req_byte]; /* case flipped */ + } + +/* Loop for handling unanchored repeated matching attempts; for anchored regexs +the loop runs just once. */ + +do + { + const uschar *save_end_subject = end_subject; + + /* Reset the maximum number of extractions we might see. */ + + if (match_block.offset_vector != NULL) + { + register int *iptr = match_block.offset_vector; + register int *iend = iptr + resetcount; + while (iptr < iend) *iptr++ = -1; + } + + /* Advance to a unique first char if possible. If firstline is TRUE, the + start of the match is constrained to the first line of a multiline string. + Implement this by temporarily adjusting end_subject so that we stop scanning + at a newline. If the match fails at the newline, later code breaks this loop. + */ + + if (firstline) + { + const uschar *t = start_match; + while (t < save_end_subject && *t != '\n') t++; + end_subject = t; + } + + /* Now test for a unique first byte */ + + if (first_byte >= 0) + { + if (first_byte_caseless) + while (start_match < end_subject && + match_block.lcc[*start_match] != first_byte) + start_match++; + else + while (start_match < end_subject && *start_match != first_byte) + start_match++; + } + + /* Or to just after \n for a multiline match if possible */ + + else if (startline) + { + if (start_match > match_block.start_subject + start_offset) + { + while (start_match < end_subject && start_match[-1] != NEWLINE) + start_match++; + } + } + + /* Or to a non-unique first char after study */ + + else if (start_bits != NULL) + { + while (start_match < end_subject) + { + register unsigned int c = *start_match; + if ((start_bits[c/8] & (1 << (c&7))) == 0) start_match++; else break; + } + } + + /* Restore fudged end_subject */ + + end_subject = save_end_subject; + +#ifdef DEBUG /* Sigh. Some compilers never learn. */ + printf(">>>> Match against: "); + pchars(start_match, end_subject - start_match, TRUE, &match_block); + printf("\n"); +#endif + + /* If req_byte is set, we know that that character must appear in the subject + for the match to succeed. If the first character is set, req_byte must be + later in the subject; otherwise the test starts at the match point. This + optimization can save a huge amount of backtracking in patterns with nested + unlimited repeats that aren't going to match. Writing separate code for + cased/caseless versions makes it go faster, as does using an autoincrement + and backing off on a match. + + HOWEVER: when the subject string is very, very long, searching to its end can + take a long time, and give bad performance on quite ordinary patterns. This + showed up when somebody was matching /^C/ on a 32-megabyte string... so we + don't do this when the string is sufficiently long. + + ALSO: this processing is disabled when partial matching is requested. + */ + + if (req_byte >= 0 && + end_subject - start_match < REQ_BYTE_MAX && + !match_block.partial) + { + register const uschar *p = start_match + ((first_byte >= 0)? 1 : 0); + + /* We don't need to repeat the search if we haven't yet reached the + place we found it at last time. */ + + if (p > req_byte_ptr) + { + if (req_byte_caseless) + { + while (p < end_subject) + { + register int pp = *p++; + if (pp == req_byte || pp == req_byte2) { p--; break; } + } + } + else + { + while (p < end_subject) + { + if (*p++ == req_byte) { p--; break; } + } + } + + /* If we can't find the required character, break the matching loop */ + + if (p >= end_subject) break; + + /* If we have found the required character, save the point where we + found it, so that we don't search again next time round the loop if + the start hasn't passed this character yet. */ + + req_byte_ptr = p; + } + } + + /* When a match occurs, substrings will be set for all internal extractions; + we just need to set up the whole thing as substring 0 before returning. If + there were too many extractions, set the return code to zero. In the case + where we had to get some local store to hold offsets for backreferences, copy + those back references that we can. In this case there need not be overflow + if certain parts of the pattern were not used. */ + + match_block.start_match = start_match; + match_block.match_call_count = 0; + + rc = match(start_match, match_block.start_code, 2, &match_block, ims, NULL, + match_isgroup); + + /* When the result is no match, if the subject's first character was a + newline and the PCRE_FIRSTLINE option is set, break (which will return + PCRE_ERROR_NOMATCH). The option requests that a match occur before the first + newline in the subject. Otherwise, advance the pointer to the next character + and continue - but the continuation will actually happen only when the + pattern is not anchored. */ + + if (rc == MATCH_NOMATCH) + { + if (firstline && *start_match == NEWLINE) break; + start_match++; +#ifdef SUPPORT_UTF8 + if (match_block.utf8) + while(start_match < end_subject && (*start_match & 0xc0) == 0x80) + start_match++; +#endif + continue; + } + + if (rc != MATCH_MATCH) + { + DPRINTF((">>>> error: returning %d\n", rc)); + return rc; + } + + /* We have a match! Copy the offset information from temporary store if + necessary */ + + if (using_temporary_offsets) + { + if (offsetcount >= 4) + { + memcpy(offsets + 2, match_block.offset_vector + 2, + (offsetcount - 2) * sizeof(int)); + DPRINTF(("Copied offsets from temporary memory\n")); + } + if (match_block.end_offset_top > offsetcount) + match_block.offset_overflow = TRUE; + + DPRINTF(("Freeing temporary memory\n")); + (pcre_free)(match_block.offset_vector); + } + + rc = match_block.offset_overflow? 0 : match_block.end_offset_top/2; + + if (offsetcount < 2) rc = 0; else + { + offsets[0] = start_match - match_block.start_subject; + offsets[1] = match_block.end_match_ptr - match_block.start_subject; + } + + DPRINTF((">>>> returning %d\n", rc)); + return rc; + } + +/* This "while" is the end of the "do" above */ + +while (!anchored && start_match <= end_subject); + +if (using_temporary_offsets) + { + DPRINTF(("Freeing temporary memory\n")); + (pcre_free)(match_block.offset_vector); + } + +if (match_block.partial && match_block.hitend) + { + DPRINTF((">>>> returning PCRE_ERROR_PARTIAL\n")); + return PCRE_ERROR_PARTIAL; + } +else + { + DPRINTF((">>>> returning PCRE_ERROR_NOMATCH\n")); + return PCRE_ERROR_NOMATCH; + } +} + +/* End of pcre_exec.c */ diff --git a/pcre_fullinfo.c b/pcre_fullinfo.c new file mode 100644 index 0000000..ac80e65 --- /dev/null +++ b/pcre_fullinfo.c @@ -0,0 +1,149 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/*PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains the external function pcre_fullinfo(), which returns +information about a compiled pattern. */ + + +#include "pcre_internal.h" + + +/************************************************* +* Return info about compiled pattern * +*************************************************/ + +/* This is a newer "info" function which has an extensible interface so +that additional items can be added compatibly. + +Arguments: + argument_re points to compiled code + extra_data points extra data, or NULL + what what information is required + where where to put the information + +Returns: 0 if data returned, negative on error +*/ + +EXPORT int +pcre_fullinfo(const pcre *argument_re, const pcre_extra *extra_data, int what, + void *where) +{ +real_pcre internal_re; +pcre_study_data internal_study; +const real_pcre *re = (const real_pcre *)argument_re; +const pcre_study_data *study = NULL; + +if (re == NULL || where == NULL) return PCRE_ERROR_NULL; + +if (extra_data != NULL && (extra_data->flags & PCRE_EXTRA_STUDY_DATA) != 0) + study = (const pcre_study_data *)extra_data->study_data; + +if (re->magic_number != MAGIC_NUMBER) + { + re = _pcre_try_flipped(re, &internal_re, study, &internal_study); + if (re == NULL) return PCRE_ERROR_BADMAGIC; + if (study != NULL) study = &internal_study; + } + +switch (what) + { + case PCRE_INFO_OPTIONS: + *((unsigned long int *)where) = re->options & PUBLIC_OPTIONS; + break; + + case PCRE_INFO_SIZE: + *((size_t *)where) = re->size; + break; + + case PCRE_INFO_STUDYSIZE: + *((size_t *)where) = (study == NULL)? 0 : study->size; + break; + + case PCRE_INFO_CAPTURECOUNT: + *((int *)where) = re->top_bracket; + break; + + case PCRE_INFO_BACKREFMAX: + *((int *)where) = re->top_backref; + break; + + case PCRE_INFO_FIRSTBYTE: + *((int *)where) = + ((re->options & PCRE_FIRSTSET) != 0)? re->first_byte : + ((re->options & PCRE_STARTLINE) != 0)? -1 : -2; + break; + + /* Make sure we pass back the pointer to the bit vector in the external + block, not the internal copy (with flipped integer fields). */ + + case PCRE_INFO_FIRSTTABLE: + *((const uschar **)where) = + (study != NULL && (study->options & PCRE_STUDY_MAPPED) != 0)? + ((const pcre_study_data *)extra_data->study_data)->start_bits : NULL; + break; + + case PCRE_INFO_LASTLITERAL: + *((int *)where) = + ((re->options & PCRE_REQCHSET) != 0)? re->req_byte : -1; + break; + + case PCRE_INFO_NAMEENTRYSIZE: + *((int *)where) = re->name_entry_size; + break; + + case PCRE_INFO_NAMECOUNT: + *((int *)where) = re->name_count; + break; + + case PCRE_INFO_NAMETABLE: + *((const uschar **)where) = (const uschar *)re + re->name_table_offset; + break; + + case PCRE_INFO_DEFAULT_TABLES: + *((const uschar **)where) = (const uschar *)(_pcre_default_tables); + break; + + default: return PCRE_ERROR_BADOPTION; + } + +return 0; +} + +/* End of pcre_fullinfo.c */ @@ -2,14 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -This is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. See -the file Tech.Notes for some information on the internals. +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2003 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -40,15 +37,13 @@ POSSIBILITY OF SUCH DAMAGE. ----------------------------------------------------------------------------- */ + /* This module contains some convenience functions for extracting substrings from the subject string after a regex match has succeeded. The original idea for these functions came from Scott Wimer. */ -/* Include the internals header, which itself includes Standard C headers plus -the external pcre header. */ - -#include "internal.h" +#include "pcre_internal.h" /************************************************* @@ -354,4 +349,4 @@ pcre_free_substring(const char *pointer) (pcre_free)((void *)pointer); } -/* End of get.c */ +/* End of pcre_get.c */ diff --git a/pcre_globals.c b/pcre_globals.c new file mode 100644 index 0000000..1a83980 --- /dev/null +++ b/pcre_globals.c @@ -0,0 +1,69 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains global variables that are exported by the PCRE library. +PCRE is thread-clean and doesn't use any global variables in the normal sense. +However, it calls memory allocation and freeing functions via the four +indirections below, and it can optionally do callouts, using the fifth +indirection. These values can be changed by the caller, but are shared between +all threads. However, when compiling for Virtual Pascal, things are done +differently, and global variables are not used (see pcre.in). */ + + +#include "pcre_internal.h" + + +#ifndef VPCOMPAT +#ifdef __cplusplus +extern "C" void *(*pcre_malloc)(size_t) = malloc; +extern "C" void (*pcre_free)(void *) = free; +extern "C" void *(*pcre_stack_malloc)(size_t) = malloc; +extern "C" void (*pcre_stack_free)(void *) = free; +extern "C" int (*pcre_callout)(pcre_callout_block *) = NULL; +#else +void *(*pcre_malloc)(size_t) = malloc; +void (*pcre_free)(void *) = free; +void *(*pcre_stack_malloc)(size_t) = malloc; +void (*pcre_stack_free)(void *) = free; +int (*pcre_callout)(pcre_callout_block *) = NULL; +#endif +#endif + +/* End of pcre_globals.c */ diff --git a/pcre_info.c b/pcre_info.c new file mode 100644 index 0000000..228949d --- /dev/null +++ b/pcre_info.c @@ -0,0 +1,89 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains the external function pcre_info(), which gives some +information about a compiled pattern. However, use of this function is now +deprecated, as it has been superseded by pcre_fullinfo(). */ + + +#include "pcre_internal.h" + + +/************************************************* +* (Obsolete) Return info about compiled pattern * +*************************************************/ + +/* This is the original "info" function. It picks potentially useful data out +of the private structure, but its interface was too rigid. It remains for +backwards compatibility. The public options are passed back in an int - though +the re->options field has been expanded to a long int, all the public options +at the low end of it, and so even on 16-bit systems this will still be OK. +Therefore, I haven't changed the API for pcre_info(). + +Arguments: + argument_re points to compiled code + optptr where to pass back the options + first_byte where to pass back the first character, + or -1 if multiline and all branches start ^, + or -2 otherwise + +Returns: number of capturing subpatterns + or negative values on error +*/ + +EXPORT int +pcre_info(const pcre *argument_re, int *optptr, int *first_byte) +{ +real_pcre internal_re; +const real_pcre *re = (const real_pcre *)argument_re; +if (re == NULL) return PCRE_ERROR_NULL; +if (re->magic_number != MAGIC_NUMBER) + { + re = _pcre_try_flipped(re, &internal_re, NULL, NULL); + if (re == NULL) return PCRE_ERROR_BADMAGIC; + } +if (optptr != NULL) *optptr = (int)(re->options & PUBLIC_OPTIONS); +if (first_byte != NULL) + *first_byte = ((re->options & PCRE_FIRSTSET) != 0)? re->first_byte : + ((re->options & PCRE_STARTLINE) != 0)? -1 : -2; +return re->top_bracket; +} + +/* End of pcre_info.c */ diff --git a/internal.h b/pcre_internal.h index 5d14331..67c0186 100644 --- a/internal.h +++ b/pcre_internal.h @@ -3,13 +3,11 @@ *************************************************/ -/* This is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. See -the file doc/Tech.Notes for some information on the internals. +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2004 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -41,7 +39,27 @@ POSSIBILITY OF SUCH DAMAGE. */ /* This header contains definitions that are shared between the different -modules, but which are not relevant to the outside. */ +modules, but which are not relevant to the exported API. This includes some +functions whose names all begin with "_pcre_". */ + + +/* Define DEBUG to get debugging output on stdout. */ + +/**** +#define DEBUG +****/ + +/* Use a macro for debugging printing, 'cause that eliminates the use of #ifdef +inline, and there are *still* stupid compilers about that don't like indented +pre-processor statements, or at least there were when I first wrote this. After +all, it had only been about 10 years then... */ + +#ifdef DEBUG +#define DPRINTF(p) printf p +#else +#define DPRINTF(p) /*nothing*/ +#endif + /* Get the definitions provided by running "configure" */ @@ -98,6 +116,12 @@ typedef unsigned char uschar; #include "pcre.h" +/* Include the (copy of) the public ucp header, changing the external name into +a private one. This does no harm, even if we aren't compiling UCP support. */ + +#define ucp_findchar _pcre_ucp_findchar +#include "ucp.h" + /* When compiling for use with the Virtual Pascal compiler, these functions need to have their names changed. PCRE must be compiled with the -DVPCOMPAT option on the command line. */ @@ -211,6 +235,116 @@ capturing parenthesis numbers in back references. */ #define PUT2INC(a,n,d) PUT2(a,n,d), a += 2 +/* When UTF-8 encoding is being used, a character is no longer just a single +byte. The macros for character handling generate simple sequences when used in +byte-mode, and more complicated ones for UTF-8 characters. */ + +#ifndef SUPPORT_UTF8 +#define GETCHAR(c, eptr) c = *eptr; +#define GETCHARTEST(c, eptr) c = *eptr; +#define GETCHARINC(c, eptr) c = *eptr++; +#define GETCHARINCTEST(c, eptr) c = *eptr++; +#define GETCHARLEN(c, eptr, len) c = *eptr; +#define BACKCHAR(eptr) + +#else /* SUPPORT_UTF8 */ + +/* Get the next UTF-8 character, not advancing the pointer. This is called when +we know we are in UTF-8 mode. */ + +#define GETCHAR(c, eptr) \ + c = *eptr; \ + if ((c & 0xc0) == 0xc0) \ + { \ + int gcii; \ + int gcaa = _pcre_utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ + int gcss = 6*gcaa; \ + c = (c & _pcre_utf8_table3[gcaa]) << gcss; \ + for (gcii = 1; gcii <= gcaa; gcii++) \ + { \ + gcss -= 6; \ + c |= (eptr[gcii] & 0x3f) << gcss; \ + } \ + } + +/* Get the next UTF-8 character, testing for UTF-8 mode, and not advancing the +pointer. */ + +#define GETCHARTEST(c, eptr) \ + c = *eptr; \ + if (utf8 && (c & 0xc0) == 0xc0) \ + { \ + int gcii; \ + int gcaa = _pcre_utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ + int gcss = 6*gcaa; \ + c = (c & _pcre_utf8_table3[gcaa]) << gcss; \ + for (gcii = 1; gcii <= gcaa; gcii++) \ + { \ + gcss -= 6; \ + c |= (eptr[gcii] & 0x3f) << gcss; \ + } \ + } + +/* Get the next UTF-8 character, advancing the pointer. This is called when we +know we are in UTF-8 mode. */ + +#define GETCHARINC(c, eptr) \ + c = *eptr++; \ + if ((c & 0xc0) == 0xc0) \ + { \ + int gcaa = _pcre_utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ + int gcss = 6*gcaa; \ + c = (c & _pcre_utf8_table3[gcaa]) << gcss; \ + while (gcaa-- > 0) \ + { \ + gcss -= 6; \ + c |= (*eptr++ & 0x3f) << gcss; \ + } \ + } + +/* Get the next character, testing for UTF-8 mode, and advancing the pointer */ + +#define GETCHARINCTEST(c, eptr) \ + c = *eptr++; \ + if (utf8 && (c & 0xc0) == 0xc0) \ + { \ + int gcaa = _pcre_utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ + int gcss = 6*gcaa; \ + c = (c & _pcre_utf8_table3[gcaa]) << gcss; \ + while (gcaa-- > 0) \ + { \ + gcss -= 6; \ + c |= (*eptr++ & 0x3f) << gcss; \ + } \ + } + +/* Get the next UTF-8 character, not advancing the pointer, incrementing length +if there are extra bytes. This is called when we know we are in UTF-8 mode. */ + +#define GETCHARLEN(c, eptr, len) \ + c = *eptr; \ + if ((c & 0xc0) == 0xc0) \ + { \ + int gcii; \ + int gcaa = _pcre_utf8_table4[c & 0x3f]; /* Number of additional bytes */ \ + int gcss = 6*gcaa; \ + c = (c & _pcre_utf8_table3[gcaa]) << gcss; \ + for (gcii = 1; gcii <= gcaa; gcii++) \ + { \ + gcss -= 6; \ + c |= (eptr[gcii] & 0x3f) << gcss; \ + } \ + len += gcaa; \ + } + +/* If the pointer is not at the start of a character, move it back until +it is. Called only in UTF-8 mode. */ + +#define BACKCHAR(eptr) while((*eptr & 0xc0) == 0x80) eptr--; + +#endif + + /* In case there is no definition of offsetof() provided - though any proper Standard C system should have one. */ @@ -226,8 +360,7 @@ Standard C system should have one. */ /* Private options flags start at the most significant end of the four bytes, but skip the top bit so we can use ints for convenience without getting tangled with negative values. The public options defined in pcre.h start at the least -significant end. Make sure they don't overlap, though now that we have expanded -to four bytes, there is plenty of space. */ +significant end. Make sure they don't overlap! */ #define PCRE_FIRSTSET 0x40000000 /* first_byte is set */ #define PCRE_REQCHSET 0x20000000 /* req_byte is set */ @@ -239,21 +372,26 @@ to four bytes, there is plenty of space. */ #define PCRE_STUDY_MAPPED 0x01 /* a map of starting chars exists */ -/* Masks for identifying the public options which are permitted at compile -time, run time or study time, respectively. */ +/* Masks for identifying the public options that are permitted at compile +time, run time, or study time, respectively. */ #define PUBLIC_OPTIONS \ (PCRE_CASELESS|PCRE_EXTENDED|PCRE_ANCHORED|PCRE_MULTILINE| \ PCRE_DOTALL|PCRE_DOLLAR_ENDONLY|PCRE_EXTRA|PCRE_UNGREEDY|PCRE_UTF8| \ - PCRE_NO_AUTO_CAPTURE|PCRE_NO_UTF8_CHECK|PCRE_AUTO_CALLOUT) + PCRE_NO_AUTO_CAPTURE|PCRE_NO_UTF8_CHECK|PCRE_AUTO_CALLOUT|PCRE_FIRSTLINE) #define PUBLIC_EXEC_OPTIONS \ (PCRE_ANCHORED|PCRE_NOTBOL|PCRE_NOTEOL|PCRE_NOTEMPTY|PCRE_NO_UTF8_CHECK| \ PCRE_PARTIAL) +#define PUBLIC_DFA_EXEC_OPTIONS \ + (PCRE_ANCHORED|PCRE_NOTBOL|PCRE_NOTEOL|PCRE_NOTEMPTY|PCRE_NO_UTF8_CHECK| \ + PCRE_PARTIAL|PCRE_DFA_SHORTEST|PCRE_DFA_RESTART) + #define PUBLIC_STUDY_OPTIONS 0 /* None defined */ -/* Magic number to provide a small check against being handed junk. */ +/* Magic number to provide a small check against being handed junk. Also used +to detect whether a pattern was compiled on a host of different endianness. */ #define MAGIC_NUMBER 0x50435245UL /* 'PCRE' */ @@ -262,6 +400,11 @@ time, run time or study time, respectively. */ #define REQ_UNSET (-2) #define REQ_NONE (-1) +/* The maximum remaining length of subject we are prepared to search for a +req_byte match. */ + +#define REQ_BYTE_MAX 1000 + /* Flags added to firstbyte or reqbyte; a "non-literal" item is either a variable-length repeat, or a anything other than literal characters. */ @@ -461,9 +604,8 @@ opcodes. */ #define EXTRACT_BASIC_MAX 100 -/* This macro defines textual names for all the opcodes. There are used only -for debugging, in pcre.c when DEBUG is defined, and also in pcretest.c. The -macro is referenced only in printint.c. */ +/* This macro defines textual names for all the opcodes. These are used only +for debugging. The macro is referenced only in pcre_printint.c. */ #define OP_NAME_LIST \ "End", "\\A", "\\G", "\\B", "\\b", "\\D", "\\d", \ @@ -484,7 +626,7 @@ macro is referenced only in printint.c. */ /* This macro defines the length of fixed length operations in the compiled regex. The lengths are used when searching for specific things, and also in the debugging printing of a compiled regex. We use a macro so that it can be -incorporated both into pcre.c and pcretest.c without being publicly exposed. +defined close to the definitions of the opcodes themselves. As things have been extended, some of these are no longer fixed lenths, but are minima instead. For example, the length of a single-character repeat may vary @@ -538,66 +680,23 @@ in UTF-8 mode. The code that uses this table must know about such things. */ #define CREF_RECURSE 0xffff -/* The texts of compile-time error messages are defined as macros here so that -they can be accessed by the POSIX wrapper and converted into error codes. Yes, -I could have used error codes in the first place, but didn't feel like changing -just to accommodate the POSIX wrapper. */ - -#define ERR1 "\\ at end of pattern" -#define ERR2 "\\c at end of pattern" -#define ERR3 "unrecognized character follows \\" -#define ERR4 "numbers out of order in {} quantifier" -#define ERR5 "number too big in {} quantifier" -#define ERR6 "missing terminating ] for character class" -#define ERR7 "invalid escape sequence in character class" -#define ERR8 "range out of order in character class" -#define ERR9 "nothing to repeat" -#define ERR10 "operand of unlimited repeat could match the empty string" -#define ERR11 "internal error: unexpected repeat" -#define ERR12 "unrecognized character after (?" -#define ERR13 "POSIX named classes are supported only within a class" -#define ERR14 "missing )" -#define ERR15 "reference to non-existent subpattern" -#define ERR16 "erroffset passed as NULL" -#define ERR17 "unknown option bit(s) set" -#define ERR18 "missing ) after comment" -#define ERR19 "parentheses nested too deeply" -#define ERR20 "regular expression too large" -#define ERR21 "failed to get memory" -#define ERR22 "unmatched parentheses" -#define ERR23 "internal error: code overflow" -#define ERR24 "unrecognized character after (?<" -#define ERR25 "lookbehind assertion is not fixed length" -#define ERR26 "malformed number after (?(" -#define ERR27 "conditional group contains more than two branches" -#define ERR28 "assertion expected after (?(" -#define ERR29 "(?R or (?digits must be followed by )" -#define ERR30 "unknown POSIX class name" -#define ERR31 "POSIX collating elements are not supported" -#define ERR32 "this version of PCRE is not compiled with PCRE_UTF8 support" -#define ERR33 "spare error" -#define ERR34 "character value in \\x{...} sequence is too large" -#define ERR35 "invalid condition (?(0)" -#define ERR36 "\\C not allowed in lookbehind assertion" -#define ERR37 "PCRE does not support \\L, \\l, \\N, \\U, or \\u" -#define ERR38 "number after (?C is > 255" -#define ERR39 "closing ) for (?C expected" -#define ERR40 "recursive call could loop indefinitely" -#define ERR41 "unrecognized character after (?P" -#define ERR42 "syntax error after (?P" -#define ERR43 "two named groups have the same name" -#define ERR44 "invalid UTF-8 string" -#define ERR45 "support for \\P, \\p, and \\X has not been compiled" -#define ERR46 "malformed \\P or \\p sequence" -#define ERR47 "unknown property name after \\P or \\p" +/* Error code numbers. They are given names so that they can more easily be +tracked. */ + +enum { ERR0, ERR1, ERR2, ERR3, ERR4, ERR5, ERR6, ERR7, ERR8, ERR9, + ERR10, ERR11, ERR12, ERR13, ERR14, ERR15, ERR16, ERR17, ERR18, ERR19, + ERR20, ERR21, ERR22, ERR23, ERR24, ERR25, ERR26, ERR27, ERR28, ERR29, + ERR30, ERR31, ERR32, ERR33, ERR34, ERR35, ERR36, ERR37, ERR38, ERR39, + ERR40, ERR41, ERR42, ERR43, ERR44, ERR45, ERR46, ERR47 }; /* The real format of the start of the pcre block; the index of names and the code vector run on as long as necessary after the end. We store an explicit offset to the name table so that if a regex is compiled on one host, saved, and then run on another where the size of pointers is different, all might still be well. For the case of compiled-on-4 and run-on-8, we include an extra -pointer that is always NULL. For future-proofing, we also include a few dummy -fields - even though you can never get this planning right! +pointer that is always NULL. For future-proofing, a few dummy fields were +originally included - even though you can never get this planning right - but +there is only one left now. NOTE NOTE NOTE: Because people can now save and re-use compiled patterns, any additions to this @@ -620,7 +719,7 @@ typedef struct real_pcre { pcre_uint16 name_table_offset; /* Offset to name table that follows */ pcre_uint16 name_entry_size; /* Size of any name items */ pcre_uint16 name_count; /* Number of name items */ - pcre_uint16 dummy2; /* For future use, maybe */ + pcre_uint16 ref_count; /* Reference count */ const unsigned char *tables; /* Pointer to tables or NULL for std */ const unsigned char *nullpad; /* NULL padding */ @@ -685,7 +784,7 @@ NOTE: This isn't used for a "normal" compilation of pcre. */ struct heapframe; /* Structure for passing "static" information around between the functions -doing the matching, so that they are thread-safe. */ +doing traditional NFA matching, so that they are thread-safe. */ typedef struct match_data { unsigned long int match_call_count; /* As it says */ @@ -716,6 +815,19 @@ typedef struct match_data { struct heapframe *thisframe; /* Used only when compiling for no recursion */ } match_data; +/* A similar structure is used for the same purpose by the DFA matching +functions. */ + +typedef struct dfa_match_data { + const uschar *start_code; /* Start of the compiled pattern */ + const uschar *start_subject; /* Start of the subject string */ + const uschar *end_subject; /* End of subject string */ + const uschar *tables; /* Character tables */ + int moptions; /* Match options */ + int poptions; /* Pattern options */ + void *callout_data; /* To pass back to callouts */ +} dfa_match_data; + /* Bit definitions for entries in the pcre_ctypes table. */ #define ctype_space 0x01 @@ -749,4 +861,45 @@ total length. */ #define ctypes_offset (cbits_offset + cbit_length) #define tables_length (ctypes_offset + 256) -/* End of internal.h */ +/* Layout of the UCP type table that translates property names into codes for +ucp_findchar(). */ + +typedef struct { + const char *name; + int value; +} ucp_type_table; + + +/* Internal shared data tables. These are tables that are used by more than one +of the exported public functions. They have to be "external" in the C sense, +but are not part of the PCRE public API. The data for these tables is in the +pcre_tables.c module. */ + +extern const int _pcre_utf8_table1[]; +extern const int _pcre_utf8_table2[]; +extern const int _pcre_utf8_table3[]; +extern const uschar _pcre_utf8_table4[]; + +extern const int _pcre_utf8_table1_size; + +extern const ucp_type_table _pcre_utt[]; +extern const int _pcre_utt_size; + +extern const uschar _pcre_default_tables[]; + +extern const uschar _pcre_OP_lengths[]; + + +/* Internal shared functions. These are functions that are used by more than +one of the exported public functions. They have to be "external" in the C +sense, but are not part of the PCRE public API. */ + +extern int _pcre_ord2utf8(int, uschar *); +extern void _pcre_printint(pcre *, FILE *); +extern real_pcre * _pcre_try_flipped(const real_pcre *, real_pcre *, + const pcre_study_data *, pcre_study_data *); +extern int _pcre_ucp_findchar(const int, int *, int *); +extern int _pcre_valid_utf8(const uschar *, int); +extern BOOL _pcre_xclass(int, const uschar *); + +/* End of pcre_internal.h */ diff --git a/maketables.c b/pcre_maketables.c index f1c7b9a..c4954b2 100644 --- a/maketables.c +++ b/pcre_maketables.c @@ -2,13 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -PCRE is a library of functions to support regular expressions whose syntax +/* PCRE is a library of functions to support regular expressions whose syntax and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2003 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -40,16 +38,17 @@ POSSIBILITY OF SUCH DAMAGE. */ -/* This file is compiled on its own as part of the PCRE library. However, -it is also included in the compilation of dftables.c, in which case the macro -DFTABLES is defined. */ +/* This module contains the external function pcre_maketables(), which builds +character tables for PCRE in the current locale. The file is compiled on its +own as part of the PCRE library. However, it is also included in the +compilation of dftables.c, in which case the macro DFTABLES is defined. */ + #ifndef DFTABLES -#include "internal.h" +#include "pcre_internal.h" #endif - /************************************************* * Create PCRE character tables * *************************************************/ @@ -143,4 +142,4 @@ for (i = 0; i < 256; i++) return yield; } -/* End of maketables.c */ +/* End of pcre_maketables.c */ diff --git a/pcre_ord2utf8.c b/pcre_ord2utf8.c new file mode 100644 index 0000000..f6a06fc --- /dev/null +++ b/pcre_ord2utf8.c @@ -0,0 +1,78 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This file contains a private PCRE function that converts an ordinal +character value into a UTF8 string. */ + + +#include "pcre_internal.h" + + +/************************************************* +* Convert character value to UTF-8 * +*************************************************/ + +/* This function takes an integer value in the range 0 - 0x7fffffff +and encodes it as a UTF-8 character in 0 to 6 bytes. + +Arguments: + cvalue the character value + buffer pointer to buffer for result - at least 6 bytes long + +Returns: number of characters placed in the buffer +*/ + +EXPORT int +_pcre_ord2utf8(int cvalue, uschar *buffer) +{ +register int i, j; +for (i = 0; i < _pcre_utf8_table1_size; i++) + if (cvalue <= _pcre_utf8_table1[i]) break; +buffer += i; +for (j = i; j > 0; j--) + { + *buffer-- = 0x80 | (cvalue & 0x3f); + cvalue >>= 6; + } +*buffer = _pcre_utf8_table2[i] | cvalue; +return i + 1; +} + +/* End of pcre_ord2utf8.c */ diff --git a/printint.c b/pcre_printint.c index 8e5da42..d18f399 100644 --- a/printint.c +++ b/pcre_printint.c @@ -2,14 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -This is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. See -the file Tech.Notes for some information on the internals. +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2004 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -41,10 +38,12 @@ POSSIBILITY OF SUCH DAMAGE. */ -/* This module contains a debugging function for printing out the internal form -of a compiled regular expression. It is kept in a separate file so that it can -be #included both in the pcretest program, and in the library itself when -compiled with the debugging switch. */ +/* This module contains an PCRE private debugging function for printing out the +internal form of a compiled regular expression, along with some supporting +local functions. */ + + +#include "pcre_internal.h" static const char *OP_names[] = { OP_NAME_LIST }; @@ -54,18 +53,6 @@ static const char *OP_names[] = { OP_NAME_LIST }; * Print single- or multi-byte character * *************************************************/ -/* These tables are actually copies of ones in pcre.c. If we compile the -library with debugging, they are included twice, but that isn't really a -problem - compiling with debugging is pretty rare and these are very small. */ - -static const int utf8_t3[] = { 0xff, 0x1f, 0x0f, 0x07, 0x03, 0x01}; - -static const uschar utf8_t4[] = { - 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, - 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, - 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2, - 3,3,3,3,3,3,3,3,4,4,4,4,5,5,5,5 }; - static int print_char(FILE *f, uschar *ptr, BOOL utf8) { @@ -79,9 +66,9 @@ if (!utf8 || (c & 0xc0) != 0xc0) else { int i; - int a = utf8_t4[c & 0x3f]; /* Number of additional bytes */ + int a = _pcre_utf8_table4[c & 0x3f]; /* Number of additional bytes */ int s = 6*a; - c = (c & utf8_t3[a]) << s; + c = (c & _pcre_utf8_table3[a]) << s; for (i = 1; i <= a; i++) { /* This is a check for malformed UTF-8; it should only occur if the sanity @@ -106,7 +93,6 @@ else - /************************************************* * Find Unicode property name * *************************************************/ @@ -114,12 +100,16 @@ else static const char * get_ucpname(int property) { +#ifdef SUPPORT_UCP int i; -for (i = sizeof(utt)/sizeof(ucp_type_table); i >= 0; i--) +for (i = _pcre_utt_size; i >= 0; i--) { - if (property == utt[i].value) break; + if (property == _pcre_utt[i].value) break; } -return (i >= 0)? utt[i].name : "??"; +return (i >= 0)? _pcre_utt[i].name : "??"; +#else +return "??"; +#endif } @@ -131,8 +121,8 @@ return (i >= 0)? utt[i].name : "??"; /* Make this function work for a regex with integers either byte order. However, we assume that what we are passed is a compiled regex. */ -static void -print_internals(pcre *external_re, FILE *f) +EXPORT void +_pcre_printint(pcre *external_re, FILE *f) { real_pcre *re = (real_pcre *)external_re; uschar *codestart, *code; @@ -163,7 +153,7 @@ for(;;) int c; int extra = 0; - fprintf(f, "%3d ", code - codestart); + fprintf(f, "%3d ", (int)(code - codestart)); if (*code >= OP_BRA) { @@ -171,7 +161,7 @@ for(;;) fprintf(f, "%3d Bra extra\n", GET(code, 1)); else fprintf(f, "%3d Bra %d\n", GET(code, 1), *code - OP_BRA); - code += OP_lengths[OP_BRA]; + code += _pcre_OP_lengths[OP_BRA]; continue; } @@ -312,7 +302,7 @@ for(;;) case OP_NOTMINUPTO: if (isprint(c = code[3])) fprintf(f, " [^%c]{", c); else fprintf(f, " [^\\x%02x]{", c); - if (*code != OP_NOTEXACT) fprintf(f, ","); + if (*code != OP_NOTEXACT) fprintf(f, "0,"); fprintf(f, "%d}", GET2(code,1)); if (*code == OP_NOTMINUPTO) fprintf(f, "?"); break; @@ -323,7 +313,7 @@ for(;;) case OP_REF: fprintf(f, " \\%d", GET2(code,1)); - ccode = code + OP_lengths[*code]; + ccode = code + _pcre_OP_lengths[*code]; goto CLASS_REF_REPEAT; case OP_CALLOUT: @@ -430,7 +420,7 @@ for(;;) case OP_CRQUERY: case OP_CRMINQUERY: fprintf(f, "%s", OP_names[*ccode]); - extra += OP_lengths[*ccode]; + extra += _pcre_OP_lengths[*ccode]; break; case OP_CRRANGE: @@ -440,7 +430,7 @@ for(;;) if (max == 0) fprintf(f, "{%d,}", min); else fprintf(f, "{%d,%d}", min, max); if (*ccode == OP_CRMINRANGE) fprintf(f, "?"); - extra += OP_lengths[*ccode]; + extra += _pcre_OP_lengths[*ccode]; break; } } @@ -453,9 +443,9 @@ for(;;) break; } - code += OP_lengths[*code] + extra; + code += _pcre_OP_lengths[*code] + extra; fprintf(f, "\n"); } } -/* End of printint.c */ +/* End of pcre_printint.c */ diff --git a/pcre_refcount.c b/pcre_refcount.c new file mode 100644 index 0000000..35a7ee8 --- /dev/null +++ b/pcre_refcount.c @@ -0,0 +1,77 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains the external function pcre_refcount(), which is an +auxiliary function that can be used to maintain a reference count in a compiled +pattern data block. This might be helpful in applications where the block is +shared by different users. */ + +#include "pcre_internal.h" + + +/************************************************* +* Maintain reference count * +*************************************************/ + +/* The reference count is a 16-bit field, initialized to zero. It is not +possible to transfer a non-zero count from one host to a different host that +has a different byte order - though I can't see why anyone in their right mind +would ever want to do that! + +Arguments: + argument_re points to compiled code + adjust value to add to the count + +Returns: the (possibly updated) count value (a non-negative number), or + a negative error number +*/ + +EXPORT int +pcre_refcount(pcre *argument_re, int adjust) +{ +real_pcre *re = (real_pcre *)argument_re; +if (re == NULL) return PCRE_ERROR_NULL; +re->ref_count = (-adjust > re->ref_count)? 0 : + (adjust + re->ref_count > 65535)? 65535 : + re->ref_count + adjust; +return re->ref_count; +} + +/* End of pcre_refcount.c */ diff --git a/pcre_scanner.cc b/pcre_scanner.cc new file mode 100644 index 0000000..4a66fdf --- /dev/null +++ b/pcre_scanner.cc @@ -0,0 +1,169 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: Sanjay Ghemawat + +#include <vector> +#include <algorithm> // for count() +#include <assert.h> +#include "config.h" +#include "pcre_scanner.h" + +using std::vector; + +namespace pcrecpp { + +Scanner::Scanner() + : data_(), + input_(data_), + skip_(NULL), + should_skip_(false), + save_comments_(false), + comments_(NULL), + comments_offset_(0) { +} + +Scanner::Scanner(const string& in) + : data_(in), + input_(data_), + skip_(NULL), + should_skip_(false), + save_comments_(false), + comments_(NULL), + comments_offset_(0) { +} + +Scanner::~Scanner() { + delete skip_; + delete comments_; +} + +void Scanner::Skip(const char* re) { + delete skip_; + if (re != NULL) { + skip_ = new RE(re); + should_skip_ = true; + ConsumeSkip(); + } else { + skip_ = NULL; + should_skip_ = false; + } +} + +void Scanner::DisableSkip() { + assert(skip_ != NULL); + should_skip_ = false; +} + +void Scanner::EnableSkip() { + assert(skip_ != NULL); + should_skip_ = true; + ConsumeSkip(); +} + +int Scanner::LineNumber() const { + // TODO: Make it more efficient by keeping track of the last point + // where we computed line numbers and counting newlines since then. + return 1 + std::count(data_.data(), input_.data(), '\n'); +} + +int Scanner::Offset() const { + return input_.data() - data_.c_str(); +} + +bool Scanner::LookingAt(const RE& re) const { + int consumed; + return re.DoMatch(input_, RE::ANCHOR_START, &consumed, 0, 0); +} + + +bool Scanner::Consume(const RE& re, + const Arg& arg0, + const Arg& arg1, + const Arg& arg2) { + const bool result = re.Consume(&input_, arg0, arg1, arg2); + if (result && should_skip_) ConsumeSkip(); + return result; +} + +// helper function to consume *skip_ and honour save_comments_ +void Scanner::ConsumeSkip() { + if (save_comments_) { + if (NULL == comments_) { + comments_ = new vector<StringPiece>; + } + const char *start_data = input_.data(); + skip_->Consume(&input_); + // already pointing one past end, so no need to +1 + int length = input_.data() - start_data; + if (length > 0) { + comments_->push_back(StringPiece(start_data, length)); + } + } else { + skip_->Consume(&input_); + } +} + + +void Scanner::GetComments(int start, int end, vector<StringPiece> *ranges) { + // short circuit out if we've not yet initialized comments_ + // (e.g., when save_comments is false) + if (!comments_) { + return; + } + // TODO: if we guarantee that comments_ will contain StringPieces + // that are ordered by their start, then we can do a binary search + // for the first StringPiece at or past start and then scan for the + // ones contained in the range, quit early (use equal_range or + // lower_bound) + for (vector<StringPiece>::const_iterator it = comments_->begin(); + it != comments_->end(); ++it) { + if ((it->data() >= data_.c_str() + start && + it->data() + it->size() <= data_.c_str() + end)) { + ranges->push_back(*it); + } + } +} + + +void Scanner::GetNextComments(vector<StringPiece> *ranges) { + // short circuit out if we've not yet initialized comments_ + // (e.g., when save_comments is false) + if (!comments_) { + return; + } + for (vector<StringPiece>::const_iterator it = + comments_->begin() + comments_offset_; + it != comments_->end(); ++it) { + ranges->push_back(*it); + ++comments_offset_; + } +} + +} // namespace pcrecpp diff --git a/pcre_scanner.h b/pcre_scanner.h new file mode 100644 index 0000000..a73b72f --- /dev/null +++ b/pcre_scanner.h @@ -0,0 +1,163 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: Sanjay Ghemawat +// +// Regular-expression based scanner for parsing an input stream. +// +// Example 1: parse a sequence of "var = number" entries from input: +// +// Scanner scanner(input); +// string var; +// int number; +// scanner.Skip("\\s+"); // Skip any white space we encounter +// while (scanner.Consume("(\\w+) = (\\d+)", &var, &number)) { +// ...; +// } + +#ifndef _PCRE_SCANNER_H +#define _PCRE_SCANNER_H + +#include <assert.h> +#include <string> +#include <vector> +#include <pcrecpp.h> +#include <pcre_stringpiece.h> + +namespace pcrecpp { + +class Scanner { + public: + Scanner(); + explicit Scanner(const std::string& input); + ~Scanner(); + + // Return current line number. The returned line-number is + // one-based. I.e. it returns 1 + the number of consumed newlines. + // + // Note: this method may be slow. It may take time proportional to + // the size of the input. + int LineNumber() const; + + // Return the byte-offset that the scanner is looking in the + // input data; + int Offset() const; + + // Return true iff the start of the remaining input matches "re" + bool LookingAt(const RE& re) const; + + // Return true iff all of the following are true + // a. the start of the remaining input matches "re", + // b. if any arguments are supplied, matched sub-patterns can be + // parsed and stored into the arguments. + // If it returns true, it skips over the matched input and any + // following input that matches the "skip" regular expression. + bool Consume(const RE& re, + const Arg& arg0 = no_arg, + const Arg& arg1 = no_arg, + const Arg& arg2 = no_arg + // TODO: Allow more arguments? + ); + + // Set the "skip" regular expression. If after consuming some data, + // a prefix of the input matches this RE, it is automatically + // skipped. For example, a programming language scanner would use + // a skip RE that matches white space and comments. + // + // scanner.Skip("(\\s|//.*|/[*](.|\n)*?[*]/)*"); + // + // You can pass NULL for "re" if you do not want any data to be skipped. + void Skip(const char* re); + + // Temporarily pause "skip"ing. This + // Skip("Foo"); code ; DisableSkip(); code; EnableSkip() + // is similar to + // Skip("Foo"); code ; Skip(NULL); code ; Skip("Foo"); + // but avoids creating/deleting new RE objects. + void DisableSkip(); + + // Reenable previously paused skipping. Any prefix of the input + // that matches the skip pattern is immediately dropped. + void EnableSkip(); + + /***** Special wrappers around SetSkip() for some common idioms *****/ + + // Arranges to skip whitespace, C comments, C++ comments. + // The overall RE is a repeated disjunction of the following REs: + // \\s whitespace + // //.*\n C++ comment + // /[*](.|\n)*?[*]/ C comment (x*? means minimal repetitions of x) + void SkipCXXComments() { + Skip("((\\s|//.*\n|/[*](.|\n)*?[*]/)*)"); + } + + void set_save_comments(bool comments) { + save_comments_ = comments; + } + + bool save_comments() { + return save_comments_; + } + + // Append to vector ranges the comments found in the + // byte range [start,end] (inclusive) of the input data. + // Only comments that were extracted entirely within that + // range are returned: no range splitting of atomically-extracted + // comments is performed. + void GetComments(int start, int end, std::vector<StringPiece> *ranges); + + // Append to vector ranges the comments added + // since the last time this was called. This + // functionality is provided for efficiency when + // interleaving scanning with parsing. + void GetNextComments(std::vector<StringPiece> *ranges); + + private: + std::string data_; // All the input data + StringPiece input_; // Unprocessed input + RE* skip_; // If non-NULL, RE for skipping input + bool should_skip_; // If true, use skip_ + bool save_comments_; // If true, aggregate the skip expression + + // the skipped comments + // TODO: later consider requiring that the StringPieces be added + // in order by their start position + std::vector<StringPiece> *comments_; + + // the offset into comments_ that has been returned by GetNextComments + int comments_offset_; + + // helper function to consume *skip_ and honour + // save_comments_ + void ConsumeSkip(); +}; + +} // namespace pcrecpp + +#endif /* _PCRE_SCANNER_H */ diff --git a/pcre_scanner_unittest.cc b/pcre_scanner_unittest.cc new file mode 100644 index 0000000..e000a07 --- /dev/null +++ b/pcre_scanner_unittest.cc @@ -0,0 +1,125 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: Greg J. Badros +// +// Unittest for scanner, especially GetNextComments and GetComments() +// functionality. + +#include <stdio.h> +#include <vector> +#include <pcre_stringpiece.h> +#include <pcre_scanner.h> + +// Dies with a fatal error if the two values are not equal. +#define CHECK_EQ(a, b) do { \ + if ( (a) != (b) ) { \ + fprintf(stderr, "%s:%d: Check failed because %s != %s\n", \ + __FILE__, __LINE__, #a, #b); \ + exit(1); \ + } \ +} while (0) + +using std::vector; +using pcrecpp::StringPiece; +using pcrecpp::Scanner; + +static void TestScanner() { + const char input[] = "\n" + "alpha = 1; // this sets alpha\n" + "bravo = 2; // bravo is set here\n" + "gamma = 33; /* and here is gamma */\n"; + + const char *re = "(\\w+) = (\\d+);"; + + Scanner s(input); + string var; + int number; + s.SkipCXXComments(); + s.set_save_comments(true); + vector<StringPiece> comments; + + s.Consume(re, &var, &number); + CHECK_EQ(var, "alpha"); + CHECK_EQ(number, 1); + s.GetNextComments(&comments); + CHECK_EQ(comments.size(), 1); + CHECK_EQ(comments[0].as_string(), " // this sets alpha\n"); + comments.resize(0); + + s.Consume(re, &var, &number); + CHECK_EQ(var, "bravo"); + CHECK_EQ(number, 2); + s.GetNextComments(&comments); + CHECK_EQ(comments.size(), 1); + CHECK_EQ(comments[0].as_string(), " // bravo is set here\n"); + comments.resize(0); + + s.Consume(re, &var, &number); + CHECK_EQ(var, "gamma"); + CHECK_EQ(number, 33); + s.GetNextComments(&comments); + CHECK_EQ(comments.size(), 1); + CHECK_EQ(comments[0].as_string(), " /* and here is gamma */\n"); + comments.resize(0); + + s.GetComments(0, sizeof(input), &comments); + CHECK_EQ(comments.size(), 3); + CHECK_EQ(comments[0].as_string(), " // this sets alpha\n"); + CHECK_EQ(comments[1].as_string(), " // bravo is set here\n"); + CHECK_EQ(comments[2].as_string(), " /* and here is gamma */\n"); + comments.resize(0); + + s.GetComments(0, strchr(input, '/') - input, &comments); + CHECK_EQ(comments.size(), 0); + comments.resize(0); + + s.GetComments(strchr(input, '/') - input - 1, sizeof(input), + &comments); + CHECK_EQ(comments.size(), 3); + CHECK_EQ(comments[0].as_string(), " // this sets alpha\n"); + CHECK_EQ(comments[1].as_string(), " // bravo is set here\n"); + CHECK_EQ(comments[2].as_string(), " /* and here is gamma */\n"); + comments.resize(0); + + s.GetComments(strchr(input, '/') - input - 1, + strchr(input + 1, '\n') - input + 1, &comments); + CHECK_EQ(comments.size(), 1); + CHECK_EQ(comments[0].as_string(), " // this sets alpha\n"); + comments.resize(0); +} + +int main(int argc, char** argv) { + TestScanner(); + + // Done + printf("OK\n"); + + return 0; +} diff --git a/pcre_stringpiece.cc b/pcre_stringpiece.cc new file mode 100644 index 0000000..dbdb509 --- /dev/null +++ b/pcre_stringpiece.cc @@ -0,0 +1,39 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wilsonh@google.com (Wilson Hsieh) +// + +#include <iostream> +#include "config.h" +#include "pcre_stringpiece.h" + +std::ostream& operator<<(std::ostream& o, const pcrecpp::StringPiece& piece) { + return (o << piece.as_string()); +} diff --git a/pcre_stringpiece.h.in b/pcre_stringpiece.h.in new file mode 100644 index 0000000..26eae42 --- /dev/null +++ b/pcre_stringpiece.h.in @@ -0,0 +1,172 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: Sanjay Ghemawat +// +// A string like object that points into another piece of memory. +// Useful for providing an interface that allows clients to easily +// pass in either a "const char*" or a "string". +// +// Arghh! I wish C++ literals were automatically of type "string". + +#ifndef _PCRE_STRINGPIECE_H +#define _PCRE_STRINGPIECE_H + +#include <string.h> +#include <string> +#include <iosfwd> // for ostream forward-declaration + +#if @pcre_has_type_traits@ +#define HAVE_TYPE_TRAITS +#include <type_traits.h> +#elif @pcre_has_bits_type_traits@ +#define HAVE_TYPE_TRAITS +#include <bits/type_traits.h> +#endif + +using std::string; + +namespace pcrecpp { + +class StringPiece { + private: + const char* ptr_; + int length_; + + public: + // We provide non-explicit singleton constructors so users can pass + // in a "const char*" or a "string" wherever a "StringPiece" is + // expected. + StringPiece() + : ptr_(NULL), length_(0) { } + StringPiece(const char* str) + : ptr_(str), length_(static_cast<int>(strlen(str))) { } + StringPiece(const string& str) + : ptr_(str.data()), length_(static_cast<int>(str.size())) { } + StringPiece(const char* offset, int len) + : ptr_(offset), length_(len) { } + + // data() may return a pointer to a buffer with embedded NULs, and the + // returned buffer may or may not be null terminated. Therefore it is + // typically a mistake to pass data() to a routine that expects a NUL + // terminated string. Use "as_string().c_str()" if you really need to do + // this. Or better yet, change your routine so it does not rely on NUL + // termination. + const char* data() const { return ptr_; } + int size() const { return length_; } + bool empty() const { return length_ == 0; } + + void clear() { ptr_ = NULL; length_ = 0; } + void set(const char* buffer, int len) { ptr_ = buffer; length_ = len; } + void set(const char* str) { + ptr_ = str; + length_ = static_cast<int>(strlen(str)); + } + void set(const void* buffer, int len) { + ptr_ = reinterpret_cast<const char*>(buffer); + length_ = len; + } + + char operator[](int i) const { return ptr_[i]; } + + void remove_prefix(int n) { + ptr_ += n; + length_ -= n; + } + + void remove_suffix(int n) { + length_ -= n; + } + + bool operator==(const StringPiece& x) const { + return ((length_ == x.length_) && + (memcmp(ptr_, x.ptr_, length_) == 0)); + } + bool operator!=(const StringPiece& x) const { + return !(*this == x); + } + +#define STRINGPIECE_BINARY_PREDICATE(cmp,auxcmp) \ + bool operator cmp (const StringPiece& x) const { \ + int r = memcmp(ptr_, x.ptr_, length_ < x.length_ ? length_ : x.length_); \ + return ((r auxcmp 0) || ((r == 0) && (length_ cmp x.length_))); \ + } + STRINGPIECE_BINARY_PREDICATE(<, <); + STRINGPIECE_BINARY_PREDICATE(<=, <); + STRINGPIECE_BINARY_PREDICATE(>=, >); + STRINGPIECE_BINARY_PREDICATE(>, >); +#undef STRINGPIECE_BINARY_PREDICATE + + int compare(const StringPiece& x) const { + int r = memcmp(ptr_, x.ptr_, length_ < x.length_ ? length_ : x.length_); + if (r == 0) { + if (length_ < x.length_) r = -1; + else if (length_ > x.length_) r = +1; + } + return r; + } + + string as_string() const { + return string(data(), size()); + } + + void CopyToString(string* target) const { + target->assign(ptr_, length_); + } + + // Does "this" start with "x" + bool starts_with(const StringPiece& x) const { + return ((length_ >= x.length_) && (memcmp(ptr_, x.ptr_, x.length_) == 0)); + } +}; + +} // namespace pcrecpp + +// ------------------------------------------------------------------ +// Functions used to create STL containers that use StringPiece +// Remember that a StringPiece's lifetime had better be less than +// that of the underlying string or char*. If it is not, then you +// cannot safely store a StringPiece into an STL container +// ------------------------------------------------------------------ + +#ifdef HAVE_TYPE_TRAITS +// This makes vector<StringPiece> really fast for some STL implementations +template<> struct __type_traits<pcrecpp::StringPiece> { + typedef __true_type has_trivial_default_constructor; + typedef __true_type has_trivial_copy_constructor; + typedef __true_type has_trivial_assignment_operator; + typedef __true_type has_trivial_destructor; + typedef __true_type is_POD_type; +}; +#endif + +// allow StringPiece to be logged +std::ostream& operator<<(std::ostream& o, const pcrecpp::StringPiece& piece); + +#endif /* _PCRE_STRINGPIECE_H */ diff --git a/pcre_stringpiece_unittest.cc b/pcre_stringpiece_unittest.cc new file mode 100644 index 0000000..d6a89e8 --- /dev/null +++ b/pcre_stringpiece_unittest.cc @@ -0,0 +1,145 @@ +// Copyright 2003 and onwards Google Inc. +// Author: Sanjay Ghemawat + +#include <stdio.h> +#include <map> +#include <algorithm> // for make_pair +#include <pcre_stringpiece.h> + +// CHECK dies with a fatal error if condition is not true. It is *not* +// controlled by NDEBUG, so the check will be executed regardless of +// compilation mode. Therefore, it is safe to do things like: +// CHECK(fp->Write(x) == 4) +#define CHECK(condition) do { \ + if (!(condition)) { \ + fprintf(stderr, "%s:%d: Check failed: %s\n", \ + __FILE__, __LINE__, #condition); \ + exit(1); \ + } \ +} while (0) + +using std::map; +using std::make_pair; +using pcrecpp::StringPiece; + +static void CheckSTLComparator() { + string s1("foo"); + string s2("bar"); + string s3("baz"); + + StringPiece p1(s1); + StringPiece p2(s2); + StringPiece p3(s3); + + typedef map<StringPiece, int> TestMap; + TestMap map; + + map.insert(make_pair(p1, 0)); + map.insert(make_pair(p2, 1)); + map.insert(make_pair(p3, 2)); + CHECK(map.size() == 3); + + TestMap::const_iterator iter = map.begin(); + CHECK(iter->second == 1); + ++iter; + CHECK(iter->second == 2); + ++iter; + CHECK(iter->second == 0); + ++iter; + CHECK(iter == map.end()); + + TestMap::iterator new_iter = map.find("zot"); + CHECK(new_iter == map.end()); + + new_iter = map.find("bar"); + CHECK(new_iter != map.end()); + + map.erase(new_iter); + CHECK(map.size() == 2); + + iter = map.begin(); + CHECK(iter->second == 2); + ++iter; + CHECK(iter->second == 0); + ++iter; + CHECK(iter == map.end()); +} + +static void CheckComparisonOperators() { +#define CMP_Y(op, x, y) \ + CHECK( (StringPiece((x)) op StringPiece((y)))); \ + CHECK( (StringPiece((x)).compare(StringPiece((y))) op 0)) + +#define CMP_N(op, x, y) \ + CHECK(!(StringPiece((x)) op StringPiece((y)))); \ + CHECK(!(StringPiece((x)).compare(StringPiece((y))) op 0)) + + CMP_Y(==, "", ""); + CMP_Y(==, "a", "a"); + CMP_Y(==, "aa", "aa"); + CMP_N(==, "a", ""); + CMP_N(==, "", "a"); + CMP_N(==, "a", "b"); + CMP_N(==, "a", "aa"); + CMP_N(==, "aa", "a"); + + CMP_N(!=, "", ""); + CMP_N(!=, "a", "a"); + CMP_N(!=, "aa", "aa"); + CMP_Y(!=, "a", ""); + CMP_Y(!=, "", "a"); + CMP_Y(!=, "a", "b"); + CMP_Y(!=, "a", "aa"); + CMP_Y(!=, "aa", "a"); + + CMP_Y(<, "a", "b"); + CMP_Y(<, "a", "aa"); + CMP_Y(<, "aa", "b"); + CMP_Y(<, "aa", "bb"); + CMP_N(<, "a", "a"); + CMP_N(<, "b", "a"); + CMP_N(<, "aa", "a"); + CMP_N(<, "b", "aa"); + CMP_N(<, "bb", "aa"); + + CMP_Y(<=, "a", "a"); + CMP_Y(<=, "a", "b"); + CMP_Y(<=, "a", "aa"); + CMP_Y(<=, "aa", "b"); + CMP_Y(<=, "aa", "bb"); + CMP_N(<=, "b", "a"); + CMP_N(<=, "aa", "a"); + CMP_N(<=, "b", "aa"); + CMP_N(<=, "bb", "aa"); + + CMP_N(>=, "a", "b"); + CMP_N(>=, "a", "aa"); + CMP_N(>=, "aa", "b"); + CMP_N(>=, "aa", "bb"); + CMP_Y(>=, "a", "a"); + CMP_Y(>=, "b", "a"); + CMP_Y(>=, "aa", "a"); + CMP_Y(>=, "b", "aa"); + CMP_Y(>=, "bb", "aa"); + + CMP_N(>, "a", "a"); + CMP_N(>, "a", "b"); + CMP_N(>, "a", "aa"); + CMP_N(>, "aa", "b"); + CMP_N(>, "aa", "bb"); + CMP_Y(>, "b", "a"); + CMP_Y(>, "aa", "a"); + CMP_Y(>, "b", "aa"); + CMP_Y(>, "bb", "aa"); + +#undef CMP_Y +#undef CMP_N +} + +int main(int argc, char** argv) { + CheckComparisonOperators(); + CheckSTLComparator(); + + printf("OK\n"); + return 0; +} @@ -2,14 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -This is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. See -the file Tech.Notes for some information on the internals. +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2004 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -41,11 +38,11 @@ POSSIBILITY OF SUCH DAMAGE. */ -/* Include the internals header, which itself includes Standard C headers plus -the external pcre header. */ +/* This module contains the external function pcre_study(), along with local +supporting functions. */ -#include "internal.h" +#include "pcre_internal.h" /************************************************* @@ -441,7 +438,8 @@ if ((re->options & (PCRE_ANCHORED|PCRE_FIRSTSET|PCRE_STARTLINE)) != 0) tables = re->tables; if (tables == NULL) - (void)pcre_fullinfo(external_re, NULL, PCRE_INFO_DEFAULT_TABLES, &tables); + (void)pcre_fullinfo(external_re, NULL, PCRE_INFO_DEFAULT_TABLES, + (void *)(&tables)); compile_block.lcc = tables + lcc_offset; compile_block.fcc = tables + fcc_offset; @@ -481,4 +479,4 @@ memcpy(study->start_bits, start_bits, sizeof(start_bits)); return extra; } -/* End of study.c */ +/* End of pcre_study.c */ diff --git a/ucptypetable.c b/pcre_tables.c index 129529b..4f442ea 100644 --- a/ucptypetable.c +++ b/pcre_tables.c @@ -2,14 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -This is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. See -the file Tech.Notes for some information on the internals. +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2004 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -40,17 +37,54 @@ POSSIBILITY OF SUCH DAMAGE. ----------------------------------------------------------------------------- */ -/* This module contains a table for translating Unicode property names into -code values for the ucp_findchar function. It is in a separate module so that -it can be included both in the main pcre library, and into pcretest (for -printing out internals). */ -typedef struct { - const char *name; - int value; -} ucp_type_table; +/* This module contains some fixed tables that are used by more than one of the +PCRE code modules. */ + + +#include "pcre_internal.h" + + +/* Table of sizes for the fixed-length opcodes. It's defined in a macro so that +the definition is next to the definition of the opcodes in internal.h. */ + +const uschar _pcre_OP_lengths[] = { OP_LENGTHS }; + + + +/************************************************* +* Tables for UTF-8 support * +*************************************************/ + +/* These are the breakpoints for different numbers of bytes in a UTF-8 +character. */ + +const int _pcre_utf8_table1[] = + { 0x7f, 0x7ff, 0xffff, 0x1fffff, 0x3ffffff, 0x7fffffff}; + +const int _pcre_utf8_table1_size = sizeof(_pcre_utf8_table1)/sizeof(int); + +/* These are the indicator bits and the mask for the data bits to set in the +first byte of a character, indexed by the number of additional bytes. */ -static ucp_type_table utt[] = { +const int _pcre_utf8_table2[] = { 0, 0xc0, 0xe0, 0xf0, 0xf8, 0xfc}; +const int _pcre_utf8_table3[] = { 0xff, 0x1f, 0x0f, 0x07, 0x03, 0x01}; + +/* Table of the number of extra characters, indexed by the first character +masked with 0x3f. The highest number for a valid UTF-8 character is in fact +0x3d. */ + +const uschar _pcre_utf8_table4[] = { + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, + 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, + 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2, + 3,3,3,3,3,3,3,3,4,4,4,4,5,5,5,5 }; + +/* This table translates Unicode property names into code values for the +ucp_findchar() function. It is used by pcretest as well as by the library +functions. */ + +const ucp_type_table _pcre_utt[] = { { "C", 128 + ucp_C }, { "Cc", ucp_Cc }, { "Cf", ucp_Cf }, @@ -90,4 +124,6 @@ static ucp_type_table utt[] = { { "Zs", ucp_Zs } }; -/* End of ucptypetable.c */ +const int _pcre_utt_size = sizeof(_pcre_utt)/sizeof(ucp_type_table); + +/* End of pcre_tables.c */ diff --git a/pcre_try_flipped.c b/pcre_try_flipped.c new file mode 100644 index 0000000..a07bb23 --- /dev/null +++ b/pcre_try_flipped.c @@ -0,0 +1,132 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains an internal function that tests a compiled pattern to +see if it was compiled with the opposite endianness. If so, it uses an +auxiliary local function to flip the appropriate bytes. */ + + +#include "pcre_internal.h" + + +/************************************************* +* Flip bytes in an integer * +*************************************************/ + +/* This function is called when the magic number in a regex doesn't match, in +order to flip its bytes to see if we are dealing with a pattern that was +compiled on a host of different endianness. If so, this function is used to +flip other byte values. + +Arguments: + value the number to flip + n the number of bytes to flip (assumed to be 2 or 4) + +Returns: the flipped value +*/ + +static long int +byteflip(long int value, int n) +{ +if (n == 2) return ((value & 0x00ff) << 8) | ((value & 0xff00) >> 8); +return ((value & 0x000000ff) << 24) | + ((value & 0x0000ff00) << 8) | + ((value & 0x00ff0000) >> 8) | + ((value & 0xff000000) >> 24); +} + + + +/************************************************* +* Test for a byte-flipped compiled regex * +*************************************************/ + +/* This function is called from pcre_exec(), pcre_dfa_exec(), and also from +pcre_fullinfo(). Its job is to test whether the regex is byte-flipped - that +is, it was compiled on a system of opposite endianness. The function is called +only when the native MAGIC_NUMBER test fails. If the regex is indeed flipped, +we flip all the relevant values into a different data block, and return it. + +Arguments: + re points to the regex + study points to study data, or NULL + internal_re points to a new regex block + internal_study points to a new study block + +Returns: the new block if is is indeed a byte-flipped regex + NULL if it is not +*/ + +EXPORT real_pcre * +_pcre_try_flipped(const real_pcre *re, real_pcre *internal_re, + const pcre_study_data *study, pcre_study_data *internal_study) +{ +if (byteflip(re->magic_number, sizeof(re->magic_number)) != MAGIC_NUMBER) + return NULL; + +*internal_re = *re; /* To copy other fields */ +internal_re->size = byteflip(re->size, sizeof(re->size)); +internal_re->options = byteflip(re->options, sizeof(re->options)); +internal_re->top_bracket = + (pcre_uint16)byteflip(re->top_bracket, sizeof(re->top_bracket)); +internal_re->top_backref = + (pcre_uint16)byteflip(re->top_backref, sizeof(re->top_backref)); +internal_re->first_byte = + (pcre_uint16)byteflip(re->first_byte, sizeof(re->first_byte)); +internal_re->req_byte = + (pcre_uint16)byteflip(re->req_byte, sizeof(re->req_byte)); +internal_re->name_table_offset = + (pcre_uint16)byteflip(re->name_table_offset, sizeof(re->name_table_offset)); +internal_re->name_entry_size = + (pcre_uint16)byteflip(re->name_entry_size, sizeof(re->name_entry_size)); +internal_re->name_count = + (pcre_uint16)byteflip(re->name_count, sizeof(re->name_count)); + +if (study != NULL) + { + *internal_study = *study; /* To copy other fields */ + internal_study->size = byteflip(study->size, sizeof(study->size)); + internal_study->options = byteflip(study->options, sizeof(study->options)); + } + +return internal_re; +} + +/* End of pcre_tryflipped.c */ diff --git a/pcre_ucp_findchar.c b/pcre_ucp_findchar.c new file mode 100644 index 0000000..9f8de06 --- /dev/null +++ b/pcre_ucp_findchar.c @@ -0,0 +1,53 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module compiles code for supporting the use of Unicode character +properties. We use the (embryonic at the time of writing) UCP library, by +including some of its files, copies of which have been put in the PCRE +distribution. There is a macro in pcre_internal.h that changes the name +ucp_findchar into _pcre_ucp_findchar. */ + + +#include "pcre_internal.h" + +#include "ucp_findchar.c" + + +/* End of pcre_ucp_findchar.c */ diff --git a/pcre_valid_utf8.c b/pcre_valid_utf8.c new file mode 100644 index 0000000..72f0f5e --- /dev/null +++ b/pcre_valid_utf8.c @@ -0,0 +1,130 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains an internal function for validating UTF-8 character +strings. */ + + +#include "pcre_internal.h" + + +/************************************************* +* Validate a UTF-8 string * +*************************************************/ + +/* This function is called (optionally) at the start of compile or match, to +validate that a supposed UTF-8 string is actually valid. The early check means +that subsequent code can assume it is dealing with a valid string. The check +can be turned off for maximum performance, but the consequences of supplying +an invalid string are then undefined. + +Arguments: + string points to the string + length length of string, or -1 if the string is zero-terminated + +Returns: < 0 if the string is a valid UTF-8 string + >= 0 otherwise; the value is the offset of the bad byte +*/ + +EXPORT int +_pcre_valid_utf8(const uschar *string, int length) +{ +register const uschar *p; + +if (length < 0) + { + for (p = string; *p != 0; p++); + length = p - string; + } + +for (p = string; length-- > 0; p++) + { + register int ab; + register int c = *p; + if (c < 128) continue; + if ((c & 0xc0) != 0xc0) return p - string; + ab = _pcre_utf8_table4[c & 0x3f]; /* Number of additional bytes */ + if (length < ab) return p - string; + length -= ab; + + /* Check top bits in the second byte */ + if ((*(++p) & 0xc0) != 0x80) return p - string; + + /* Check for overlong sequences for each different length */ + switch (ab) + { + /* Check for xx00 000x */ + case 1: + if ((c & 0x3e) == 0) return p - string; + continue; /* We know there aren't any more bytes to check */ + + /* Check for 1110 0000, xx0x xxxx */ + case 2: + if (c == 0xe0 && (*p & 0x20) == 0) return p - string; + break; + + /* Check for 1111 0000, xx00 xxxx */ + case 3: + if (c == 0xf0 && (*p & 0x30) == 0) return p - string; + break; + + /* Check for 1111 1000, xx00 0xxx */ + case 4: + if (c == 0xf8 && (*p & 0x38) == 0) return p - string; + break; + + /* Check for leading 0xfe or 0xff, and then for 1111 1100, xx00 00xx */ + case 5: + if (c == 0xfe || c == 0xff || + (c == 0xfc && (*p & 0x3c) == 0)) return p - string; + break; + } + + /* Check for valid bytes after the 2nd, if any; all must start 10 */ + while (--ab > 0) + { + if ((*(++p) & 0xc0) != 0x80) return p - string; + } + } + +return -1; +} + +/* End of pcre_valid_utf8.c */ diff --git a/pcre_version.c b/pcre_version.c new file mode 100644 index 0000000..d296eea --- /dev/null +++ b/pcre_version.c @@ -0,0 +1,61 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains the external function pcre_version(), which returns a +string that identifies the PCRE version that is in use. */ + + +#include "pcre_internal.h" + + +/************************************************* +* Return version string * +*************************************************/ + +#define STRING(a) # a +#define XSTRING(s) STRING(s) + +EXPORT const char * +pcre_version(void) +{ +return XSTRING(PCRE_MAJOR) "." XSTRING(PCRE_MINOR) " " XSTRING(PCRE_DATE); +} + +/* End of pcre_version.c */ diff --git a/pcre_xclass.c b/pcre_xclass.c new file mode 100644 index 0000000..40d2654 --- /dev/null +++ b/pcre_xclass.c @@ -0,0 +1,121 @@ +/************************************************* +* Perl-Compatible Regular Expressions * +*************************************************/ + +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. + + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge + +----------------------------------------------------------------------------- +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + * Neither the name of the University of Cambridge nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. +----------------------------------------------------------------------------- +*/ + + +/* This module contains an internal function that is used to match an extended +class (one that contains characters whose values are > 255). It is used by both +pcre_exec() and pcre_def_exec(). */ + + +#include "pcre_internal.h" + + +/************************************************* +* Match character against an XCLASS * +*************************************************/ + +/* This function is called to match a character against an extended class that +might contain values > 255. + +Arguments: + c the character + data points to the flag byte of the XCLASS data + +Returns: TRUE if character matches, else FALSE +*/ + +EXPORT BOOL +_pcre_xclass(int c, const uschar *data) +{ +int t; +BOOL negated = (*data & XCL_NOT) != 0; + +/* Character values < 256 are matched against a bitmap, if one is present. If +not, we still carry on, because there may be ranges that start below 256 in the +additional data. */ + +if (c < 256) + { + if ((*data & XCL_MAP) != 0 && (data[1 + c/8] & (1 << (c&7))) != 0) + return !negated; /* char found */ + } + +/* First skip the bit map if present. Then match against the list of Unicode +properties or large chars or ranges that end with a large char. We won't ever +encounter XCL_PROP or XCL_NOTPROP when UCP support is not compiled. */ + +if ((*data++ & XCL_MAP) != 0) data += 32; + +while ((t = *data++) != XCL_END) + { + int x, y; + if (t == XCL_SINGLE) + { + GETCHARINC(x, data); + if (c == x) return !negated; + } + else if (t == XCL_RANGE) + { + GETCHARINC(x, data); + GETCHARINC(y, data); + if (c >= x && c <= y) return !negated; + } + +#ifdef SUPPORT_UCP + else /* XCL_PROP & XCL_NOTPROP */ + { + int chartype, othercase; + int rqdtype = *data++; + int category = ucp_findchar(c, &chartype, &othercase); + if (rqdtype >= 128) + { + if ((rqdtype - 128 == category) == (t == XCL_PROP)) return !negated; + } + else + { + if ((rqdtype == chartype) == (t == XCL_PROP)) return !negated; + } + } +#endif /* SUPPORT_UCP */ + } + +return negated; /* char did not match */ +} + +/* End of pcre_xclass.c */ diff --git a/pcrecpp.cc b/pcrecpp.cc new file mode 100644 index 0000000..0876abf --- /dev/null +++ b/pcrecpp.cc @@ -0,0 +1,770 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: Sanjay Ghemawat + +#include <stdlib.h> +#include <stdio.h> +#include <ctype.h> +#include <limits.h> /* for SHRT_MIN, USHRT_MAX, etc */ +#include <assert.h> +#include <errno.h> +#include <string> +#include "config.h" +// We need this to compile the proper dll on windows/msys. This is copied +// from pcre_internal.h. It would probably be better just to include that. +#define PCRE_DEFINITION /* Win32 __declspec(export) trigger for .dll */ +#include "pcre.h" +#include "pcre_stringpiece.h" +#include "pcrecpp.h" + + +namespace pcrecpp { + +// Maximum number of args we can set +static const int kMaxArgs = 16; +static const int kVecSize = (1 + kMaxArgs) * 3; // results + PCRE workspace + +// Special object that stands-in for no argument +Arg no_arg((void*)NULL); + +// If a regular expression has no error, its error_ field points here +static const string empty_string; + +// If the user doesn't ask for any options, we just use this one +static RE_Options default_options; + +void RE::Init(const char* pat, const RE_Options* options) { + pattern_ = pat; + if (options == NULL) { + options_ = default_options; + } else { + options_ = *options; + } + error_ = &empty_string; + re_full_ = NULL; + re_partial_ = NULL; + + re_partial_ = Compile(UNANCHORED); + if (re_partial_ != NULL) { + // Check for complicated patterns. The following change is + // conservative in that it may treat some "simple" patterns + // as "complex" (e.g., if the vertical bar is in a character + // class or is escaped). But it seems good enough. + if (strchr(pat, '|') == NULL) { + // Simple pattern: we can use position-based checks to perform + // fully anchored matches + re_full_ = re_partial_; + } else { + // We need a special pattern for anchored matches + re_full_ = Compile(ANCHOR_BOTH); + } + } +} + +RE::~RE() { + if (re_full_ != NULL && re_full_ != re_partial_) (*pcre_free)(re_full_); + if (re_partial_ != NULL) (*pcre_free)(re_partial_); + if (error_ != &empty_string) delete error_; +} + +pcre* RE::Compile(Anchor anchor) { + // First, convert RE_Options into pcre options + int pcre_options = 0; + if (options_.utf8()) + pcre_options |= PCRE_UTF8; + + // Special treatment for anchoring. This is needed because at + // runtime pcre only provides an option for anchoring at the + // beginning of a string (unless you use offset). + // + // There are three types of anchoring we want: + // UNANCHORED Compile the original pattern, and use + // a pcre unanchored match. + // ANCHOR_START Compile the original pattern, and use + // a pcre anchored match. + // ANCHOR_BOTH Tack a "\z" to the end of the original pattern + // and use a pcre anchored match. + + const char* compile_error; + int eoffset; + pcre* re; + if (anchor != ANCHOR_BOTH) { + re = pcre_compile(pattern_.c_str(), pcre_options, + &compile_error, &eoffset, NULL); + } else { + // Tack a '\z' at the end of RE. Parenthesize it first so that + // the '\z' applies to all top-level alternatives in the regexp. + string wrapped = "(?:"; // A non-counting grouping operator + wrapped += pattern_; + wrapped += ")\\z"; + re = pcre_compile(wrapped.c_str(), pcre_options, + &compile_error, &eoffset, NULL); + } + if (re == NULL) { + if (error_ == &empty_string) error_ = new string(compile_error); + } + return re; +} + +/***** Matching interfaces *****/ + +bool RE::FullMatch(const StringPiece& text, + const Arg& ptr1, + const Arg& ptr2, + const Arg& ptr3, + const Arg& ptr4, + const Arg& ptr5, + const Arg& ptr6, + const Arg& ptr7, + const Arg& ptr8, + const Arg& ptr9, + const Arg& ptr10, + const Arg& ptr11, + const Arg& ptr12, + const Arg& ptr13, + const Arg& ptr14, + const Arg& ptr15, + const Arg& ptr16) const { + const Arg* args[kMaxArgs]; + int n = 0; + if (&ptr1 == &no_arg) goto done; args[n++] = &ptr1; + if (&ptr2 == &no_arg) goto done; args[n++] = &ptr2; + if (&ptr3 == &no_arg) goto done; args[n++] = &ptr3; + if (&ptr4 == &no_arg) goto done; args[n++] = &ptr4; + if (&ptr5 == &no_arg) goto done; args[n++] = &ptr5; + if (&ptr6 == &no_arg) goto done; args[n++] = &ptr6; + if (&ptr7 == &no_arg) goto done; args[n++] = &ptr7; + if (&ptr8 == &no_arg) goto done; args[n++] = &ptr8; + if (&ptr9 == &no_arg) goto done; args[n++] = &ptr9; + if (&ptr10 == &no_arg) goto done; args[n++] = &ptr10; + if (&ptr11 == &no_arg) goto done; args[n++] = &ptr11; + if (&ptr12 == &no_arg) goto done; args[n++] = &ptr12; + if (&ptr13 == &no_arg) goto done; args[n++] = &ptr13; + if (&ptr14 == &no_arg) goto done; args[n++] = &ptr14; + if (&ptr15 == &no_arg) goto done; args[n++] = &ptr15; + if (&ptr16 == &no_arg) goto done; args[n++] = &ptr16; + done: + + int consumed; + int vec[kVecSize]; + return DoMatchImpl(text, ANCHOR_BOTH, &consumed, args, n, vec, kVecSize); +} + +bool RE::PartialMatch(const StringPiece& text, + const Arg& ptr1, + const Arg& ptr2, + const Arg& ptr3, + const Arg& ptr4, + const Arg& ptr5, + const Arg& ptr6, + const Arg& ptr7, + const Arg& ptr8, + const Arg& ptr9, + const Arg& ptr10, + const Arg& ptr11, + const Arg& ptr12, + const Arg& ptr13, + const Arg& ptr14, + const Arg& ptr15, + const Arg& ptr16) const { + const Arg* args[kMaxArgs]; + int n = 0; + if (&ptr1 == &no_arg) goto done; args[n++] = &ptr1; + if (&ptr2 == &no_arg) goto done; args[n++] = &ptr2; + if (&ptr3 == &no_arg) goto done; args[n++] = &ptr3; + if (&ptr4 == &no_arg) goto done; args[n++] = &ptr4; + if (&ptr5 == &no_arg) goto done; args[n++] = &ptr5; + if (&ptr6 == &no_arg) goto done; args[n++] = &ptr6; + if (&ptr7 == &no_arg) goto done; args[n++] = &ptr7; + if (&ptr8 == &no_arg) goto done; args[n++] = &ptr8; + if (&ptr9 == &no_arg) goto done; args[n++] = &ptr9; + if (&ptr10 == &no_arg) goto done; args[n++] = &ptr10; + if (&ptr11 == &no_arg) goto done; args[n++] = &ptr11; + if (&ptr12 == &no_arg) goto done; args[n++] = &ptr12; + if (&ptr13 == &no_arg) goto done; args[n++] = &ptr13; + if (&ptr14 == &no_arg) goto done; args[n++] = &ptr14; + if (&ptr15 == &no_arg) goto done; args[n++] = &ptr15; + if (&ptr16 == &no_arg) goto done; args[n++] = &ptr16; + done: + + int consumed; + int vec[kVecSize]; + return DoMatchImpl(text, UNANCHORED, &consumed, args, n, vec, kVecSize); +} + +bool RE::Consume(StringPiece* input, + const Arg& ptr1, + const Arg& ptr2, + const Arg& ptr3, + const Arg& ptr4, + const Arg& ptr5, + const Arg& ptr6, + const Arg& ptr7, + const Arg& ptr8, + const Arg& ptr9, + const Arg& ptr10, + const Arg& ptr11, + const Arg& ptr12, + const Arg& ptr13, + const Arg& ptr14, + const Arg& ptr15, + const Arg& ptr16) const { + const Arg* args[kMaxArgs]; + int n = 0; + if (&ptr1 == &no_arg) goto done; args[n++] = &ptr1; + if (&ptr2 == &no_arg) goto done; args[n++] = &ptr2; + if (&ptr3 == &no_arg) goto done; args[n++] = &ptr3; + if (&ptr4 == &no_arg) goto done; args[n++] = &ptr4; + if (&ptr5 == &no_arg) goto done; args[n++] = &ptr5; + if (&ptr6 == &no_arg) goto done; args[n++] = &ptr6; + if (&ptr7 == &no_arg) goto done; args[n++] = &ptr7; + if (&ptr8 == &no_arg) goto done; args[n++] = &ptr8; + if (&ptr9 == &no_arg) goto done; args[n++] = &ptr9; + if (&ptr10 == &no_arg) goto done; args[n++] = &ptr10; + if (&ptr11 == &no_arg) goto done; args[n++] = &ptr11; + if (&ptr12 == &no_arg) goto done; args[n++] = &ptr12; + if (&ptr13 == &no_arg) goto done; args[n++] = &ptr13; + if (&ptr14 == &no_arg) goto done; args[n++] = &ptr14; + if (&ptr15 == &no_arg) goto done; args[n++] = &ptr15; + if (&ptr16 == &no_arg) goto done; args[n++] = &ptr16; + done: + + int consumed; + int vec[kVecSize]; + if (DoMatchImpl(*input, ANCHOR_START, &consumed, + args, n, vec, kVecSize)) { + input->remove_prefix(consumed); + return true; + } else { + return false; + } +} + +bool RE::FindAndConsume(StringPiece* input, + const Arg& ptr1, + const Arg& ptr2, + const Arg& ptr3, + const Arg& ptr4, + const Arg& ptr5, + const Arg& ptr6, + const Arg& ptr7, + const Arg& ptr8, + const Arg& ptr9, + const Arg& ptr10, + const Arg& ptr11, + const Arg& ptr12, + const Arg& ptr13, + const Arg& ptr14, + const Arg& ptr15, + const Arg& ptr16) const { + const Arg* args[kMaxArgs]; + int n = 0; + if (&ptr1 == &no_arg) goto done; args[n++] = &ptr1; + if (&ptr2 == &no_arg) goto done; args[n++] = &ptr2; + if (&ptr3 == &no_arg) goto done; args[n++] = &ptr3; + if (&ptr4 == &no_arg) goto done; args[n++] = &ptr4; + if (&ptr5 == &no_arg) goto done; args[n++] = &ptr5; + if (&ptr6 == &no_arg) goto done; args[n++] = &ptr6; + if (&ptr7 == &no_arg) goto done; args[n++] = &ptr7; + if (&ptr8 == &no_arg) goto done; args[n++] = &ptr8; + if (&ptr9 == &no_arg) goto done; args[n++] = &ptr9; + if (&ptr10 == &no_arg) goto done; args[n++] = &ptr10; + if (&ptr11 == &no_arg) goto done; args[n++] = &ptr11; + if (&ptr12 == &no_arg) goto done; args[n++] = &ptr12; + if (&ptr13 == &no_arg) goto done; args[n++] = &ptr13; + if (&ptr14 == &no_arg) goto done; args[n++] = &ptr14; + if (&ptr15 == &no_arg) goto done; args[n++] = &ptr15; + if (&ptr16 == &no_arg) goto done; args[n++] = &ptr16; + done: + + int consumed; + int vec[kVecSize]; + if (DoMatchImpl(*input, UNANCHORED, &consumed, + args, n, vec, kVecSize)) { + input->remove_prefix(consumed); + return true; + } else { + return false; + } +} + +bool RE::Replace(const StringPiece& rewrite, + string *str) const { + int vec[kVecSize]; + int matches = TryMatch(*str, 0, UNANCHORED, vec, kVecSize); + if (matches == 0) + return false; + + string s; + if (!Rewrite(&s, rewrite, *str, vec, matches)) + return false; + + assert(vec[0] >= 0); + assert(vec[1] >= 0); + str->replace(vec[0], vec[1] - vec[0], s); + return true; +} + +int RE::GlobalReplace(const StringPiece& rewrite, + string *str) const { + int count = 0; + int vec[kVecSize]; + string out; + int start = 0; + int lastend = -1; + + for (; start <= static_cast<int>(str->length()); count++) { + int matches = TryMatch(*str, start, UNANCHORED, vec, kVecSize); + if (matches <= 0) + break; + int matchstart = vec[0], matchend = vec[1]; + assert(matchstart >= start); + assert(matchend >= matchstart); + if (matchstart == matchend && matchstart == lastend) { + // advance one character if we matched an empty string at the same + // place as the last match occurred + if (start < static_cast<int>(str->length())) + out.push_back((*str)[start]); + start++; + } else { + out.append(*str, start, matchstart - start); + Rewrite(&out, rewrite, *str, vec, matches); + start = matchend; + lastend = matchend; + count++; + } + } + + if (count == 0) + return 0; + + if (start < static_cast<int>(str->length())) + out.append(*str, start, str->length() - start); + swap(out, *str); + return count; +} + +bool RE::Extract(const StringPiece& rewrite, + const StringPiece& text, + string *out) const { + int vec[kVecSize]; + int matches = TryMatch(text, 0, UNANCHORED, vec, kVecSize); + if (matches == 0) + return false; + out->clear(); + return Rewrite(out, rewrite, text, vec, matches); +} + +/***** Actual matching and rewriting code *****/ + +int RE::TryMatch(const StringPiece& text, + int startpos, + Anchor anchor, + int *vec, + int vecsize) const { + pcre* re = (anchor == ANCHOR_BOTH) ? re_full_ : re_partial_; + if (re == NULL) { + //fprintf(stderr, "Matching against invalid re: %s\n", error_->c_str()); + return 0; + } + + pcre_extra extra = { 0 }; + if (options_.match_limit() > 0) { + extra.flags = PCRE_EXTRA_MATCH_LIMIT; + extra.match_limit = options_.match_limit(); + } + int rc = pcre_exec(re, // The regular expression object + &extra, + text.data(), + text.size(), + startpos, + (anchor == UNANCHORED) ? 0 : PCRE_ANCHORED, + vec, + vecsize); + + // Handle errors + if (rc == PCRE_ERROR_NOMATCH) { + return 0; + } else if (rc < 0) { + //fprintf(stderr, "Unexpected return code: %d when matching '%s'\n", + // re, pattern_.c_str()); + return 0; + } else if (rc == 0) { + // pcre_exec() returns 0 as a special case when the number of + // capturing subpatterns exceeds the size of the vector. + // When this happens, there is a match and the output vector + // is filled, but we miss out on the positions of the extra subpatterns. + rc = vecsize / 2; + } + + if ((anchor == ANCHOR_BOTH) && (re_full_ == re_partial_)) { + // We need an extra check to make sure that the match extended + // to the end of the input string + assert(vec[0] == 0); // PCRE_ANCHORED forces starting match + if (vec[1] != text.size()) return 0; // Did not get ending match + } + + return rc; +} + +bool RE::DoMatchImpl(const StringPiece& text, + Anchor anchor, + int* consumed, + const Arg* const* args, + int n, + int* vec, + int vecsize) const { + assert((1 + n) * 3 <= vecsize); // results + PCRE workspace + int matches = TryMatch(text, 0, anchor, vec, vecsize); + assert(matches >= 0); // TryMatch never returns negatives + if (matches == 0) + return false; + + *consumed = vec[1]; + + if (args == NULL) { + // We are not interested in results + return true; + } + + // If we got here, we must have matched the whole pattern. + // We do not need (can not do) any more checks on the value of 'matches' here + // -- see the comment for TryMatch. + for (int i = 0; i < n; i++) { + const int start = vec[2*(i+1)]; + const int limit = vec[2*(i+1)+1]; + if (!args[i]->Parse(text.data() + start, limit-start)) { + // TODO: Should we indicate what the error was? + return false; + } + } + + return true; +} + +bool RE::DoMatch(const StringPiece& text, + Anchor anchor, + int* consumed, + const Arg* const args[], + int n) const { + assert(n >= 0); + size_t const vecsize = (1 + n) * 3; // results + PCRE workspace + // (as for kVecSize) + int space[21]; // use stack allocation for small vecsize (common case) + int* vec = vecsize <= 21 ? space : new int[vecsize]; + bool retval = DoMatchImpl(text, anchor, consumed, args, n, vec, vecsize); + if (vec != space) delete [] vec; + return retval; +} + +bool RE::Rewrite(string *out, const StringPiece &rewrite, + const StringPiece &text, int *vec, int veclen) const { + for (const char *s = rewrite.data(), *end = s + rewrite.size(); + s < end; s++) { + int c = *s; + if (c == '\\') { + c = *++s; + if (isdigit(c)) { + int n = (c - '0'); + if (n >= veclen) { + //fprintf(stderr, requested group %d in regexp %.*s\n", + // n, rewrite.size(), rewrite.data()); + return false; + } + int start = vec[2 * n]; + if (start >= 0) + out->append(text.data() + start, vec[2 * n + 1] - start); + } else if (c == '\\') { + out->push_back('\\'); + } else { + //fprintf(stderr, "invalid rewrite pattern: %.*s\n", + // rewrite.size(), rewrite.data()); + return false; + } + } else { + out->push_back(c); + } + } + return true; +} + +// Return the number of capturing subpatterns, or -1 if the +// regexp wasn't valid on construction. +int RE::NumberOfCapturingGroups() { + if (re_partial_ == NULL) return -1; + + int result; + int pcre_retval = pcre_fullinfo(re_partial_, // The regular expression object + NULL, // We did not study the pattern + PCRE_INFO_CAPTURECOUNT, + &result); + assert(pcre_retval == 0); + return result; +} + +/***** Parsers for various types *****/ + +bool Arg::parse_null(const char* str, int n, void* dest) { + // We fail if somebody asked us to store into a non-NULL void* pointer + return (dest == NULL); +} + +bool Arg::parse_string(const char* str, int n, void* dest) { + reinterpret_cast<string*>(dest)->assign(str, n); + return true; +} + +bool Arg::parse_stringpiece(const char* str, int n, void* dest) { + reinterpret_cast<StringPiece*>(dest)->set(str, n); + return true; +} + +bool Arg::parse_char(const char* str, int n, void* dest) { + if (n != 1) return false; + *(reinterpret_cast<char*>(dest)) = str[0]; + return true; +} + +bool Arg::parse_uchar(const char* str, int n, void* dest) { + if (n != 1) return false; + *(reinterpret_cast<unsigned char*>(dest)) = str[0]; + return true; +} + +// Largest number spec that we are willing to parse +static const int kMaxNumberLength = 32; + +// REQUIRES "buf" must have length at least kMaxNumberLength+1 +// REQUIRES "n > 0" +// Copies "str" into "buf" and null-terminates if necessary. +// Returns one of: +// a. "str" if no termination is needed +// b. "buf" if the string was copied and null-terminated +// c. "" if the input was invalid and has no hope of being parsed +static const char* TerminateNumber(char* buf, const char* str, int n) { + if ((n > 0) && isspace(*str)) { + // We are less forgiving than the strtoxxx() routines and do not + // allow leading spaces. + return ""; + } + + // See if the character right after the input text may potentially + // look like a digit. + if (isdigit(str[n]) || + ((str[n] >= 'a') && (str[n] <= 'f')) || + ((str[n] >= 'A') && (str[n] <= 'F'))) { + if (n > kMaxNumberLength) return ""; // Input too big to be a valid number + memcpy(buf, str, n); + buf[n] = '\0'; + return buf; + } else { + // We can parse right out of the supplied string, so return it. + return str; + } +} + +bool Arg::parse_long_radix(const char* str, + int n, + void* dest, + int radix) { + if (n == 0) return false; + char buf[kMaxNumberLength+1]; + str = TerminateNumber(buf, str, n); + char* end; + errno = 0; + long r = strtol(str, &end, radix); + if (end != str + n) return false; // Leftover junk + if (errno) return false; + *(reinterpret_cast<long*>(dest)) = r; + return true; +} + +bool Arg::parse_ulong_radix(const char* str, + int n, + void* dest, + int radix) { + if (n == 0) return false; + char buf[kMaxNumberLength+1]; + str = TerminateNumber(buf, str, n); + char* end; + errno = 0; + unsigned long r = strtoul(str, &end, radix); + if (end != str + n) return false; // Leftover junk + if (errno) return false; + *(reinterpret_cast<unsigned long*>(dest)) = r; + return true; +} + +bool Arg::parse_short_radix(const char* str, + int n, + void* dest, + int radix) { + long r; + if (!parse_long_radix(str, n, &r, radix)) return false; // Could not parse + if (r < SHRT_MIN || r > SHRT_MAX) return false; // Out of range + *(reinterpret_cast<short*>(dest)) = r; + return true; +} + +bool Arg::parse_ushort_radix(const char* str, + int n, + void* dest, + int radix) { + unsigned long r; + if (!parse_ulong_radix(str, n, &r, radix)) return false; // Could not parse + if (r > USHRT_MAX) return false; // Out of range + *(reinterpret_cast<unsigned short*>(dest)) = r; + return true; +} + +bool Arg::parse_int_radix(const char* str, + int n, + void* dest, + int radix) { + long r; + if (!parse_long_radix(str, n, &r, radix)) return false; // Could not parse + if (r < INT_MIN || r > INT_MAX) return false; // Out of range + *(reinterpret_cast<int*>(dest)) = r; + return true; +} + +bool Arg::parse_uint_radix(const char* str, + int n, + void* dest, + int radix) { + unsigned long r; + if (!parse_ulong_radix(str, n, &r, radix)) return false; // Could not parse + if (r > UINT_MAX) return false; // Out of range + *(reinterpret_cast<unsigned int*>(dest)) = r; + return true; +} + +bool Arg::parse_longlong_radix(const char* str, + int n, + void* dest, + int radix) { +#ifndef HAVE_LONG_LONG + return false; +#else + if (n == 0) return false; + char buf[kMaxNumberLength+1]; + str = TerminateNumber(buf, str, n); + char* end; + errno = 0; +#if defined HAVE_STRTOQ + long long r = strtoq(str, &end, radix); +#elif defined HAVE_STRTOLL + long long r = strtoll(str, &end, radix); +#else +#error parse_longlong_radix: cannot convert input to a long-long +#endif + if (end != str + n) return false; // Leftover junk + if (errno) return false; + *(reinterpret_cast<long long*>(dest)) = r; + return true; +#endif /* HAVE_LONG_LONG */ +} + +bool Arg::parse_ulonglong_radix(const char* str, + int n, + void* dest, + int radix) { +#ifndef HAVE_UNSIGNED_LONG_LONG + return false; +#else + if (n == 0) return false; + char buf[kMaxNumberLength+1]; + str = TerminateNumber(buf, str, n); + char* end; + errno = 0; +#if defined HAVE_STRTOQ + unsigned long long r = strtouq(str, &end, radix); +#elif defined HAVE_STRTOLL + unsigned long long r = strtoull(str, &end, radix); +#else +#error parse_ulonglong_radix: cannot convert input to a long-long +#endif + if (end != str + n) return false; // Leftover junk + if (errno) return false; + *(reinterpret_cast<unsigned long long*>(dest)) = r; + return true; +#endif /* HAVE_UNSIGNED_LONG_LONG */ +} + +bool Arg::parse_double(const char* str, int n, void* dest) { + if (n == 0) return false; + static const int kMaxLength = 200; + char buf[kMaxLength]; + if (n >= kMaxLength) return false; + memcpy(buf, str, n); + buf[n] = '\0'; + errno = 0; + char* end; + double r = strtod(buf, &end); + if (end != buf + n) return false; // Leftover junk + if (errno) return false; + *(reinterpret_cast<double*>(dest)) = r; + return true; +} + +bool Arg::parse_float(const char* str, int n, void* dest) { + double r; + if (!parse_double(str, n, &r)) return false; + *(reinterpret_cast<float*>(dest)) = static_cast<float>(r); + return true; +} + + +#define DEFINE_INTEGER_PARSERS(name) \ + bool Arg::parse_##name(const char* str, int n, void* dest) { \ + return parse_##name##_radix(str, n, dest, 10); \ + } \ + bool Arg::parse_##name##_hex(const char* str, int n, void* dest) { \ + return parse_##name##_radix(str, n, dest, 16); \ + } \ + bool Arg::parse_##name##_octal(const char* str, int n, void* dest) { \ + return parse_##name##_radix(str, n, dest, 8); \ + } \ + bool Arg::parse_##name##_cradix(const char* str, int n, void* dest) { \ + return parse_##name##_radix(str, n, dest, 0); \ + } + +DEFINE_INTEGER_PARSERS(short); +DEFINE_INTEGER_PARSERS(ushort); +DEFINE_INTEGER_PARSERS(int); +DEFINE_INTEGER_PARSERS(uint); +DEFINE_INTEGER_PARSERS(long); +DEFINE_INTEGER_PARSERS(ulong); +DEFINE_INTEGER_PARSERS(longlong); +DEFINE_INTEGER_PARSERS(ulonglong); + +#undef DEFINE_INTEGER_PARSERS + +} // namespace pcrecpp diff --git a/pcrecpp.h.in b/pcrecpp.h.in new file mode 100644 index 0000000..c0d3050 --- /dev/null +++ b/pcrecpp.h.in @@ -0,0 +1,607 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: Sanjay Ghemawat + +#ifndef _PCRE_REGEXP_H +#define _PCRE_REGEXP_H + +// C++ interface to the pcre regular-expression library. RE supports +// Perl-style regular expressions (with extensions like \d, \w, \s, +// ...). +// +// ----------------------------------------------------------------------- +// REGEXP SYNTAX: +// +// This module is part of the pcre library and hence supports its syntax +// for regular expressions. +// +// The syntax is pretty similar to Perl's. For those not familiar +// with Perl's regular expressions, here are some examples of the most +// commonly used extensions: +// +// "hello (\\w+) world" -- \w matches a "word" character +// "version (\\d+)" -- \d matches a digit +// "hello\\s+world" -- \s matches any whitespace character +// "\\b(\\w+)\\b" -- \b matches empty string at a word boundary +// "(?i)hello" -- (?i) turns on case-insensitive matching +// "/\\*(.*?)\\*/" -- .*? matches . minimum no. of times possible +// +// ----------------------------------------------------------------------- +// MATCHING INTERFACE: +// +// The "FullMatch" operation checks that supplied text matches a +// supplied pattern exactly. +// +// Example: successful match +// pcrecpp::RE re("h.*o"); +// re.FullMatch("hello"); +// +// Example: unsuccessful match (requires full match): +// pcrecpp::RE re("e"); +// !re.FullMatch("hello"); +// +// Example: creating a temporary RE object: +// pcrecpp::RE("h.*o").FullMatch("hello"); +// +// You can pass in a "const char*" or a "string" for "text". The +// examples below tend to use a const char*. +// +// You can, as in the different examples above, store the RE object +// explicitly in a variable or use a temporary RE object. The +// examples below use one mode or the other arbitrarily. Either +// could correctly be used for any of these examples. +// +// ----------------------------------------------------------------------- +// MATCHING WITH SUB-STRING EXTRACTION: +// +// You can supply extra pointer arguments to extract matched subpieces. +// +// Example: extracts "ruby" into "s" and 1234 into "i" +// int i; +// string s; +// pcrecpp::RE re("(\\w+):(\\d+)"); +// re.FullMatch("ruby:1234", &s, &i); +// +// Example: does not try to extract any extra sub-patterns +// re.FullMatch("ruby:1234", &s); +// +// Example: does not try to extract into NULL +// re.FullMatch("ruby:1234", NULL, &i); +// +// Example: integer overflow causes failure +// !re.FullMatch("ruby:1234567891234", NULL, &i); +// +// Example: fails because there aren't enough sub-patterns: +// !pcrecpp::RE("\\w+:\\d+").FullMatch("ruby:1234", &s); +// +// Example: fails because string cannot be stored in integer +// !pcrecpp::RE("(.*)").FullMatch("ruby", &i); +// +// The provided pointer arguments can be pointers to any scalar numeric +// type, or one of +// string (matched piece is copied to string) +// StringPiece (StringPiece is mutated to point to matched piece) +// T (where "bool T::ParseFrom(const char*, int)" exists) +// NULL (the corresponding matched sub-pattern is not copied) +// +// ----------------------------------------------------------------------- +// DO_MATCH +// +// The matching interface supports at most 16 arguments per call. +// If you need more, consider using the more general interface +// pcrecpp::RE::DoMatch(). See pcrecpp.h for the signature for DoMatch. +// +// ----------------------------------------------------------------------- +// PARTIAL MATCHES +// +// You can use the "PartialMatch" operation when you want the pattern +// to match any substring of the text. +// +// Example: simple search for a string: +// pcrecpp::RE("ell").PartialMatch("hello"); +// +// Example: find first number in a string: +// int number; +// pcrecpp::RE re("(\\d+)"); +// re.PartialMatch("x*100 + 20", &number); +// assert(number == 100); +// +// ----------------------------------------------------------------------- +// UTF-8 AND THE MATCHING INTERFACE: +// +// By default, pattern and text are plain text, one byte per character. +// The UTF8 flag, passed to the constructor, causes both pattern +// and string to be treated as UTF-8 text, still a byte stream but +// potentially multiple bytes per character. In practice, the text +// is likelier to be UTF-8 than the pattern, but the match returned +// may depend on the UTF8 flag, so always use it when matching +// UTF8 text. E.g., "." will match one byte normally but with UTF8 +// set may match up to three bytes of a multi-byte character. +// +// Example: +// pcrecpp::RE_Options options; +// options.set_utf8(); +// pcrecpp::RE re(utf8_pattern, options); +// re.FullMatch(utf8_string); +// +// Example: using the convenience function UTF8(): +// pcrecpp::RE re(utf8_pattern, pcrecpp::UTF8()); +// re.FullMatch(utf8_string); +// +// NOTE: The UTF8 option is ignored if pcre was not configured with the +// --enable-utf8 flag. +// +// ----------------------------------------------------------------------- +// SCANNING TEXT INCREMENTALLY +// +// The "Consume" operation may be useful if you want to repeatedly +// match regular expressions at the front of a string and skip over +// them as they match. This requires use of the "StringPiece" type, +// which represents a sub-range of a real string. Like RE, StringPiece +// is defined in the pcrecpp namespace. +// +// Example: read lines of the form "var = value" from a string. +// string contents = ...; // Fill string somehow +// pcrecpp::StringPiece input(contents); // Wrap in a StringPiece +// +// string var; +// int value; +// pcrecpp::RE re("(\\w+) = (\\d+)\n"); +// while (re.Consume(&input, &var, &value)) { +// ...; +// } +// +// Each successful call to "Consume" will set "var/value", and also +// advance "input" so it points past the matched text. +// +// The "FindAndConsume" operation is similar to "Consume" but does not +// anchor your match at the beginning of the string. For example, you +// could extract all words from a string by repeatedly calling +// pcrecpp::RE("(\\w+)").FindAndConsume(&input, &word) +// +// ----------------------------------------------------------------------- +// PARSING HEX/OCTAL/C-RADIX NUMBERS +// +// By default, if you pass a pointer to a numeric value, the +// corresponding text is interpreted as a base-10 number. You can +// instead wrap the pointer with a call to one of the operators Hex(), +// Octal(), or CRadix() to interpret the text in another base. The +// CRadix operator interprets C-style "0" (base-8) and "0x" (base-16) +// prefixes, but defaults to base-10. +// +// Example: +// int a, b, c, d; +// pcrecpp::RE re("(.*) (.*) (.*) (.*)"); +// re.FullMatch("100 40 0100 0x40", +// pcrecpp::Octal(&a), pcrecpp::Hex(&b), +// pcrecpp::CRadix(&c), pcrecpp::CRadix(&d)); +// will leave 64 in a, b, c, and d. +// +// ----------------------------------------------------------------------- +// REPLACING PARTS OF STRINGS +// +// You can replace the first match of "pattern" in "str" with +// "rewrite". Within "rewrite", backslash-escaped digits (\1 to \9) +// can be used to insert text matching corresponding parenthesized +// group from the pattern. \0 in "rewrite" refers to the entire +// matching text. E.g., +// +// string s = "yabba dabba doo"; +// pcrecpp::RE("b+").Replace("d", &s); +// +// will leave "s" containing "yada dabba doo". The result is true if +// the pattern matches and a replacement occurs, or false otherwise. +// +// GlobalReplace() is like Replace(), except that it replaces all +// occurrences of the pattern in the string with the rewrite. +// Replacements are not subject to re-matching. E.g., +// +// string s = "yabba dabba doo"; +// pcrecpp::RE("b+").GlobalReplace("d", &s); +// +// will leave "s" containing "yada dada doo". It returns the number +// of replacements made. +// +// Extract() is like Replace(), except that if the pattern matches, +// "rewrite" is copied into "out" (an additional argument) with +// substitutions. The non-matching portions of "text" are ignored. +// Returns true iff a match occurred and the extraction happened +// successfully. If no match occurs, the string is left unaffected. + + +#include <string> +// These aren't technically needed here, but we include them +// anyway so folks who include pcrecpp.h don't have to include +// all these other header files as well. +#include <pcre.h> +#include <pcre_stringpiece.h> + +namespace pcrecpp { + +// We convert user-passed pointers into special Arg objects +class Arg; +extern Arg no_arg; + +/***** Compiling regular expressions: the RE class *****/ + +// RE_Options allow you to set options to be passed along to pcre, +// along with other options we put on top of pcre. Only UTF and +// match_limit are supported now. Setting match_limit +// to a non-zero value will limit the executation of pcre to +// keep it from doing bad things like blowing the stack or taking +// an eternity to return a result. A value of 5000 is good enough +// to stop stack blowup in a 2MB thread stack. +// Setting match_limit to zero will disable match limiting. +class RE_Options { + public: + // constructor + RE_Options() : match_limit_(0), utf8_(false) {} + // we're fine with the default destructor, copy constructor, etc. + + // accessors and mutators + int match_limit() const { return match_limit_; }; + void set_match_limit(int limit) { + match_limit_ = limit; + } + + bool utf8() const { return utf8_; } + void set_utf8(bool u) { + utf8_ = u; + } + + // TODO: add other pcre flags + + private: + int match_limit_; + bool utf8_; +}; + +// These functions return some common RE_Options +static inline RE_Options UTF8() { + RE_Options options; + options.set_utf8(true); + return options; +} + + +// Interface for regular expression matching. Also corresponds to a +// pre-compiled regular expression. An "RE" object is safe for +// concurrent use by multiple threads. +class RE { + public: + // We provide implicit conversions from strings so that users can + // pass in a string or a "const char*" wherever an "RE" is expected. + RE(const char* pat) { Init(pat, NULL); } + RE(const char *pat, const RE_Options& option) { Init(pat, &option); } + RE(const string& pat) { Init(pat.c_str(), NULL); } + RE(const string& pat, const RE_Options& option) { Init(pat.c_str(), &option); } + + ~RE(); + + // The string specification for this RE. E.g. + // RE re("ab*c?d+"); + // re.pattern(); // "ab*c?d+" + const string& pattern() const { return pattern_; } + + // If RE could not be created properly, returns an error string. + // Else returns the empty string. + const string& error() const { return *error_; } + + /***** The useful part: the matching interface *****/ + + // This is provided so one can do pattern.ReplaceAll() just as + // easily as ReplaceAll(pattern-text, ....) + + bool FullMatch(const StringPiece& text, + const Arg& ptr1 = no_arg, + const Arg& ptr2 = no_arg, + const Arg& ptr3 = no_arg, + const Arg& ptr4 = no_arg, + const Arg& ptr5 = no_arg, + const Arg& ptr6 = no_arg, + const Arg& ptr7 = no_arg, + const Arg& ptr8 = no_arg, + const Arg& ptr9 = no_arg, + const Arg& ptr10 = no_arg, + const Arg& ptr11 = no_arg, + const Arg& ptr12 = no_arg, + const Arg& ptr13 = no_arg, + const Arg& ptr14 = no_arg, + const Arg& ptr15 = no_arg, + const Arg& ptr16 = no_arg) const; + + bool PartialMatch(const StringPiece& text, + const Arg& ptr1 = no_arg, + const Arg& ptr2 = no_arg, + const Arg& ptr3 = no_arg, + const Arg& ptr4 = no_arg, + const Arg& ptr5 = no_arg, + const Arg& ptr6 = no_arg, + const Arg& ptr7 = no_arg, + const Arg& ptr8 = no_arg, + const Arg& ptr9 = no_arg, + const Arg& ptr10 = no_arg, + const Arg& ptr11 = no_arg, + const Arg& ptr12 = no_arg, + const Arg& ptr13 = no_arg, + const Arg& ptr14 = no_arg, + const Arg& ptr15 = no_arg, + const Arg& ptr16 = no_arg) const; + + bool Consume(StringPiece* input, + const Arg& ptr1 = no_arg, + const Arg& ptr2 = no_arg, + const Arg& ptr3 = no_arg, + const Arg& ptr4 = no_arg, + const Arg& ptr5 = no_arg, + const Arg& ptr6 = no_arg, + const Arg& ptr7 = no_arg, + const Arg& ptr8 = no_arg, + const Arg& ptr9 = no_arg, + const Arg& ptr10 = no_arg, + const Arg& ptr11 = no_arg, + const Arg& ptr12 = no_arg, + const Arg& ptr13 = no_arg, + const Arg& ptr14 = no_arg, + const Arg& ptr15 = no_arg, + const Arg& ptr16 = no_arg) const; + + bool FindAndConsume(StringPiece* input, + const Arg& ptr1 = no_arg, + const Arg& ptr2 = no_arg, + const Arg& ptr3 = no_arg, + const Arg& ptr4 = no_arg, + const Arg& ptr5 = no_arg, + const Arg& ptr6 = no_arg, + const Arg& ptr7 = no_arg, + const Arg& ptr8 = no_arg, + const Arg& ptr9 = no_arg, + const Arg& ptr10 = no_arg, + const Arg& ptr11 = no_arg, + const Arg& ptr12 = no_arg, + const Arg& ptr13 = no_arg, + const Arg& ptr14 = no_arg, + const Arg& ptr15 = no_arg, + const Arg& ptr16 = no_arg) const; + + bool Replace(const StringPiece& rewrite, + string *str) const; + + int GlobalReplace(const StringPiece& rewrite, + string *str) const; + + bool Extract(const StringPiece &rewrite, + const StringPiece &text, + string *out) const; + + /***** Generic matching interface *****/ + + // Type of match (TODO: Should be restructured as part of RE_Options) + enum Anchor { + UNANCHORED, // No anchoring + ANCHOR_START, // Anchor at start only + ANCHOR_BOTH // Anchor at start and end + }; + + // General matching routine. Stores the length of the match in + // "*consumed" if successful. + bool DoMatch(const StringPiece& text, + Anchor anchor, + int* consumed, + const Arg* const* args, int n) const; + + // Return the number of capturing subpatterns, or -1 if the + // regexp wasn't valid on construction. + int NumberOfCapturingGroups(); + + private: + + void Init(const char* pattern, const RE_Options* options); + + // Match against "text", filling in "vec" (up to "vecsize" * 2/3) with + // pairs of integers for the beginning and end positions of matched + // text. The first pair corresponds to the entire matched text; + // subsequent pairs correspond, in order, to parentheses-captured + // matches. Returns the number of pairs (one more than the number of + // the last subpattern with a match) if matching was successful + // and zero if the match failed. + // I.e. for RE("(foo)|(bar)|(baz)") it will return 2, 3, and 4 when matching + // against "foo", "bar", and "baz" respectively. + // When matching RE("(foo)|hello") against "hello", it will return 1. + // But the values for all subpattern are filled in into "vec". + int TryMatch(const StringPiece& text, + int startpos, + Anchor anchor, + int *vec, + int vecsize) const; + + // Append the "rewrite" string, with backslash subsitutions from "text" + // and "vec", to string "out". + bool Rewrite(string *out, + const StringPiece& rewrite, + const StringPiece& text, + int *vec, + int veclen) const; + + // internal implementation for DoMatch + bool DoMatchImpl(const StringPiece& text, + Anchor anchor, + int* consumed, + const Arg* const args[], + int n, + int* vec, + int vecsize) const; + + // Compile the regexp for the specified anchoring mode + pcre* Compile(Anchor anchor); + + string pattern_; + RE_Options options_; + pcre* re_full_; // For full matches + pcre* re_partial_; // For partial matches + const string* error_; // Error indicator (or points to empty string) + int match_limit_; // limit on execution resources + + // Don't allow the default copy or assignment constructors -- + // they're expensive and too easy to do by accident. + RE(const RE&); + void operator=(const RE&); +}; + + +/***** Implementation details *****/ + +// Hex/Octal/Binary? + +// Special class for parsing into objects that define a ParseFrom() method +template <class T> +class _RE_MatchObject { + public: + static inline bool Parse(const char* str, int n, void* dest) { + T* object = reinterpret_cast<T*>(dest); + return object->ParseFrom(str, n); + } +}; + +class Arg { + public: + // Empty constructor so we can declare arrays of Arg + Arg(); + + // Constructor specially designed for NULL arguments + Arg(void*); + + typedef bool (*Parser)(const char* str, int n, void* dest); + +// Type-specific parsers +#define PCRE_MAKE_PARSER(type,name) \ + Arg(type* p) : arg_(p), parser_(name) { } \ + Arg(type* p, Parser parser) : arg_(p), parser_(parser) { } + + + PCRE_MAKE_PARSER(char, parse_char); + PCRE_MAKE_PARSER(unsigned char, parse_uchar); + PCRE_MAKE_PARSER(short, parse_short); + PCRE_MAKE_PARSER(unsigned short, parse_ushort); + PCRE_MAKE_PARSER(int, parse_int); + PCRE_MAKE_PARSER(unsigned int, parse_uint); + PCRE_MAKE_PARSER(long, parse_long); + PCRE_MAKE_PARSER(unsigned long, parse_ulong); +#if @pcre_has_long_long@ + PCRE_MAKE_PARSER(long long, parse_longlong); +#endif +#if @pcre_has_ulong_long@ + PCRE_MAKE_PARSER(unsigned long long, parse_ulonglong); +#endif + PCRE_MAKE_PARSER(float, parse_float); + PCRE_MAKE_PARSER(double, parse_double); + PCRE_MAKE_PARSER(string, parse_string); + PCRE_MAKE_PARSER(StringPiece, parse_stringpiece); + +#undef PCRE_MAKE_PARSER + + // Generic constructor + template <class T> Arg(T*, Parser parser); + // Generic constructor template + template <class T> Arg(T* p) + : arg_(p), parser_(_RE_MatchObject<T>::Parse) { + } + + // Parse the data + bool Parse(const char* str, int n) const; + + private: + void* arg_; + Parser parser_; + + static bool parse_null (const char* str, int n, void* dest); + static bool parse_char (const char* str, int n, void* dest); + static bool parse_uchar (const char* str, int n, void* dest); + static bool parse_float (const char* str, int n, void* dest); + static bool parse_double (const char* str, int n, void* dest); + static bool parse_string (const char* str, int n, void* dest); + static bool parse_stringpiece (const char* str, int n, void* dest); + +#define PCRE_DECLARE_INTEGER_PARSER(name) \ + private: \ + static bool parse_ ## name(const char* str, int n, void* dest); \ + static bool parse_ ## name ## _radix( \ + const char* str, int n, void* dest, int radix); \ + public: \ + static bool parse_ ## name ## _hex(const char* str, int n, void* dest); \ + static bool parse_ ## name ## _octal(const char* str, int n, void* dest); \ + static bool parse_ ## name ## _cradix(const char* str, int n, void* dest) + + PCRE_DECLARE_INTEGER_PARSER(short); + PCRE_DECLARE_INTEGER_PARSER(ushort); + PCRE_DECLARE_INTEGER_PARSER(int); + PCRE_DECLARE_INTEGER_PARSER(uint); + PCRE_DECLARE_INTEGER_PARSER(long); + PCRE_DECLARE_INTEGER_PARSER(ulong); + PCRE_DECLARE_INTEGER_PARSER(longlong); + PCRE_DECLARE_INTEGER_PARSER(ulonglong); + +#undef PCRE_DECLARE_INTEGER_PARSER +}; + +inline Arg::Arg() : arg_(NULL), parser_(parse_null) { } +inline Arg::Arg(void* p) : arg_(p), parser_(parse_null) { } + +inline bool Arg::Parse(const char* str, int n) const { + return (*parser_)(str, n, arg_); +} + +// This part of the parser, appropriate only for ints, deals with bases +#define MAKE_INTEGER_PARSER(type, name) \ + inline Arg Hex(type* ptr) { \ + return Arg(ptr, Arg::parse_ ## name ## _hex); } \ + inline Arg Octal(type* ptr) { \ + return Arg(ptr, Arg::parse_ ## name ## _octal); } \ + inline Arg CRadix(type* ptr) { \ + return Arg(ptr, Arg::parse_ ## name ## _cradix); } + +MAKE_INTEGER_PARSER(short, short); +MAKE_INTEGER_PARSER(unsigned short, ushort); +MAKE_INTEGER_PARSER(int, int); +MAKE_INTEGER_PARSER(unsigned int, uint); +MAKE_INTEGER_PARSER(long, long); +MAKE_INTEGER_PARSER(unsigned long, ulong); +#if @pcre_has_long_long@ +MAKE_INTEGER_PARSER(long long, longlong); +#endif +#if @pcre_has_ulong_long@ +MAKE_INTEGER_PARSER(unsigned long long, ulonglong); +#endif + +#undef MAKE_INTEGER_PARSER + +} // namespace pcrecpp + +#endif /* _PCRE_REGEXP_H */ diff --git a/pcrecpp_unittest.cc b/pcrecpp_unittest.cc new file mode 100644 index 0000000..000c12e --- /dev/null +++ b/pcrecpp_unittest.cc @@ -0,0 +1,814 @@ +// Copyright (c) 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: Sanjay Ghemawat +// +// TODO: Test extractions for PartialMatch/Consume + +#include <stdio.h> +#include <vector> +#include "config.h" +#include "pcrecpp.h" + +using pcrecpp::StringPiece; +using pcrecpp::RE; +using pcrecpp::RE_Options; +using pcrecpp::Hex; +using pcrecpp::Octal; +using pcrecpp::CRadix; + +// CHECK dies with a fatal error if condition is not true. It is *not* +// controlled by NDEBUG, so the check will be executed regardless of +// compilation mode. Therefore, it is safe to do things like: +// CHECK_EQ(fp->Write(x), 4) +#define CHECK(condition) do { \ + if (!(condition)) { \ + fprintf(stderr, "%s:%d: Check failed: %s\n", \ + __FILE__, __LINE__, #condition); \ + exit(1); \ + } \ +} while (0) + +#define CHECK_EQ(a, b) CHECK(a == b) + +static void Timing1(int num_iters) { + // Same pattern lots of times + RE pattern("ruby:\\d+"); + StringPiece p("ruby:1234"); + for (int j = num_iters; j > 0; j--) { + CHECK(pattern.FullMatch(p)); + } +} + +static void Timing2(int num_iters) { + // Same pattern lots of times + RE pattern("ruby:(\\d+)"); + int i; + for (int j = num_iters; j > 0; j--) { + CHECK(pattern.FullMatch("ruby:1234", &i)); + CHECK_EQ(i, 1234); + } +} + +static void Timing3(int num_iters) { + string text_string; + for (int j = num_iters; j > 0; j--) { + text_string += "this is another line\n"; + } + + RE line_matcher(".*\n"); + string line; + StringPiece text(text_string); + int counter = 0; + while (line_matcher.Consume(&text)) { + counter++; + } + printf("Matched %d lines\n", counter); +} + +#if 0 // uncomment this if you have a way of defining VirtualProcessSize() + +static void LeakTest() { + // Check for memory leaks + unsigned long long initial_size = 0; + for (int i = 0; i < 100000; i++) { + if (i == 50000) { + initial_size = VirtualProcessSize(); + printf("Size after 50000: %llu\n", initial_size); + } + char buf[100]; + snprintf(buf, sizeof(buf), "pat%09d", i); + RE newre(buf); + } + uint64 final_size = VirtualProcessSize(); + printf("Size after 100000: %llu\n", final_size); + const double growth = double(final_size - initial_size) / final_size; + printf("Growth: %0.2f%%", growth * 100); + CHECK(growth < 0.02); // Allow < 2% growth +} + +#endif + +static void RadixTests() { + printf("Testing hex\n"); + +#define CHECK_HEX(type, value) \ + do { \ + type v; \ + CHECK(RE("([0-9a-fA-F]+)[uUlL]*").FullMatch(#value, Hex(&v))); \ + CHECK_EQ(v, 0x ## value); \ + CHECK(RE("([0-9a-fA-FxX]+)[uUlL]*").FullMatch("0x" #value, CRadix(&v))); \ + CHECK_EQ(v, 0x ## value); \ + } while(0) + + CHECK_HEX(short, 2bad); + CHECK_HEX(unsigned short, 2badU); + CHECK_HEX(int, dead); + CHECK_HEX(unsigned int, deadU); + CHECK_HEX(long, 7eadbeefL); + CHECK_HEX(unsigned long, deadbeefUL); +#ifdef HAVE_LONG_LONG + CHECK_HEX(long long, 12345678deadbeefLL); +#endif +#ifdef HAVE_UNSIGNED_LONG_LONG + CHECK_HEX(unsigned long long, cafebabedeadbeefULL); +#endif + +#undef CHECK_HEX + + printf("Testing octal\n"); + +#define CHECK_OCTAL(type, value) \ + do { \ + type v; \ + CHECK(RE("([0-7]+)[uUlL]*").FullMatch(#value, Octal(&v))); \ + CHECK_EQ(v, 0 ## value); \ + CHECK(RE("([0-9a-fA-FxX]+)[uUlL]*").FullMatch("0" #value, CRadix(&v))); \ + CHECK_EQ(v, 0 ## value); \ + } while(0) + + CHECK_OCTAL(short, 77777); + CHECK_OCTAL(unsigned short, 177777U); + CHECK_OCTAL(int, 17777777777); + CHECK_OCTAL(unsigned int, 37777777777U); + CHECK_OCTAL(long, 17777777777L); + CHECK_OCTAL(unsigned long, 37777777777UL); +#ifdef HAVE_LONG_LONG + CHECK_OCTAL(long long, 777777777777777777777LL); +#endif +#ifdef HAVE_UNSIGNED_LONG_LONG + CHECK_OCTAL(unsigned long long, 1777777777777777777777ULL); +#endif + +#undef CHECK_OCTAL + + printf("Testing decimal\n"); + +#define CHECK_DECIMAL(type, value) \ + do { \ + type v; \ + CHECK(RE("(-?[0-9]+)[uUlL]*").FullMatch(#value, &v)); \ + CHECK_EQ(v, value); \ + CHECK(RE("(-?[0-9a-fA-FxX]+)[uUlL]*").FullMatch(#value, CRadix(&v))); \ + CHECK_EQ(v, value); \ + } while(0) + + CHECK_DECIMAL(short, -1); + CHECK_DECIMAL(unsigned short, 9999); + CHECK_DECIMAL(int, -1000); + CHECK_DECIMAL(unsigned int, 12345U); + CHECK_DECIMAL(long, -10000000L); + CHECK_DECIMAL(unsigned long, 3083324652U); +#ifdef HAVE_LONG_LONG + CHECK_DECIMAL(long long, -100000000000000LL); +#endif +#ifdef HAVE_UNSIGNED_LONG_LONG + CHECK_DECIMAL(unsigned long long, 1234567890987654321ULL); +#endif + +#undef CHECK_DECIMAL + +} + +static void TestReplace() { + printf("Testing Replace\n"); + + struct ReplaceTest { + const char *regexp; + const char *rewrite; + const char *original; + const char *single; + const char *global; + }; + static const ReplaceTest tests[] = { + { "(qu|[b-df-hj-np-tv-z]*)([a-z]+)", + "\\2\\1ay", + "the quick brown fox jumps over the lazy dogs.", + "ethay quick brown fox jumps over the lazy dogs.", + "ethay ickquay ownbray oxfay umpsjay overay ethay azylay ogsday." }, + { "\\w+", + "\\0-NOSPAM", + "paul.haahr@google.com", + "paul-NOSPAM.haahr@google.com", + "paul-NOSPAM.haahr-NOSPAM@google-NOSPAM.com-NOSPAM" }, + { "^", + "(START)", + "foo", + "(START)foo", + "(START)foo" }, + { "^", + "(START)", + "", + "(START)", + "(START)" }, + { "$", + "(END)", + "", + "(END)", + "(END)" }, + { "b", + "bb", + "ababababab", + "abbabababab", + "abbabbabbabbabb" }, + { "b", + "bb", + "bbbbbb", + "bbbbbbb", + "bbbbbbbbbbbb" }, + { "b+", + "bb", + "bbbbbb", + "bb", + "bb" }, + { "b*", + "bb", + "bbbbbb", + "bb", + "bb" }, + { "b*", + "bb", + "aaaaa", + "bbaaaaa", + "bbabbabbabbabbabb" }, + { "", NULL, NULL, NULL, NULL } + }; + + for (const ReplaceTest *t = tests; t->original != NULL; ++t) { + string one(t->original); + CHECK(RE(t->regexp).Replace(t->rewrite, &one)); + CHECK_EQ(one, t->single); + string all(t->original); + CHECK(RE(t->regexp).GlobalReplace(t->rewrite, &all) > 0); + CHECK_EQ(all, t->global); + } +} + +static void TestExtract() { + printf("Testing Extract\n"); + + string s; + + CHECK(RE("(.*)@([^.]*)").Extract("\\2!\\1", "boris@kremvax.ru", &s)); + CHECK_EQ(s, "kremvax!boris"); + + // check the RE interface as well + CHECK(RE(".*").Extract("'\\0'", "foo", &s)); + CHECK_EQ(s, "'foo'"); + CHECK(!RE("bar").Extract("'\\0'", "baz", &s)); + CHECK_EQ(s, "'foo'"); +} + +static void TestConsume() { + printf("Testing Consume\n"); + + string word; + + string s(" aaa b!@#$@#$cccc"); + StringPiece input(s); + + RE r("\\s*(\\w+)"); // matches a word, possibly proceeded by whitespace + CHECK(r.Consume(&input, &word)); + CHECK_EQ(word, "aaa"); + CHECK(r.Consume(&input, &word)); + CHECK_EQ(word, "b"); + CHECK(! r.Consume(&input, &word)); +} + +static void TestFindAndConsume() { + printf("Testing FindAndConsume\n"); + + string word; + + string s(" aaa b!@#$@#$cccc"); + StringPiece input(s); + + RE r("(\\w+)"); // matches a word + CHECK(r.FindAndConsume(&input, &word)); + CHECK_EQ(word, "aaa"); + CHECK(r.FindAndConsume(&input, &word)); + CHECK_EQ(word, "b"); + CHECK(r.FindAndConsume(&input, &word)); + CHECK_EQ(word, "cccc"); + CHECK(! r.FindAndConsume(&input, &word)); +} + +static void TestMatchNumberPeculiarity() { + printf("Testing match-number peculiaraity\n"); + + string word1; + string word2; + string word3; + + RE r("(foo)|(bar)|(baz)"); + CHECK(r.PartialMatch("foo", &word1, &word2, &word3)); + CHECK_EQ(word1, "foo"); + CHECK_EQ(word2, ""); + CHECK_EQ(word3, ""); + CHECK(r.PartialMatch("bar", &word1, &word2, &word3)); + CHECK_EQ(word1, ""); + CHECK_EQ(word2, "bar"); + CHECK_EQ(word3, ""); + CHECK(r.PartialMatch("baz", &word1, &word2, &word3)); + CHECK_EQ(word1, ""); + CHECK_EQ(word2, ""); + CHECK_EQ(word3, "baz"); + CHECK(!r.PartialMatch("f", &word1, &word2, &word3)); + + string a; + CHECK(RE("(foo)|hello").FullMatch("hello", &a)); + CHECK_EQ(a, ""); +} + +static void TestRecursion(int size, const char *pattern, int match_limit) { + printf("Testing recursion\n"); + + // Fill up a string repeating the pattern given + string domain; + domain.resize(size); + int patlen = strlen(pattern); + for (int i = 0; i < size; ++i) { + domain[i] = pattern[i % patlen]; + } + // Just make sure it doesn't crash due to too much recursion. + RE_Options options; + options.set_match_limit(match_limit); + RE re("([a-zA-Z0-9]|-)+(\\.([a-zA-Z0-9]|-)+)*(\\.)?", options); + re.FullMatch(domain); +} + + +int main(int argc, char** argv) { + // Treat any flag as --help + if (argc > 1 && argv[1][0] == '-') { + printf("Usage: %s [timing1|timing2|timing3 num-iters]\n" + " If 'timingX ###' is specified, run the given timing test\n" + " with the given number of iterations, rather than running\n" + " the default corectness test.\n", argv[0]); + return 0; + } + + if (argc > 1) { + if ( argc == 2 || atoi(argv[2]) == 0) { + printf("timing mode needs a num-iters argument\n"); + return 1; + } + if (!strcmp(argv[1], "timing1")) + Timing1(atoi(argv[2])); + else if (!strcmp(argv[1], "timing2")) + Timing2(atoi(argv[2])); + else if (!strcmp(argv[1], "timing3")) + Timing3(atoi(argv[2])); + else + printf("Unknown argument '%s'\n", argv[1]); + return 0; + } + + printf("Testing FullMatch\n"); + + int i; + string s; + + /***** FullMatch with no args *****/ + + CHECK(RE("h.*o").FullMatch("hello")); + CHECK(!RE("h.*o").FullMatch("othello")); + CHECK(!RE("h.*o").FullMatch("hello!")); + + /***** FullMatch with args *****/ + + // Zero-arg + CHECK(RE("\\d+").FullMatch("1001")); + + // Single-arg + CHECK(RE("(\\d+)").FullMatch("1001", &i)); + CHECK_EQ(i, 1001); + CHECK(RE("(-?\\d+)").FullMatch("-123", &i)); + CHECK_EQ(i, -123); + CHECK(!RE("()\\d+").FullMatch("10", &i)); + CHECK(!RE("(\\d+)").FullMatch("1234567890123456789012345678901234567890", + &i)); + + // Digits surrounding integer-arg + CHECK(RE("1(\\d*)4").FullMatch("1234", &i)); + CHECK_EQ(i, 23); + CHECK(RE("(\\d)\\d+").FullMatch("1234", &i)); + CHECK_EQ(i, 1); + CHECK(RE("(-\\d)\\d+").FullMatch("-1234", &i)); + CHECK_EQ(i, -1); + CHECK(RE("(\\d)").PartialMatch("1234", &i)); + CHECK_EQ(i, 1); + CHECK(RE("(-\\d)").PartialMatch("-1234", &i)); + CHECK_EQ(i, -1); + + // String-arg + CHECK(RE("h(.*)o").FullMatch("hello", &s)); + CHECK_EQ(s, string("ell")); + + // StringPiece-arg + StringPiece sp; + CHECK(RE("(\\w+):(\\d+)").FullMatch("ruby:1234", &sp, &i)); + CHECK_EQ(sp.size(), 4); + CHECK(memcmp(sp.data(), "ruby", 4) == 0); + CHECK_EQ(i, 1234); + + // Multi-arg + CHECK(RE("(\\w+):(\\d+)").FullMatch("ruby:1234", &s, &i)); + CHECK_EQ(s, string("ruby")); + CHECK_EQ(i, 1234); + + // Ignored arg + CHECK(RE("(\\w+)(:)(\\d+)").FullMatch("ruby:1234", &s, (void*)NULL, &i)); + CHECK_EQ(s, string("ruby")); + CHECK_EQ(i, 1234); + + // Type tests + { + char c; + CHECK(RE("(H)ello").FullMatch("Hello", &c)); + CHECK_EQ(c, 'H'); + } + { + unsigned char c; + CHECK(RE("(H)ello").FullMatch("Hello", &c)); + CHECK_EQ(c, static_cast<unsigned char>('H')); + } + { + short v; + CHECK(RE("(-?\\d+)").FullMatch("100", &v)); CHECK_EQ(v, 100); + CHECK(RE("(-?\\d+)").FullMatch("-100", &v)); CHECK_EQ(v, -100); + CHECK(RE("(-?\\d+)").FullMatch("32767", &v)); CHECK_EQ(v, 32767); + CHECK(RE("(-?\\d+)").FullMatch("-32768", &v)); CHECK_EQ(v, -32768); + CHECK(!RE("(-?\\d+)").FullMatch("-32769", &v)); + CHECK(!RE("(-?\\d+)").FullMatch("32768", &v)); + } + { + unsigned short v; + CHECK(RE("(\\d+)").FullMatch("100", &v)); CHECK_EQ(v, 100); + CHECK(RE("(\\d+)").FullMatch("32767", &v)); CHECK_EQ(v, 32767); + CHECK(RE("(\\d+)").FullMatch("65535", &v)); CHECK_EQ(v, 65535); + CHECK(!RE("(\\d+)").FullMatch("65536", &v)); + } + { + int v; + static const int max_value = 0x7fffffff; + static const int min_value = -max_value - 1; + CHECK(RE("(-?\\d+)").FullMatch("100", &v)); CHECK_EQ(v, 100); + CHECK(RE("(-?\\d+)").FullMatch("-100", &v)); CHECK_EQ(v, -100); + CHECK(RE("(-?\\d+)").FullMatch("2147483647", &v)); CHECK_EQ(v, max_value); + CHECK(RE("(-?\\d+)").FullMatch("-2147483648", &v)); CHECK_EQ(v, min_value); + CHECK(!RE("(-?\\d+)").FullMatch("-2147483649", &v)); + CHECK(!RE("(-?\\d+)").FullMatch("2147483648", &v)); + } + { + unsigned int v; + static const unsigned int max_value = 0xfffffffful; + CHECK(RE("(\\d+)").FullMatch("100", &v)); CHECK_EQ(v, 100); + CHECK(RE("(\\d+)").FullMatch("4294967295", &v)); CHECK_EQ(v, max_value); + CHECK(!RE("(\\d+)").FullMatch("4294967296", &v)); + } +#ifdef HAVE_LONG_LONG + { + long long v; + static const long long max_value = 0x7fffffffffffffffLL; + static const long long min_value = -max_value - 1; + char buf[32]; + + CHECK(RE("(-?\\d+)").FullMatch("100", &v)); CHECK_EQ(v, 100); + CHECK(RE("(-?\\d+)").FullMatch("-100",&v)); CHECK_EQ(v, -100); + + snprintf(buf, sizeof(buf), "%lld", max_value); + CHECK(RE("(-?\\d+)").FullMatch(buf,&v)); CHECK_EQ(v, max_value); + + snprintf(buf, sizeof(buf), "%lld", min_value); + CHECK(RE("(-?\\d+)").FullMatch(buf,&v)); CHECK_EQ(v, min_value); + + snprintf(buf, sizeof(buf), "%lld", max_value); + assert(buf[strlen(buf)-1] != '9'); + buf[strlen(buf)-1]++; + CHECK(!RE("(-?\\d+)").FullMatch(buf, &v)); + + snprintf(buf, sizeof(buf), "%lld", min_value); + assert(buf[strlen(buf)-1] != '9'); + buf[strlen(buf)-1]++; + CHECK(!RE("(-?\\d+)").FullMatch(buf, &v)); + } +#endif +#if defined HAVE_UNSIGNED_LONG_LONG && defined HAVE_LONG_LONG + { + unsigned long long v; + long long v2; + static const unsigned long long max_value = 0xffffffffffffffffULL; + char buf[32]; + + CHECK(RE("(-?\\d+)").FullMatch("100",&v)); CHECK_EQ(v, 100); + CHECK(RE("(-?\\d+)").FullMatch("-100",&v2)); CHECK_EQ(v2, -100); + + snprintf(buf, sizeof(buf), "%llu", max_value); + CHECK(RE("(-?\\d+)").FullMatch(buf,&v)); CHECK_EQ(v, max_value); + + assert(buf[strlen(buf)-1] != '9'); + buf[strlen(buf)-1]++; + CHECK(!RE("(-?\\d+)").FullMatch(buf, &v)); + } +#endif + { + float v; + CHECK(RE("(.*)").FullMatch("100", &v)); + CHECK(RE("(.*)").FullMatch("-100.", &v)); + CHECK(RE("(.*)").FullMatch("1e23", &v)); + } + { + double v; + CHECK(RE("(.*)").FullMatch("100", &v)); + CHECK(RE("(.*)").FullMatch("-100.", &v)); + CHECK(RE("(.*)").FullMatch("1e23", &v)); + } + + // Check that matching is fully anchored + CHECK(!RE("(\\d+)").FullMatch("x1001", &i)); + CHECK(!RE("(\\d+)").FullMatch("1001x", &i)); + CHECK(RE("x(\\d+)").FullMatch("x1001", &i)); CHECK_EQ(i, 1001); + CHECK(RE("(\\d+)x").FullMatch("1001x", &i)); CHECK_EQ(i, 1001); + + // Braces + CHECK(RE("[0-9a-f+.-]{5,}").FullMatch("0abcd")); + CHECK(RE("[0-9a-f+.-]{5,}").FullMatch("0abcde")); + CHECK(!RE("[0-9a-f+.-]{5,}").FullMatch("0abc")); + + // Complicated RE + CHECK(RE("foo|bar|[A-Z]").FullMatch("foo")); + CHECK(RE("foo|bar|[A-Z]").FullMatch("bar")); + CHECK(RE("foo|bar|[A-Z]").FullMatch("X")); + CHECK(!RE("foo|bar|[A-Z]").FullMatch("XY")); + + // Check full-match handling (needs '$' tacked on internally) + CHECK(RE("fo|foo").FullMatch("fo")); + CHECK(RE("fo|foo").FullMatch("foo")); + CHECK(RE("fo|foo$").FullMatch("fo")); + CHECK(RE("fo|foo$").FullMatch("foo")); + CHECK(RE("foo$").FullMatch("foo")); + CHECK(!RE("foo\\$").FullMatch("foo$bar")); + CHECK(!RE("fo|bar").FullMatch("fox")); + + // Uncomment the following if we change the handling of '$' to + // prevent it from matching a trailing newline + if (false) { + // Check that we don't get bitten by pcre's special handling of a + // '\n' at the end of the string matching '$' + CHECK(!RE("foo$").PartialMatch("foo\n")); + } + + // Number of args + int a[16]; + CHECK(RE("").FullMatch("")); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d){1}").FullMatch("1", + &a[0])); + CHECK_EQ(a[0], 1); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d)(\\d)").FullMatch("12", + &a[0], &a[1])); + CHECK_EQ(a[0], 1); + CHECK_EQ(a[1], 2); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d)(\\d)(\\d)").FullMatch("123", + &a[0], &a[1], &a[2])); + CHECK_EQ(a[0], 1); + CHECK_EQ(a[1], 2); + CHECK_EQ(a[2], 3); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d)(\\d)(\\d)(\\d)").FullMatch("1234", + &a[0], &a[1], &a[2], &a[3])); + CHECK_EQ(a[0], 1); + CHECK_EQ(a[1], 2); + CHECK_EQ(a[2], 3); + CHECK_EQ(a[3], 4); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d)(\\d)(\\d)(\\d)(\\d)").FullMatch("12345", + &a[0], &a[1], &a[2], + &a[3], &a[4])); + CHECK_EQ(a[0], 1); + CHECK_EQ(a[1], 2); + CHECK_EQ(a[2], 3); + CHECK_EQ(a[3], 4); + CHECK_EQ(a[4], 5); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)").FullMatch("123456", + &a[0], &a[1], &a[2], + &a[3], &a[4], &a[5])); + CHECK_EQ(a[0], 1); + CHECK_EQ(a[1], 2); + CHECK_EQ(a[2], 3); + CHECK_EQ(a[3], 4); + CHECK_EQ(a[4], 5); + CHECK_EQ(a[5], 6); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)").FullMatch("1234567", + &a[0], &a[1], &a[2], &a[3], + &a[4], &a[5], &a[6])); + CHECK_EQ(a[0], 1); + CHECK_EQ(a[1], 2); + CHECK_EQ(a[2], 3); + CHECK_EQ(a[3], 4); + CHECK_EQ(a[4], 5); + CHECK_EQ(a[5], 6); + CHECK_EQ(a[6], 7); + + memset(a, 0, sizeof(0)); + CHECK(RE("(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)" + "(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)(\\d)").FullMatch( + "1234567890123456", + &a[0], &a[1], &a[2], &a[3], + &a[4], &a[5], &a[6], &a[7], + &a[8], &a[9], &a[10], &a[11], + &a[12], &a[13], &a[14], &a[15])); + CHECK_EQ(a[0], 1); + CHECK_EQ(a[1], 2); + CHECK_EQ(a[2], 3); + CHECK_EQ(a[3], 4); + CHECK_EQ(a[4], 5); + CHECK_EQ(a[5], 6); + CHECK_EQ(a[6], 7); + CHECK_EQ(a[7], 8); + CHECK_EQ(a[8], 9); + CHECK_EQ(a[9], 0); + CHECK_EQ(a[10], 1); + CHECK_EQ(a[11], 2); + CHECK_EQ(a[12], 3); + CHECK_EQ(a[13], 4); + CHECK_EQ(a[14], 5); + CHECK_EQ(a[15], 6); + + /***** PartialMatch *****/ + + printf("Testing PartialMatch\n"); + + CHECK(RE("h.*o").PartialMatch("hello")); + CHECK(RE("h.*o").PartialMatch("othello")); + CHECK(RE("h.*o").PartialMatch("hello!")); + CHECK(RE("((((((((((((((((((((x))))))))))))))))))))").PartialMatch("x")); + + RadixTests(); + TestReplace(); + TestExtract(); + TestConsume(); + TestFindAndConsume(); + TestMatchNumberPeculiarity(); + + // Check the pattern() accessor + { + const string kPattern = "http://([^/]+)/.*"; + const RE re(kPattern); + CHECK_EQ(kPattern, re.pattern()); + } + + // Check RE error field. + { + RE re("foo"); + CHECK(re.error().empty()); // Must have no error + } + +#ifdef SUPPORT_UTF8 + // Check UTF-8 handling + { + printf("Testing UTF-8 handling\n"); + + // Three Japanese characters (nihongo) + const char utf8_string[] = { + 0xe6, 0x97, 0xa5, // 65e5 + 0xe6, 0x9c, 0xac, // 627c + 0xe8, 0xaa, 0x9e, // 8a9e + 0 + }; + const char utf8_pattern[] = { + '.', + 0xe6, 0x9c, 0xac, // 627c + '.', + 0 + }; + + // Both should match in either mode, bytes or UTF-8 + RE re_test1("........."); + CHECK(re_test1.FullMatch(utf8_string)); + RE re_test2("...", pcrecpp::UTF8()); + CHECK(re_test2.FullMatch(utf8_string)); + + // Check that '.' matches one byte or UTF-8 character + // according to the mode. + string ss; + RE re_test3("(.)"); + CHECK(re_test3.PartialMatch(utf8_string, &ss)); + CHECK_EQ(ss, string("\xe6")); + RE re_test4("(.)", pcrecpp::UTF8()); + CHECK(re_test4.PartialMatch(utf8_string, &ss)); + CHECK_EQ(ss, string("\xe6\x97\xa5")); + + // Check that string matches itself in either mode + RE re_test5(utf8_string); + CHECK(re_test5.FullMatch(utf8_string)); + RE re_test6(utf8_string, pcrecpp::UTF8()); + CHECK(re_test6.FullMatch(utf8_string)); + + // Check that pattern matches string only in UTF8 mode + RE re_test7(utf8_pattern); + CHECK(!re_test7.FullMatch(utf8_string)); + RE re_test8(utf8_pattern, pcrecpp::UTF8()); + CHECK(re_test8.FullMatch(utf8_string)); + } + + // Check that ungreedy, UTF8 regular expressions don't match when they + // oughtn't -- see bug 82246. + { + // This code always worked. + const char* pattern = "\\w+X"; + const string target = "a aX"; + RE match_sentence(pattern); + RE match_sentence_re(pattern, pcrecpp::UTF8()); + + CHECK(!match_sentence.FullMatch(target)); + CHECK(!match_sentence_re.FullMatch(target)); + } + + { + const char* pattern = "(?U)\\w+X"; + const string target = "a aX"; + RE match_sentence(pattern); + RE match_sentence_re(pattern, pcrecpp::UTF8()); + + CHECK(!match_sentence.FullMatch(target)); + CHECK(!match_sentence_re.FullMatch(target)); + } +#endif /* def SUPPORT_UTF8 */ + + printf("Testing error reporting\n"); + + { RE re("a\\1"); CHECK(!re.error().empty()); } + { + RE re("a[x"); + CHECK(!re.error().empty()); + } + { + RE re("a[z-a]"); + CHECK(!re.error().empty()); + } + { + RE re("a[[:foobar:]]"); + CHECK(!re.error().empty()); + } + { + RE re("a(b"); + CHECK(!re.error().empty()); + } + { + RE re("a\\"); + CHECK(!re.error().empty()); + } + + // Test that recursion is stopped: there will be some errors reported + int matchlimit = 5000; + int bytes = 15 * 1024; // enough to crash if there was no match limit + TestRecursion(bytes, ".", matchlimit); + TestRecursion(bytes, "a", matchlimit); + TestRecursion(bytes, "a.", matchlimit); + TestRecursion(bytes, "ab.", matchlimit); + TestRecursion(bytes, "abc.", matchlimit); + + // Done + printf("OK\n"); + + return 0; +} @@ -6,7 +6,7 @@ its pattern matching. On a Unix or Win32 system it can recurse into directories. - Copyright (c) 1997-2004 University of Cambridge + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -42,6 +42,11 @@ POSSIBILITY OF SUCH DAMAGE. #include <string.h> #include <stdlib.h> #include <errno.h> + +#include <sys/types.h> +#include <sys/stat.h> +#include <unistd.h> + #include "config.h" #include "pcre.h" @@ -50,51 +55,89 @@ POSSIBILITY OF SUCH DAMAGE. typedef int BOOL; -#define VERSION "3.0 14-Jan-2003" +#define VERSION "4.0 07-Jun-2005" #define MAX_PATTERN_COUNT 100 +#if BUFSIZ > 8192 +#define MBUFTHIRD BUFSIZ +#else +#define MBUFTHIRD 8192 +#endif + + /************************************************* * Global variables * *************************************************/ static char *pattern_filename = NULL; +static char *stdin_name = (char *)"(standard input)"; static int pattern_count = 0; static pcre **pattern_list; static pcre_extra **hints_list; +static char *include_pattern = NULL; +static char *exclude_pattern = NULL; + +static pcre *include_compiled = NULL; +static pcre *exclude_compiled = NULL; + +static int after_context = 0; +static int before_context = 0; +static int both_context = 0; + static BOOL count_only = FALSE; static BOOL filenames = TRUE; static BOOL filenames_only = FALSE; +static BOOL filenames_nomatch_only = FALSE; +static BOOL hyphenpending = FALSE; static BOOL invert = FALSE; +static BOOL multiline = FALSE; static BOOL number = FALSE; +static BOOL quiet = FALSE; static BOOL recurse = FALSE; static BOOL silent = FALSE; static BOOL whole_lines = FALSE; +static BOOL word_match = FALSE; /* Structure for options and list of them */ +enum { OP_NODATA, OP_STRING, OP_NUMBER }; + typedef struct option_item { + int type; int one_char; + void *dataptr; const char *long_name; const char *help_text; } option_item; static option_item optionlist[] = { - { -1, "help", "display this help and exit" }, - { 'c', "count", "print only a count of matching lines per FILE" }, - { 'h', "no-filename", "suppress the prefixing filename on output" }, - { 'i', "ignore-case", "ignore case distinctions" }, - { 'l', "files-with-matches", "print only FILE names containing matches" }, - { 'n', "line-number", "print line number with output lines" }, - { 'r', "recursive", "recursively scan sub-directories" }, - { 's', "no-messages", "suppress error messages" }, - { 'u', "utf-8", "use UTF-8 mode" }, - { 'V', "version", "print version information and exit" }, - { 'v', "invert-match", "select non-matching lines" }, - { 'x', "line-regex", "force PATTERN to match only whole lines" }, - { 'x', "line-regexp", "force PATTERN to match only whole lines" }, - { 0, NULL, NULL } + { OP_NODATA, -1, NULL, "", " terminate options" }, + { OP_NODATA, -1, NULL, "help", "display this help and exit" }, + { OP_NUMBER, 'A', &after_context, "after-context=number", "set number of following context lines" }, + { OP_NUMBER, 'B', &before_context, "before-context=number", "set number of prior context lines" }, + { OP_NUMBER, 'C', &both_context, "context=number", "set number of context lines, before & after" }, + { OP_NODATA, 'c', NULL, "count", "print only a count of matching lines per FILE" }, + { OP_STRING, 'f', &pattern_filename, "file=path", "read patterns from file" }, + { OP_NODATA, 'h', NULL, "no-filename", "suppress the prefixing filename on output" }, + { OP_NODATA, 'i', NULL, "ignore-case", "ignore case distinctions" }, + { OP_NODATA, 'l', NULL, "files-with-matches", "print only FILE names containing matches" }, + { OP_NODATA, 'L', NULL, "files-without-match","print only FILE names not containing matches" }, + { OP_STRING, -1, &stdin_name, "label=name", "set name for standard input" }, + { OP_NODATA, 'M', NULL, "multiline", "run in multiline mode" }, + { OP_NODATA, 'n', NULL, "line-number", "print line number with output lines" }, + { OP_NODATA, 'q', NULL, "quiet", "suppress output, just set return code" }, + { OP_NODATA, 'r', NULL, "recursive", "recursively scan sub-directories" }, + { OP_STRING, -1, &exclude_pattern, "exclude=pattern","exclude matching files when recursing" }, + { OP_STRING, -1, &include_pattern, "include=pattern","include matching files when recursing" }, + { OP_NODATA, 's', NULL, "no-messages", "suppress error messages" }, + { OP_NODATA, 'u', NULL, "utf-8", "use UTF-8 mode" }, + { OP_NODATA, 'V', NULL, "version", "print version information and exit" }, + { OP_NODATA, 'v', NULL, "invert-match", "select non-matching lines" }, + { OP_NODATA, 'w', NULL, "word-regex(p)", "force PATTERN to match only as a word" }, + { OP_NODATA, 'x', NULL, "line-regex(p)", "force PATTERN to match only whole lines" }, + { OP_NODATA, 0, NULL, NULL, NULL } }; @@ -154,7 +197,8 @@ closedir(dir); /************* Directory scanning in Win32 ***********/ /* I (Philip Hazel) have no means of testing this code. It was contributed by -Lionel Fourquaux. */ +Lionel Fourquaux. David Burgess added a patch to define INVALID_FILE_ATTRIBUTES +when it did not exist. */ #elif HAVE_WIN32API @@ -165,6 +209,10 @@ Lionel Fourquaux. */ #ifndef WIN32_LEAN_AND_MEAN # define WIN32_LEAN_AND_MEAN #endif +#ifndef INVALID_FILE_ATTRIBUTES +#define INVALID_FILE_ATTRIBUTES 0xFFFFFFFF +#endif + #include <windows.h> typedef struct directory_type @@ -284,59 +332,290 @@ return sys_errlist[n]; /************************************************* -* Grep an individual file * +* Print the previous "after" lines * *************************************************/ +/* This is called if we are about to lose said lines because of buffer filling, +and at the end of the file. + +Arguments: + lastmatchnumber the number of the last matching line, plus one + lastmatchrestart where we restarted after the last match + endptr end of available data + printname filename for printing + +Returns: nothing +*/ + +static void do_after_lines(int lastmatchnumber, char *lastmatchrestart, + char *endptr, char *printname) +{ +if (after_context > 0 && lastmatchnumber > 0) + { + int count = 0; + while (lastmatchrestart < endptr && count++ < after_context) + { + char *pp = lastmatchrestart; + if (printname != NULL) fprintf(stdout, "%s-", printname); + if (number) fprintf(stdout, "%d-", lastmatchnumber++); + while (*pp != '\n') pp++; + fprintf(stdout, "%.*s", pp - lastmatchrestart + 1, lastmatchrestart); + lastmatchrestart = pp + 1; + } + hyphenpending = TRUE; + } +} + + + +/************************************************* +* Grep an individual file * +*************************************************/ + +/* This is called from grep_or_recurse() below. It uses a buffer that is three +times the value of MBUFTHIRD. The matching point is never allowed to stray into +the top third of the buffer, thus keeping more of the file available for +context printing or for multiline scanning. For large files, the pointer will +be in the middle third most of the time, so the bottom third is available for +"before" context printing. + +Arguments: + in the fopened FILE stream + printname the file name if it is to be printed for each match + or NULL if the file name is not to be printed + it cannot be NULL if filenames[_nomatch]_only is set + +Returns: 0 if there was at least one match + 1 otherwise (no matches) +*/ + static int -pcregrep(FILE *in, char *name) +pcregrep(FILE *in, char *printname) { int rc = 1; -int linenumber = 0; +int linenumber = 1; +int lastmatchnumber = 0; int count = 0; int offsets[99]; -char buffer[BUFSIZ]; +char *lastmatchrestart = NULL; +char buffer[3*MBUFTHIRD]; +char *ptr = buffer; +char *endptr; +size_t bufflength; +BOOL endhyphenpending = FALSE; + +/* Do the first read into the start of the buffer and set up the pointer to +end of what we have. */ -while (fgets(buffer, sizeof(buffer), in) != NULL) +bufflength = fread(buffer, 1, 3*MBUFTHIRD, in); +endptr = buffer + bufflength; + +/* Loop while the current pointer is not at the end of the file. For large +files, endptr will be at the end of the buffer when we are in the middle of the +file, but ptr will never get there, because as soon as it gets over 2/3 of the +way, the buffer is shifted left and re-filled. */ + +while (ptr < endptr) { - BOOL match = FALSE; int i; - int length = (int)strlen(buffer); - if (length > 0 && buffer[length-1] == '\n') buffer[--length] = 0; - linenumber++; + BOOL match = FALSE; + char *t = ptr; + size_t length, linelength; + + /* At this point, ptr is at the start of a line. We need to find the length + of the subject string to pass to pcre_exec(). In multiline mode, it is the + length remainder of the data in the buffer. Otherwise, it is the length of + the next line. After matching, we always advance by the length of the next + line. In multiline mode the PCRE_FIRSTLINE option is used for compiling, so + that any match is constrained to be in the first line. */ + + linelength = 0; + while (t < endptr && *t++ != '\n') linelength++; + length = multiline? endptr - ptr : linelength; + + /* Run through all the patterns until one matches. Note that we don't include + the final newline in the subject string. */ for (i = 0; !match && i < pattern_count; i++) { - match = pcre_exec(pattern_list[i], hints_list[i], buffer, length, 0, 0, + match = pcre_exec(pattern_list[i], hints_list[i], ptr, length, 0, 0, offsets, 99) >= 0; - if (match && whole_lines && offsets[1] != length) match = FALSE; } + /* If it's a match or a not-match (as required), print what's wanted. */ + if (match != invert) { + BOOL hyphenprinted = FALSE; + + if (filenames_nomatch_only) return 1; + if (count_only) count++; else if (filenames_only) { - fprintf(stdout, "%s\n", (name == NULL)? "<stdin>" : name); + fprintf(stdout, "%s\n", printname); return 0; } - else if (silent) return 0; + else if (quiet) return 0; else { - if (name != NULL) fprintf(stdout, "%s:", name); + /* See if there is a requirement to print some "after" lines from a + previous match. We never print any overlaps. */ + + if (after_context > 0 && lastmatchnumber > 0) + { + int linecount = 0; + char *p = lastmatchrestart; + + while (p < ptr && linecount < after_context) + { + while (*p != '\n') p++; + p++; + linecount++; + } + + /* It is important to advance lastmatchrestart during this printing so + that it interacts correctly with any "before" printing below. */ + + while (lastmatchrestart < p) + { + char *pp = lastmatchrestart; + if (printname != NULL) fprintf(stdout, "%s-", printname); + if (number) fprintf(stdout, "%d-", lastmatchnumber++); + while (*pp != '\n') pp++; + fprintf(stdout, "%.*s", pp - lastmatchrestart + 1, lastmatchrestart); + lastmatchrestart = pp + 1; + } + if (lastmatchrestart != ptr) hyphenpending = TRUE; + } + + /* If there were non-contiguous lines printed above, insert hyphens. */ + + if (hyphenpending) + { + fprintf(stdout, "--\n"); + hyphenpending = FALSE; + hyphenprinted = TRUE; + } + + /* See if there is a requirement to print some "before" lines for this + match. Again, don't print overlaps. */ + + if (before_context > 0) + { + int linecount = 0; + char *p = ptr; + + while (p > buffer && (lastmatchnumber == 0 || p > lastmatchrestart) && + linecount++ < before_context) + { + p--; + while (p > buffer && p[-1] != '\n') p--; + } + + if (lastmatchnumber > 0 && p > lastmatchrestart && !hyphenprinted) + fprintf(stdout, "--\n"); + + while (p < ptr) + { + char *pp = p; + if (printname != NULL) fprintf(stdout, "%s-", printname); + if (number) fprintf(stdout, "%d-", linenumber - linecount--); + while (*pp != '\n') pp++; + fprintf(stdout, "%.*s", pp - p + 1, p); + p = pp + 1; + } + } + + /* Now print the matching line(s); ensure we set hyphenpending at the end + of the file. */ + + endhyphenpending = TRUE; + if (printname != NULL) fprintf(stdout, "%s:", printname); if (number) fprintf(stdout, "%d:", linenumber); - fprintf(stdout, "%s\n", buffer); + + /* In multiline mode, we want to print to the end of the line in which + the end of the matched string is found, so we adjust linelength and the + line number appropriately. Because the PCRE_FIRSTLINE option is set, the + start of the match will always be before the first \n character. */ + + if (multiline) + { + char *endmatch = ptr + offsets[1]; + t = ptr; + while (t < endmatch) { if (*t++ == '\n') linenumber++; } + while (endmatch < endptr && *endmatch != '\n') endmatch++; + linelength = endmatch - ptr; + } + + fprintf(stdout, "%.*s\n", linelength, ptr); } - rc = 0; + rc = 0; /* Had some success */ + + /* Remember where the last match happened for after_context. We remember + where we are about to restart, and that line's number. */ + + lastmatchrestart = ptr + linelength + 1; + lastmatchnumber = linenumber + 1; } + + /* Advance to after the newline and increment the line number. */ + + ptr += linelength + 1; + linenumber++; + + /* If we haven't yet reached the end of the file (the buffer is full), and + the current point is in the top 1/3 of the buffer, slide the buffer down by + 1/3 and refill it. Before we do this, if some unprinted "after" lines are + about to be lost, print them. */ + + if (bufflength >= sizeof(buffer) && ptr > buffer + 2*MBUFTHIRD) + { + if (after_context > 0 && + lastmatchnumber > 0 && + lastmatchrestart < buffer + MBUFTHIRD) + { + do_after_lines(lastmatchnumber, lastmatchrestart, endptr, printname); + lastmatchnumber = 0; + } + + /* Now do the shuffle */ + + memmove(buffer, buffer + MBUFTHIRD, 2*MBUFTHIRD); + ptr -= MBUFTHIRD; + bufflength = 2*MBUFTHIRD + fread(buffer + 2*MBUFTHIRD, 1, MBUFTHIRD, in); + endptr = buffer + bufflength; + + /* Adjust any last match point */ + + if (lastmatchnumber > 0) lastmatchrestart -= MBUFTHIRD; + } + } /* Loop through the whole file */ + +/* End of file; print final "after" lines if wanted; do_after_lines sets +hyphenpending if it prints something. */ + +do_after_lines(lastmatchnumber, lastmatchrestart, endptr, printname); +hyphenpending |= endhyphenpending; + +/* Print the file name if we are looking for those without matches and there +were none. If we found a match, we won't have got this far. */ + +if (filenames_nomatch_only) + { + fprintf(stdout, "%s\n", printname); + return 0; } +/* Print the match count if wanted */ + if (count_only) { - if (name != NULL) fprintf(stdout, "%s:", name); + if (printname != NULL) fprintf(stdout, "%s:", printname); fprintf(stdout, "%d\n", count); } @@ -345,41 +624,81 @@ return rc; - /************************************************* * Grep a file or recurse into a directory * *************************************************/ +/* Given a path name, if it's a directory, scan all the files if we are +recursing; if it's a file, grep it. + +Arguments: + pathname the path to investigate + dir_recurse TRUE if recursing is wanted (-r) + show_filenames TRUE if file names are wanted for multiple files, except + for the only file at top level when not filenames_only + only_one_at_top TRUE if the path is the only one at toplevel + +Returns: 0 if there was at least one match + 1 if there were no matches + 2 there was some kind of error + +However, file opening failures are suppressed if "silent" is set. +*/ + static int -grep_or_recurse(char *filename, BOOL dir_recurse, BOOL show_filenames, +grep_or_recurse(char *pathname, BOOL dir_recurse, BOOL show_filenames, BOOL only_one_at_top) { int rc = 1; int sep; FILE *in; +char *printname; + +/* If the file name is "-" we scan stdin */ + +if (strcmp(pathname, "-") == 0) + { + return pcregrep(stdin, + (filenames_only || filenames_nomatch_only || + (show_filenames && !only_one_at_top))? + stdin_name : NULL); + } -/* If the file is a directory and we are recursing, scan each file within it. -The scanning code is localized so it can be made system-specific. */ +/* If the file is a directory and we are recursing, scan each file within it, +subject to any include or exclude patterns that were set. The scanning code is +localized so it can be made system-specific. */ -if ((sep = isdirectory(filename)) != 0 && dir_recurse) +if ((sep = isdirectory(pathname)) != 0 && dir_recurse) { char buffer[1024]; char *nextfile; - directory_type *dir = opendirectory(filename); + directory_type *dir = opendirectory(pathname); if (dir == NULL) { - fprintf(stderr, "pcregrep: Failed to open directory %s: %s\n", filename, - strerror(errno)); + if (!silent) + fprintf(stderr, "pcregrep: Failed to open directory %s: %s\n", pathname, + strerror(errno)); return 2; } while ((nextfile = readdirectory(dir)) != NULL) { - int frc; - sprintf(buffer, "%.512s%c%.128s", filename, sep, nextfile); + int frc, blen; + sprintf(buffer, "%.512s%c%.128s", pathname, sep, nextfile); + blen = strlen(buffer); + + if (exclude_compiled != NULL && + pcre_exec(exclude_compiled, NULL, buffer, blen, 0, 0, NULL, 0) >= 0) + continue; + + if (include_compiled != NULL && + pcre_exec(include_compiled, NULL, buffer, blen, 0, 0, NULL, 0) < 0) + continue; + frc = grep_or_recurse(buffer, dir_recurse, TRUE, FALSE); - if (frc == 0 && rc == 1) rc = 0; + if (frc > 1) rc = frc; + else if (frc == 0 && rc == 1) rc = 0; } closedirectory(dir); @@ -391,15 +710,20 @@ the first and only argument at top level, we don't show the file name (unless we are only showing the file name). Otherwise, control is via the show_filenames variable. */ -in = fopen(filename, "r"); +in = fopen(pathname, "r"); if (in == NULL) { - fprintf(stderr, "pcregrep: Failed to open %s: %s\n", filename, strerror(errno)); + if (!silent) + fprintf(stderr, "pcregrep: Failed to open %s: %s\n", pathname, + strerror(errno)); return 2; } -rc = pcregrep(in, (filenames_only || (show_filenames && !only_one_at_top))? - filename : NULL); +printname = (filenames_only || filenames_nomatch_only || + (show_filenames && !only_one_at_top))? pathname : NULL; + +rc = pcregrep(in, printname); + fclose(in); return rc; } @@ -414,7 +738,7 @@ return rc; static int usage(int rc) { -fprintf(stderr, "Usage: pcregrep [-Vcfhilnrsvx] [long-options] [pattern] [file1 file2 ...]\n"); +fprintf(stderr, "Usage: pcregrep [-LMVcfhilnqrsvwx] [long-options] [pattern] [file1 file2 ...]\n"); fprintf(stderr, "Type `pcregrep --help' for more information.\n"); return rc; } @@ -434,6 +758,7 @@ option_item *op; printf("Usage: pcregrep [OPTION]... [PATTERN] [FILE1 FILE2 ...]\n"); printf("Search for PATTERN in each FILE or standard input.\n"); printf("PATTERN must be present if -f is not used.\n"); +printf("\"-\" can be used as a file name to mean STDIN.\n"); printf("Example: pcregrep -i 'hello.*world' menu.h main.c\n\n"); printf("Options:\n"); @@ -449,12 +774,11 @@ for (op = optionlist; op->one_char != 0; op++) printf("%.*s%s\n", n, " ", op->help_text); } -printf("\n -f<filename> or --file=<filename>\n"); -printf(" Read patterns from <filename> instead of using a command line option.\n"); -printf(" Trailing white space is removed; blanks lines are ignored.\n"); -printf(" There is a maximum of %d patterns.\n", MAX_PATTERN_COUNT); +printf("\nWhen reading patterns from a file instead of using a command line option,\n"); +printf("trailing white space is removed and blank lines are ignored.\n"); +printf("There is a maximum of %d patterns.\n", MAX_PATTERN_COUNT); -printf("\nWith no FILE, read standard input. If fewer than two FILEs given, assume -h.\n"); +printf("\nWith no FILEs, read standard input. If fewer than two FILEs given, assume -h.\n"); printf("Exit status is 0 if any matches, 1 if no matches, and 2 if trouble.\n"); } @@ -462,7 +786,7 @@ printf("Exit status is 0 if any matches, 1 if no matches, and 2 if trouble.\n"); /************************************************* -* Handle an option * +* Handle a single-letter, no data option * *************************************************/ static int @@ -474,13 +798,17 @@ switch(letter) case 'c': count_only = TRUE; break; case 'h': filenames = FALSE; break; case 'i': options |= PCRE_CASELESS; break; - case 'l': filenames_only = TRUE; + case 'l': filenames_only = TRUE; break; + case 'L': filenames_nomatch_only = TRUE; break; + case 'M': multiline = TRUE; options |= PCRE_MULTILINE|PCRE_FIRSTLINE; break; case 'n': number = TRUE; break; + case 'q': quiet = TRUE; break; case 'r': recurse = TRUE; break; case 's': silent = TRUE; break; case 'u': options |= PCRE_UTF8; break; case 'v': invert = TRUE; break; - case 'x': whole_lines = TRUE; options |= PCRE_ANCHORED; break; + case 'w': word_match = TRUE; break; + case 'x': whole_lines = TRUE; break; case 'V': fprintf(stderr, "pcregrep version %s using ", VERSION); @@ -503,6 +831,8 @@ return options; * Main program * *************************************************/ +/* Returns 0 if something matched, 1 if nothing matched, 2 after an error. */ + int main(int argc, char **argv) { @@ -517,32 +847,83 @@ BOOL only_one_at_top; for (i = 1; i < argc; i++) { + option_item *op = NULL; + char *option_data = (char *)""; /* default to keep compiler happy */ + BOOL longop; + BOOL longopwasequals = FALSE; + if (argv[i][0] != '-') break; - /* Missing options */ + /* If we hit an argument that is just "-", it may be a reference to STDIN, + but only if we have previously had -f to define the patterns. */ - if (argv[i][1] == 0) exit(usage(2)); + if (argv[i][1] == 0) + { + if (pattern_filename != NULL) break; + else exit(usage(2)); + } - /* Long name options */ + /* Handle a long name option, or -- to terminate the options */ if (argv[i][1] == '-') { - option_item *op; + char *arg = argv[i] + 2; + char *argequals = strchr(arg, '='); - if (strncmp(argv[i]+2, "file=", 5) == 0) + if (*arg == 0) /* -- terminates options */ { - pattern_filename = argv[i] + 7; - continue; + i++; + break; /* out of the options-handling loop */ } + longop = TRUE; + + /* Some long options have data that follows after =, for example file=name. + Some options have variations in the long name spelling: specifically, we + allow "regexp" because GNU grep allows it, though I personally go along + with Jeff Friedl in preferring "regex" without the "p". These options are + entered in the table as "regex(p)". No option is in both these categories, + fortunately. */ + for (op = optionlist; op->one_char != 0; op++) { - if (strcmp(argv[i]+2, op->long_name) == 0) + char *opbra = strchr(op->long_name, '('); + char *equals = strchr(op->long_name, '='); + if (opbra == NULL) /* Not a (p) case */ { - options = handle_option(op->one_char, options); - break; + if (equals == NULL) /* Not thing=data case */ + { + if (strcmp(arg, op->long_name) == 0) break; + } + else /* Special case xxx=data */ + { + int oplen = equals - op->long_name; + int arglen = (argequals == NULL)? strlen(arg) : argequals - arg; + if (oplen == arglen && strncmp(arg, op->long_name, oplen) == 0) + { + option_data = arg + arglen; + if (*option_data == '=') + { + option_data++; + longopwasequals = TRUE; + } + break; + } + } + } + else /* Special case xxxx(p) */ + { + char buff1[24]; + char buff2[24]; + int baselen = opbra - op->long_name; + sprintf(buff1, "%.*s", baselen, op->long_name); + sprintf(buff2, "%s%.*s", buff1, strlen(op->long_name) - baselen - 2, + opbra + 1); + if (strcmp(arg, buff1) == 0 || strcmp(arg, buff2) == 0) + break; } } + if (op->one_char == 0) { fprintf(stderr, "pcregrep: Unknown option %s\n", argv[i]); @@ -550,30 +931,74 @@ for (i = 1; i < argc; i++) } } - /* One-char options */ + /* One-char options; many that have no data may be in a single argument; we + continue till we hit the last one or one that needs data. */ else { char *s = argv[i] + 1; + longop = FALSE; while (*s != 0) { - if (*s == 'f') + for (op = optionlist; op->one_char != 0; op++) + { if (*s == op->one_char) break; } + if (op->one_char == 0) { - pattern_filename = s + 1; - if (pattern_filename[0] == 0) - { - if (i >= argc - 1) - { - fprintf(stderr, "pcregrep: File name missing after -f\n"); - exit(usage(2)); - } - pattern_filename = argv[++i]; - } + fprintf(stderr, "pcregrep: Unknown option letter '%c' in \"%s\"\n", + *s, argv[i]); + exit(usage(2)); + } + if (op->type != OP_NODATA || s[1] == 0) + { + option_data = s+1; break; } - else options = handle_option(*s++, options); + options = handle_option(*s++, options); } } + + /* At this point we should have op pointing to a matched option */ + + if (op->type == OP_NODATA) + options = handle_option(op->one_char, options); + else + { + if (*option_data == 0) + { + if (i >= argc - 1 || longopwasequals) + { + fprintf(stderr, "pcregrep: Data missing after %s\n", argv[i]); + exit(usage(2)); + } + option_data = argv[++i]; + } + + if (op->type == OP_STRING) *((char **)op->dataptr) = option_data; else + { + char *endptr; + int n = strtoul(option_data, &endptr, 10); + if (*endptr != 0) + { + if (longop) + fprintf(stderr, "pcregrep: Malformed number \"%s\" after --%s\n", + option_data, op->long_name); + else + fprintf(stderr, "pcregrep: Malformed number \"%s\" after -%c\n", + option_data, op->one_char); + exit(usage(2)); + } + *((int *)op->dataptr) = n; + } + } + } + +/* Options have been decoded. If -C was used, its value is used as a default +for -A and -B. */ + +if (both_context > 0) + { + if (after_context == 0) after_context = both_context; + if (before_context == 0) before_context = both_context; } pattern_list = (pcre **)malloc(MAX_PATTERN_COUNT * sizeof(pcre *)); @@ -590,53 +1015,91 @@ if (pattern_list == NULL || hints_list == NULL) if (pattern_filename != NULL) { FILE *f = fopen(pattern_filename, "r"); - char buffer[BUFSIZ]; + char buffer[MBUFTHIRD + 16]; + char *rdstart; + int adjust = 0; + if (f == NULL) { fprintf(stderr, "pcregrep: Failed to open %s: %s\n", pattern_filename, strerror(errno)); return 2; } - while (fgets(buffer, sizeof(buffer), f) != NULL) + + if (whole_lines) + { + strcpy(buffer, "^(?:"); + adjust = 4; + } + else if (word_match) + { + strcpy(buffer, "\\b"); + adjust = 2; + } + + rdstart = buffer + adjust; + while (fgets(rdstart, MBUFTHIRD, f) != NULL) { - char *s = buffer + (int)strlen(buffer); + char *s = rdstart + (int)strlen(rdstart); if (pattern_count >= MAX_PATTERN_COUNT) { fprintf(stderr, "pcregrep: Too many patterns in file (max %d)\n", MAX_PATTERN_COUNT); return 2; } - while (s > buffer && isspace((unsigned char)(s[-1]))) s--; - if (s == buffer) continue; - *s = 0; + while (s > rdstart && isspace((unsigned char)(s[-1]))) s--; + if (s == rdstart) continue; + if (whole_lines) strcpy(s, ")$"); + else if (word_match)strcpy(s, "\\b"); + else *s = 0; pattern_list[pattern_count] = pcre_compile(buffer, options, &error, &errptr, NULL); if (pattern_list[pattern_count++] == NULL) { fprintf(stderr, "pcregrep: Error in regex number %d at offset %d: %s\n", - pattern_count, errptr, error); + pattern_count, errptr - adjust, error); return 2; } } fclose(f); } -/* If no file name, a single regex must be given inline */ +/* If no file name, a single regex must be given inline. */ else { + char buffer[MBUFTHIRD + 16]; + char *pat; + int adjust = 0; + if (i >= argc) return usage(2); - pattern_list[0] = pcre_compile(argv[i++], options, &error, &errptr, NULL); + + if (whole_lines) + { + sprintf(buffer, "^(?:%.*s)$", MBUFTHIRD, argv[i++]); + pat = buffer; + adjust = 4; + } + else if (word_match) + { + sprintf(buffer, "\\b%.*s\\b", MBUFTHIRD, argv[i++]); + pat = buffer; + adjust = 2; + } + else pat = argv[i++]; + + pattern_list[0] = pcre_compile(pat, options, &error, &errptr, NULL); + if (pattern_list[0] == NULL) { - fprintf(stderr, "pcregrep: Error in regex at offset %d: %s\n", errptr, - error); + fprintf(stderr, "pcregrep: Error in regex at offset %d: %s\n", + errptr - adjust, error); return 2; } pattern_count++; } -/* Study the regular expressions, as we will be running them may times */ +/* Study the regular expressions, as we will be running them many times */ for (j = 0; j < pattern_count; j++) { @@ -650,24 +1113,50 @@ for (j = 0; j < pattern_count; j++) } } +/* If there are include or exclude patterns, compile them. */ + +if (exclude_pattern != NULL) + { + exclude_compiled = pcre_compile(exclude_pattern, 0, &error, &errptr, NULL); + if (exclude_compiled == NULL) + { + fprintf(stderr, "pcregrep: Error in 'exclude' regex at offset %d: %s\n", + errptr, error); + return 2; + } + } + +if (include_pattern != NULL) + { + include_compiled = pcre_compile(include_pattern, 0, &error, &errptr, NULL); + if (include_compiled == NULL) + { + fprintf(stderr, "pcregrep: Error in 'include' regex at offset %d: %s\n", + errptr, error); + return 2; + } + } + /* If there are no further arguments, do the business on stdin and exit */ -if (i >= argc) return pcregrep(stdin, NULL); +if (i >= argc) return pcregrep(stdin, + (filenames_only || filenames_nomatch_only)? stdin_name : NULL); /* Otherwise, work through the remaining arguments as files or directories. Pass in the fact that there is only one argument at top level - this suppresses -the file name if the argument is not a directory. */ +the file name if the argument is not a directory and filenames_only is not set. +*/ only_one_at_top = (i == argc - 1); -if (filenames_only) filenames = TRUE; for (; i < argc; i++) { int frc = grep_or_recurse(argv[i], recurse, filenames, only_one_at_top); - if (frc == 0 && rc == 1) rc = 0; + if (frc > 1) rc = frc; + else if (frc == 0 && rc == 1) rc = 0; } return rc; } -/* End */ +/* End of pcregrep */ diff --git a/pcreposix.c b/pcreposix.c index 1e8b6a7..06fd58b 100644 --- a/pcreposix.c +++ b/pcreposix.c @@ -2,17 +2,11 @@ * Perl-Compatible Regular Expressions * *************************************************/ -/* -This is a library of functions to support regular expressions whose syntax -and semantics are as close as possible to those of the Perl 5 language. See -the file Tech.Notes for some information on the internals. - -This module is a wrapper that provides a POSIX API to the underlying PCRE -functions. +/* PCRE is a library of functions to support regular expressions whose syntax +and semantics are as close as possible to those of the Perl 5 language. -Written by: Philip Hazel <ph10@cam.ac.uk> - - Copyright (c) 1997-2004 University of Cambridge + Written by Philip Hazel + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -43,69 +37,68 @@ POSSIBILITY OF SUCH DAMAGE. ----------------------------------------------------------------------------- */ -#include "internal.h" + +/* This module is a wrapper that provides a POSIX API to the underlying PCRE +functions. */ + + +#include "pcre_internal.h" #include "pcreposix.h" #include "stdlib.h" -/* Corresponding tables of PCRE error messages and POSIX error codes. */ - -static const char *const estring[] = { - ERR1, ERR2, ERR3, ERR4, ERR5, ERR6, ERR7, ERR8, ERR9, ERR10, - ERR11, ERR12, ERR13, ERR14, ERR15, ERR16, ERR17, ERR18, ERR19, ERR20, - ERR21, ERR22, ERR23, ERR24, ERR25, ERR26, ERR27, ERR29, ERR29, ERR30, - ERR31, ERR32, ERR33, ERR34, ERR35, ERR36, ERR37, ERR38, ERR39, ERR40, - ERR41, ERR42, ERR43, ERR44, ERR45, ERR46, ERR47 }; +/* Table to translate PCRE compile time error codes into POSIX error codes. */ static const int eint[] = { - REG_EESCAPE, /* "\\ at end of pattern" */ - REG_EESCAPE, /* "\\c at end of pattern" */ - REG_EESCAPE, /* "unrecognized character follows \\" */ - REG_BADBR, /* "numbers out of order in {} quantifier" */ - REG_BADBR, /* "number too big in {} quantifier" */ - REG_EBRACK, /* "missing terminating ] for character class" */ - REG_ECTYPE, /* "invalid escape sequence in character class" */ - REG_ERANGE, /* "range out of order in character class" */ - REG_BADRPT, /* "nothing to repeat" */ - REG_BADRPT, /* "operand of unlimited repeat could match the empty string" */ - REG_ASSERT, /* "internal error: unexpected repeat" */ - REG_BADPAT, /* "unrecognized character after (?" */ - REG_BADPAT, /* "POSIX named classes are supported only within a class" */ - REG_EPAREN, /* "missing )" */ - REG_ESUBREG, /* "reference to non-existent subpattern" */ - REG_INVARG, /* "erroffset passed as NULL" */ - REG_INVARG, /* "unknown option bit(s) set" */ - REG_EPAREN, /* "missing ) after comment" */ - REG_ESIZE, /* "parentheses nested too deeply" */ - REG_ESIZE, /* "regular expression too large" */ - REG_ESPACE, /* "failed to get memory" */ - REG_EPAREN, /* "unmatched brackets" */ - REG_ASSERT, /* "internal error: code overflow" */ - REG_BADPAT, /* "unrecognized character after (?<" */ - REG_BADPAT, /* "lookbehind assertion is not fixed length" */ - REG_BADPAT, /* "malformed number after (?(" */ - REG_BADPAT, /* "conditional group containe more than two branches" */ - REG_BADPAT, /* "assertion expected after (?(" */ - REG_BADPAT, /* "(?R or (?digits must be followed by )" */ - REG_ECTYPE, /* "unknown POSIX class name" */ - REG_BADPAT, /* "POSIX collating elements are not supported" */ - REG_INVARG, /* "this version of PCRE is not compiled with PCRE_UTF8 support" */ - REG_BADPAT, /* "spare error" */ - REG_BADPAT, /* "character value in \x{...} sequence is too large" */ - REG_BADPAT, /* "invalid condition (?(0)" */ - REG_BADPAT, /* "\\C not allowed in lookbehind assertion" */ - REG_EESCAPE, /* "PCRE does not support \\L, \\l, \\N, \\U, or \\u" */ - REG_BADPAT, /* "number after (?C is > 255" */ - REG_BADPAT, /* "closing ) for (?C expected" */ - REG_BADPAT, /* "recursive call could loop indefinitely" */ - REG_BADPAT, /* "unrecognized character after (?P" */ - REG_BADPAT, /* "syntax error after (?P" */ - REG_BADPAT, /* "two named groups have the same name" */ - REG_BADPAT, /* "invalid UTF-8 string" */ - REG_BADPAT, /* "support for \\P, \\p, and \\X has not been compiled" */ - REG_BADPAT, /* "malformed \\P or \\p sequence" */ - REG_BADPAT /* "unknown property name after \\P or \\p" */ + 0, /* no error */ + REG_EESCAPE, /* \ at end of pattern */ + REG_EESCAPE, /* \c at end of pattern */ + REG_EESCAPE, /* unrecognized character follows \ */ + REG_BADBR, /* numbers out of order in {} quantifier */ + REG_BADBR, /* number too big in {} quantifier */ + REG_EBRACK, /* missing terminating ] for character class */ + REG_ECTYPE, /* invalid escape sequence in character class */ + REG_ERANGE, /* range out of order in character class */ + REG_BADRPT, /* nothing to repeat */ + REG_BADRPT, /* operand of unlimited repeat could match the empty string */ + REG_ASSERT, /* internal error: unexpected repeat */ + REG_BADPAT, /* unrecognized character after (? */ + REG_BADPAT, /* POSIX named classes are supported only within a class */ + REG_EPAREN, /* missing ) */ + REG_ESUBREG, /* reference to non-existent subpattern */ + REG_INVARG, /* erroffset passed as NULL */ + REG_INVARG, /* unknown option bit(s) set */ + REG_EPAREN, /* missing ) after comment */ + REG_ESIZE, /* parentheses nested too deeply */ + REG_ESIZE, /* regular expression too large */ + REG_ESPACE, /* failed to get memory */ + REG_EPAREN, /* unmatched brackets */ + REG_ASSERT, /* internal error: code overflow */ + REG_BADPAT, /* unrecognized character after (?< */ + REG_BADPAT, /* lookbehind assertion is not fixed length */ + REG_BADPAT, /* malformed number after (?( */ + REG_BADPAT, /* conditional group containe more than two branches */ + REG_BADPAT, /* assertion expected after (?( */ + REG_BADPAT, /* (?R or (?digits must be followed by ) */ + REG_ECTYPE, /* unknown POSIX class name */ + REG_BADPAT, /* POSIX collating elements are not supported */ + REG_INVARG, /* this version of PCRE is not compiled with PCRE_UTF8 support */ + REG_BADPAT, /* spare error */ + REG_BADPAT, /* character value in \x{...} sequence is too large */ + REG_BADPAT, /* invalid condition (?(0) */ + REG_BADPAT, /* \C not allowed in lookbehind assertion */ + REG_EESCAPE, /* PCRE does not support \L, \l, \N, \U, or \u */ + REG_BADPAT, /* number after (?C is > 255 */ + REG_BADPAT, /* closing ) for (?C expected */ + REG_BADPAT, /* recursive call could loop indefinitely */ + REG_BADPAT, /* unrecognized character after (?P */ + REG_BADPAT, /* syntax error after (?P */ + REG_BADPAT, /* two named groups have the same name */ + REG_BADPAT, /* invalid UTF-8 string */ + REG_BADPAT, /* support for \P, \p, and \X has not been compiled */ + REG_BADPAT, /* malformed \P or \p sequence */ + REG_BADPAT /* unknown property name after \P or \p */ }; /* Table of texts corresponding to POSIX error codes */ @@ -135,24 +128,6 @@ static const char *const pstring[] = { /************************************************* -* Translate PCRE text code to int * -*************************************************/ - -/* PCRE compile-time errors are given as strings defined as macros. We can just -look them up in a table to turn them into POSIX-style error codes. */ - -static int -pcre_posix_error_code(const char *s) -{ -size_t i; -for (i = 0; i < sizeof(estring)/sizeof(char *); i++) - if (strcmp(s, estring[i]) == 0) return eint[i]; -return REG_ASSERT; -} - - - -/************************************************* * Translate error code to string * *************************************************/ @@ -219,15 +194,18 @@ regcomp(regex_t *preg, const char *pattern, int cflags) { const char *errorptr; int erroffset; +int errorcode; int options = 0; if ((cflags & REG_ICASE) != 0) options |= PCRE_CASELESS; if ((cflags & REG_NEWLINE) != 0) options |= PCRE_MULTILINE; +if ((cflags & REG_DOTALL) != 0) options |= PCRE_DOTALL; -preg->re_pcre = pcre_compile(pattern, options, &errorptr, &erroffset, NULL); +preg->re_pcre = pcre_compile2(pattern, options, &errorcode, &errorptr, + &erroffset, NULL); preg->re_erroffset = erroffset; -if (preg->re_pcre == NULL) return pcre_posix_error_code(errorptr); +if (preg->re_pcre == NULL) return eint[errorcode]; preg->re_nsub = pcre_info((const pcre *)preg->re_pcre, NULL, NULL); return 0; diff --git a/pcreposix.h b/pcreposix.h index a8056bd..4f1b1ab 100644 --- a/pcreposix.h +++ b/pcreposix.h @@ -9,7 +9,7 @@ Compatible Regular Expression library. It defines the things POSIX says should be there. I hope. - Copyright (c) 1997-2004 University of Cambridge + Copyright (c) 1997-2005 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -57,6 +57,10 @@ extern "C" { #define REG_NOTBOL 0x04 #define REG_NOTEOL 0x08 +/* Additional options, not defined by POSIX, but somebody wanted them. */ + +#define REG_DOTALL 0x10 + /* These are not used by PCRE, but by defining them we make it easier to slot PCRE into existing programs that make POSIX calls. */ @@ -4,7 +4,7 @@ /* This program was hacked up as a tester for PCRE. I really should have written it more tidily in the first place. Will I ever learn? It has grown and -been extended and consequently is now rather untidy in places. +been extended and consequently is now rather, er, *very* untidy in places. ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -44,11 +44,15 @@ POSSIBILITY OF SUCH DAMAGE. #include <locale.h> #include <errno.h> -/* We need the internal info for displaying the results of pcre_study(). Also -for getting the opcodes for showing compiled code. */ - #define PCRE_SPY /* For Win32 build, import data, not export */ -#include "internal.h" + +/* We need the internal info for displaying the results of pcre_study() and +other internal data; pcretest also uses some of the fixed tables, and generally +has "inside information" compared to a program that strictly follows the PCRE +API. */ + +#include "pcre_internal.h" + /* It is possible to compile this test program without including support for testing the POSIX interface, though this is not available via the standard @@ -87,34 +91,6 @@ static size_t gotten_store; static uschar *pbuffer = NULL; -static const int utf8_table1[] = { - 0x0000007f, 0x000007ff, 0x0000ffff, 0x001fffff, 0x03ffffff, 0x7fffffff}; - -static const int utf8_table2[] = { - 0, 0xc0, 0xe0, 0xf0, 0xf8, 0xfc}; - -static const int utf8_table3[] = { - 0xff, 0x1f, 0x0f, 0x07, 0x03, 0x01}; - - - -/************************************************* -* Print compiled regex * -*************************************************/ - -/* The code for doing this is held in a separate file that is also included in -pcre.c when it is compiled with the debug switch. It defines a function called -print_internals(), which uses a table of opcode lengths defined by the macro -OP_LENGTHS, whose name must be OP_lengths. It also uses a table that translates -Unicode property names to numbers; this is kept in a separate file. */ - -static uschar OP_lengths[] = { OP_LENGTHS }; - -#include "ucp.h" -#include "ucptypetable.c" -#include "printint.c" - - /************************************************* * Read number from string * @@ -143,42 +119,6 @@ return(result); -/************************************************* -* Convert character value to UTF-8 * -*************************************************/ - -/* This function takes an integer value in the range 0 - 0x7fffffff -and encodes it as a UTF-8 character in 0 to 6 bytes. - -Arguments: - cvalue the character value - buffer pointer to buffer for result - at least 6 bytes long - -Returns: number of characters placed in the buffer - -1 if input character is negative - 0 if input character is positive but too big (only when - int is longer than 32 bits) -*/ - -static int -ord2utf8(int cvalue, unsigned char *buffer) -{ -register int i, j; -for (i = 0; i < sizeof(utf8_table1)/sizeof(int); i++) - if (cvalue <= utf8_table1[i]) break; -if (i >= sizeof(utf8_table1)/sizeof(int)) return 0; -if (cvalue < 0) return -1; - -buffer += i; -for (j = i; j > 0; j--) - { - *buffer-- = 0x80 | (cvalue & 0x3f); - cvalue >>= 6; - } -*buffer = utf8_table2[i] | cvalue; -return i + 1; -} - /************************************************* * Convert UTF-8 string to value * @@ -214,7 +154,7 @@ if (i == 0 || i == 6) return 0; /* invalid UTF-8 */ /* i now has a value in the range 1-5 */ s = 6*i; -d = (c & utf8_table3[i]) << s; +d = (c & _pcre_utf8_table3[i]) << s; for (j = 0; j < i; j++) { @@ -226,8 +166,8 @@ for (j = 0; j < i; j++) /* Check that encoding was the correct unique one */ -for (j = 0; j < sizeof(utf8_table1)/sizeof(int); j++) - if (d <= utf8_table1[j]) break; +for (j = 0; j < _pcre_utf8_table1_size; j++) + if (d <= _pcre_utf8_table1[j]) break; if (j != i) return -(i+1); /* Valid value */ @@ -403,7 +343,7 @@ static void *new_malloc(size_t size) void *block = malloc(size); gotten_store = size; if (show_malloc) - fprintf(outfile, "malloc %3d %p\n", size, block); + fprintf(outfile, "malloc %3d %p\n", (int)size, block); return block; } @@ -421,7 +361,7 @@ static void *stack_malloc(size_t size) { void *block = malloc(size); if (show_malloc) - fprintf(outfile, "stack_malloc %3d %p\n", size, block); + fprintf(outfile, "stack_malloc %3d %p\n", (int)size, block); return block; } @@ -484,12 +424,14 @@ int showinfo = 0; int showstore = 0; int size_offsets = 45; int size_offsets_max; -int *offsets; +int *offsets = NULL; #if !defined NOPOSIX int posix = 0; #endif int debug = 0; int done = 0; +int all_use_dfa = 0; +int yield = 0; unsigned char *buffer; unsigned char *dbuffer; @@ -522,6 +464,7 @@ while (argc > 1 && argv[op][0] == '-') else if (strcmp(argv[op], "-t") == 0) timeit = 1; else if (strcmp(argv[op], "-i") == 0) showinfo = 1; else if (strcmp(argv[op], "-d") == 0) showinfo = debug = 1; + else if (strcmp(argv[op], "-dfa") == 0) all_use_dfa = 1; else if (strcmp(argv[op], "-o") == 0 && argc > 2 && ((size_offsets = get_value((unsigned char *)argv[op+1], &endptr)), *endptr == 0)) @@ -558,8 +501,9 @@ while (argc > 1 && argv[op][0] == '-') printf("** Unknown or malformed option %s\n", argv[op]); printf("Usage: pcretest [-d] [-i] [-o <n>] [-p] [-s] [-t] [<input> [<output>]]\n"); printf(" -C show PCRE compile-time options and exit\n"); - printf(" -d debug: show compiled code; implies -i\n" - " -i show information about compiled pattern\n" + printf(" -d debug: show compiled code; implies -i\n"); + printf(" -dfa force DFA matching for all subjects\n"); + printf(" -i show information about compiled pattern\n" " -m output memory used information\n" " -o <n> set size of offsets vector to <n>\n"); #if !defined NOPOSIX @@ -567,7 +511,8 @@ while (argc > 1 && argv[op][0] == '-') #endif printf(" -s output store (memory) used information\n" " -t time compilation and execution\n"); - return 1; + yield = 1; + goto EXIT; } op++; argc--; @@ -581,7 +526,8 @@ if (offsets == NULL) { printf("** Failed to get %d bytes of memory for offsets vector\n", size_offsets_max * sizeof(int)); - return 1; + yield = 1; + goto EXIT; } /* Sort out the input and output files */ @@ -592,7 +538,8 @@ if (argc > 1) if (infile == NULL) { printf("** Failed to open %s\n", argv[op]); - return 1; + yield = 1; + goto EXIT; } } @@ -602,7 +549,8 @@ if (argc > 2) if (outfile == NULL) { printf("** Failed to open %s\n", argv[op+1]); - return 1; + yield = 1; + goto EXIT; } } @@ -802,6 +750,7 @@ while (!done) { switch (*pp++) { + case 'f': options |= PCRE_FIRSTLINE; break; case 'g': do_g = 1; break; case 'i': options |= PCRE_CASELESS; break; case 'm': options |= PCRE_MULTILINE; break; @@ -831,7 +780,8 @@ while (!done) case 'L': ppp = pp; - while (*ppp != '\n' && *ppp != ' ') ppp++; + /* The '\r' test here is so that it works on Windows */ + while (*ppp != '\n' && *ppp != '\r' && *ppp != ' ') ppp++; *ppp = 0; if (setlocale(LC_CTYPE, (const char *)pp) == NULL) { @@ -849,7 +799,10 @@ while (!done) *pp = 0; break; - case '\n': case ' ': break; + case '\r': /* So that it works in Windows */ + case '\n': + case ' ': + break; default: fprintf(outfile, "** Unknown option '%c'\n", pp[-1]); @@ -869,6 +822,7 @@ while (!done) if ((options & PCRE_CASELESS) != 0) cflags |= REG_ICASE; if ((options & PCRE_MULTILINE) != 0) cflags |= REG_NEWLINE; + if ((options & PCRE_DOTALL) != 0) cflags |= REG_DOTALL; rc = regcomp(&preg, (char *)p, cflags); /* Compilation failed; go back for another re, skipping to blank line @@ -1016,7 +970,7 @@ while (!done) if (do_debug) { fprintf(outfile, "------------------------------------------------------------------\n"); - print_internals(re, outfile); + _pcre_printint(re, outfile); } new_info(re, NULL, PCRE_INFO_OPTIONS, &get_options); @@ -1049,7 +1003,7 @@ while (!done) if (size != regex_gotten_store) fprintf(outfile, "Size disagreement: pcre_fullinfo=%d call to malloc for %d\n", - size, regex_gotten_store); + (int)size, (int)regex_gotten_store); fprintf(outfile, "Capturing subpattern count = %d\n", count); if (backrefmax > 0) @@ -1080,11 +1034,12 @@ while (!done) fprintf(outfile, "Partial matching not supported\n"); if (get_options == 0) fprintf(outfile, "No options\n"); - else fprintf(outfile, "Options:%s%s%s%s%s%s%s%s%s%s\n", + else fprintf(outfile, "Options:%s%s%s%s%s%s%s%s%s%s%s\n", ((get_options & PCRE_ANCHORED) != 0)? " anchored" : "", ((get_options & PCRE_CASELESS) != 0)? " caseless" : "", ((get_options & PCRE_EXTENDED) != 0)? " extended" : "", ((get_options & PCRE_MULTILINE) != 0)? " multiline" : "", + ((get_options & PCRE_FIRSTLINE) != 0)? " firstline" : "", ((get_options & PCRE_DOTALL) != 0)? " dotall" : "", ((get_options & PCRE_DOLLAR_ENDONLY) != 0)? " dollar_endonly" : "", ((get_options & PCRE_EXTRA) != 0)? " extra" : "", @@ -1222,6 +1177,10 @@ while (!done) } fclose(f); } + + new_free(re); + if (extra != NULL) new_free(extra); + if (tables != NULL) new_free((void *)tables); continue; /* With next regex */ } } /* End of non-POSIX compile */ @@ -1244,6 +1203,7 @@ while (!done) int gmatched = 0; int start_offset = 0; int g_notempty = 0; + int use_dfa = 0; options = 0; @@ -1309,7 +1269,7 @@ while (!done) { unsigned char buff8[8]; int ii, utn; - utn = ord2utf8(c, buff8); + utn = _pcre_ord2utf8(c, buff8); for (ii = 0; ii < utn - 1; ii++) *q++ = buff8[ii]; c = buff8[ii]; /* Last byte */ p = pt + 1; @@ -1397,6 +1357,17 @@ while (!done) } continue; + case 'D': + if (posix || do_posix) + printf("** Can't use dfa matching in POSIX mode: \\D ignored\n"); + else + use_dfa = 1; + continue; + + case 'F': + options |= PCRE_DFA_SHORTEST; + continue; + case 'G': if (isdigit(*p)) { @@ -1439,7 +1410,8 @@ while (!done) { printf("** Failed to get %d bytes of memory for offsets vector\n", size_offsets_max * sizeof(int)); - return 1; + yield = 1; + goto EXIT; } } use_size_offsets = n; @@ -1450,6 +1422,10 @@ while (!done) options |= PCRE_PARTIAL; continue; + case 'R': + options |= PCRE_DFA_RESTART; + continue; + case 'S': show_malloc = 1; continue; @@ -1467,6 +1443,12 @@ while (!done) *q = 0; len = q - dbuffer; + if ((all_use_dfa || use_dfa) && find_match_limit) + { + printf("**Match limit not relevant for DFA matching: ignored\n"); + find_match_limit = 0; + } + /* Handle matching via the POSIX interface, which does not support timing or playing with the match limit or callout data. */ @@ -1524,9 +1506,21 @@ while (!done) register int i; clock_t time_taken; clock_t start_time = clock(); + + if (all_use_dfa || use_dfa) + { + int workspace[1000]; + for (i = 0; i < LOOPREPEAT; i++) + count = pcre_dfa_exec(re, NULL, (char *)bptr, len, start_offset, + options | g_notempty, use_offsets, use_size_offsets, workspace, + sizeof(workspace)/sizeof(int)); + } + else + for (i = 0; i < LOOPREPEAT; i++) count = pcre_exec(re, extra, (char *)bptr, len, start_offset, options | g_notempty, use_offsets, use_size_offsets); + time_taken = clock() - start_time; fprintf(outfile, "Execute time %.3f milliseconds\n", (((double)time_taken * 1000.0) / (double)LOOPREPEAT) / @@ -1597,16 +1591,28 @@ while (!done) /* The normal case is just to do the match once, with the default value of match_limit. */ - else + else if (all_use_dfa || use_dfa) { - count = pcre_exec(re, extra, (char *)bptr, len, - start_offset, options | g_notempty, use_offsets, use_size_offsets); + int workspace[1000]; + count = pcre_dfa_exec(re, NULL, (char *)bptr, len, start_offset, + options | g_notempty, use_offsets, use_size_offsets, workspace, + sizeof(workspace)/sizeof(int)); + if (count == 0) + { + fprintf(outfile, "Matched, but too many subsidiary matches\n"); + count = use_size_offsets/2; + } } - if (count == 0) + else { - fprintf(outfile, "Matched, but too many substrings\n"); - count = use_size_offsets/3; + count = pcre_exec(re, extra, (char *)bptr, len, + start_offset, options | g_notempty, use_offsets, use_size_offsets); + if (count == 0) + { + fprintf(outfile, "Matched, but too many substrings\n"); + count = use_size_offsets/3; + } } /* Matched */ @@ -1692,7 +1698,11 @@ while (!done) else if (count == PCRE_ERROR_PARTIAL) { - fprintf(outfile, "Partial match\n"); + fprintf(outfile, "Partial match"); + if ((all_use_dfa || use_dfa) && use_size_offsets > 2) + fprintf(outfile, ": %.*s", use_offsets[1] - use_offsets[0], + bptr + use_offsets[0]); + fprintf(outfile, "\n"); break; /* Out of the /g loop */ } @@ -1770,17 +1780,28 @@ while (!done) if (posix || do_posix) regfree(&preg); #endif - if (re != NULL) free(re); - if (extra != NULL) free(extra); + if (re != NULL) new_free(re); + if (extra != NULL) new_free(extra); if (tables != NULL) { - free((void *)tables); + new_free((void *)tables); setlocale(LC_CTYPE, "C"); } } if (infile == stdin) fprintf(outfile, "\n"); -return 0; + +EXIT: + +if (infile != NULL && infile != stdin) fclose(infile); +if (outfile != NULL && outfile != stdout) fclose(outfile); + +free(buffer); +free(dbuffer); +free(pbuffer); +free(offsets); + +return yield; } -/* End */ +/* End of pcretest.c */ diff --git a/testdata/grepinput b/testdata/grepinput new file mode 100644 index 0000000..2a89c2e --- /dev/null +++ b/testdata/grepinput @@ -0,0 +1,588 @@ +This is a file of miscellaneous text that is used as test data for checking +that the pcregrep command is working correctly. The file must be more than 24K +long so that it needs more than a single read() call to process it. New +features should be added at the end, because some of the tests involve the +output of line numbers, and we don't want these to change. + +PATTERN at the start of a line. +In the middle of a line, PATTERN appears. + +This pattern is in lower case. + +Here follows a whole lot of stuff that makes the file over 24K long. + +------------------------------------------------------------------------------- +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. + +The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the +lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox +jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick +brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. +------------------------------------------------------------------------------- + +Check up on PATTERN near the end. + +This is the last line of this file. diff --git a/testdata/grepinputx b/testdata/grepinputx new file mode 100644 index 0000000..aebba02 --- /dev/null +++ b/testdata/grepinputx @@ -0,0 +1,42 @@ +This is a second file of input for the pcregrep tests. + +Here is the pattern again. + +Pattern +That time it was on a line by itself. + +To pat or not to pat, that is the question. + +complete pair +of lines + +That was a complete pair +of lines all by themselves. + +complete pair +of lines + +And there they were again, to check line numbers. + +one +two +three +four +five +six +seven +eight +nine +ten +eleven +twelve +thirteen +fourteen +fifteen +sixteen +seventeen +eighteen +nineteen +twenty + +This is the last line of this file. diff --git a/testdata/greplist b/testdata/greplist new file mode 100644 index 0000000..bb2705e --- /dev/null +++ b/testdata/greplist @@ -0,0 +1,4 @@ +This is a file of patterns for testing the -f option. Don't include any blank +lines because they will match everything! +pattern +line by itself diff --git a/testdata/grepoutput b/testdata/grepoutput new file mode 100644 index 0000000..e72bdff --- /dev/null +++ b/testdata/grepoutput @@ -0,0 +1,336 @@ +pcregrep version 4.0 07-Jun-2005 using PCRE version 6.0 07-Jun-2005 +---------------------------- Test 1 ------------------------------ +PATTERN at the start of a line. +In the middle of a line, PATTERN appears. +Check up on PATTERN near the end. +---------------------------- Test 2 ------------------------------ +PATTERN at the start of a line. +---------------------------- Test 3 ------------------------------ +7:PATTERN at the start of a line. +8:In the middle of a line, PATTERN appears. +10:This pattern is in lower case. +586:Check up on PATTERN near the end. +---------------------------- Test 4 ------------------------------ +4 +---------------------------- Test 5 ------------------------------ +./testdata/grepinput:7:PATTERN at the start of a line. +./testdata/grepinput:8:In the middle of a line, PATTERN appears. +./testdata/grepinput:10:This pattern is in lower case. +./testdata/grepinput:586:Check up on PATTERN near the end. +-- +./testdata/grepinputx:3:Here is the pattern again. +./testdata/grepinputx:5:Pattern +---------------------------- Test 6 ------------------------------ +7:PATTERN at the start of a line. +8:In the middle of a line, PATTERN appears. +10:This pattern is in lower case. +586:Check up on PATTERN near the end. +-- +3:Here is the pattern again. +5:Pattern +---------------------------- Test 7 ------------------------------ +./testdata/grepinput +./testdata/grepinputx +---------------------------- Test 8 ------------------------------ +./testdata/grepinput +---------------------------- Test 9 ------------------------------ +RC=0 +---------------------------- Test 10 ----------------------------- +RC=1 +---------------------------- Test 11 ----------------------------- +1:This is a second file of input for the pcregrep tests. +2: +4: +5:Pattern +6:That time it was on a line by itself. +7: +8:To pat or not to pat, that is the question. +9: +10:complete pair +11:of lines +12: +13:That was a complete pair +14:of lines all by themselves. +15: +16:complete pair +17:of lines +18: +19:And there they were again, to check line numbers. +20: +21:one +22:two +23:three +24:four +25:five +26:six +27:seven +28:eight +29:nine +30:ten +31:eleven +32:twelve +33:thirteen +34:fourteen +35:fifteen +36:sixteen +37:seventeen +38:eighteen +39:nineteen +40:twenty +41: +42:This is the last line of this file. +---------------------------- Test 12 ----------------------------- +Pattern +---------------------------- Test 13 ----------------------------- +Here is the pattern again. +That time it was on a line by itself. +---------------------------- Test 14 ----------------------------- +./testdata/grepinputx:To pat or not to pat, that is the question. +---------------------------- Test 15 ----------------------------- +pcregrep: Error in regex at offset 4: nothing to repeat +---------------------------- Test 16 ----------------------------- +pcregrep: Failed to open ./testdata/nonexistfile: No such file or directory +---------------------------- Test 17 ----------------------------- +features should be added at the end, because some of the tests involve the +output of line numbers, and we don't want these to change. +---------------------------- Test 18 ----------------------------- +4:features should be added at the end, because some of the tests involve the +output of line numbers, and we don't want these to change. +583:brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. +------------------------------------------------------------------------------- +---------------------------- Test 19 ----------------------------- +Pattern +---------------------------- Test 20 ----------------------------- +10:complete pair +of lines +16:complete pair +of lines +---------------------------- Test 21 ----------------------------- +24:four +25-five +26-six +27-seven +-- +34:fourteen +35-fifteen +36-sixteen +37-seventeen +---------------------------- Test 22 ----------------------------- +20-one +21-two +22-three +24:four +-- +30-eleven +31-twelve +32-thirteen +34:fourteen +---------------------------- Test 23 ----------------------------- +one +two +three +four +five +six +seven +-- +eleven +twelve +thirteen +fourteen +fifteen +sixteen +seventeen +---------------------------- Test 24 ----------------------------- +four +five +six +seven +eight +nine +ten +eleven +twelve +thirteen +fourteen +fifteen +sixteen +seventeen +eighteen +nineteen +twenty + +This is the last line of this file. +---------------------------- Test 25 ----------------------------- +14- +15-complete pair +16-of lines +17- +18-And there they were again, to check line numbers. +19- +20-one +21-two +22-three +24:four +25-five +26-six +27-seven +28-eight +29-nine +30-ten +31-eleven +32-twelve +33-thirteen +34:fourteen +---------------------------- Test 26 ----------------------------- + +complete pair +of lines + +And there they were again, to check line numbers. + +one +two +three +four +five +six +seven +eight +nine +ten +eleven +twelve +thirteen +fourteen +fifteen +sixteen +seventeen +eighteen +nineteen +twenty + +This is the last line of this file. +---------------------------- Test 27 ----------------------------- +four +five +six +seven +eight +nine +ten +eleven +twelve +thirteen +fourteen +fifteen +sixteen +seventeen +eighteen +nineteen +twenty + +This is the last line of this file. +---------------------------- Test 28 ----------------------------- +13-of lines all by themselves. +14- +15-complete pair +16-of lines +17- +18-And there they were again, to check line numbers. +19- +20-one +21-two +22-three +24:four +25-five +26-six +27-seven +28-eight +29-nine +30-ten +31-eleven +32-twelve +33-thirteen +34:fourteen +---------------------------- Test 29 ----------------------------- +of lines all by themselves. + +complete pair +of lines + +And there they were again, to check line numbers. + +one +two +three +four +five +six +seven +eight +nine +ten +eleven +twelve +thirteen +fourteen +fifteen +sixteen +seventeen +eighteen +nineteen +twenty + +This is the last line of this file. +---------------------------- Test 30 ----------------------------- +./testdata/grepinput-3-features should be added at the end, because some of the tests involve the +./testdata/grepinput-4-output of line numbers, and we don't want these to change. +./testdata/grepinput-5- +./testdata/grepinput:7:PATTERN at the start of a line. +./testdata/grepinput:8:In the middle of a line, PATTERN appears. +./testdata/grepinput-9- +./testdata/grepinput:10:This pattern is in lower case. +-- +./testdata/grepinput-582-brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. +./testdata/grepinput-583-------------------------------------------------------------------------------- +./testdata/grepinput-584- +./testdata/grepinput:586:Check up on PATTERN near the end. +-- +./testdata/grepinputx-1-This is a second file of input for the pcregrep tests. +./testdata/grepinputx-2- +./testdata/grepinputx:3:Here is the pattern again. +./testdata/grepinputx-4- +./testdata/grepinputx:5:Pattern +---------------------------- Test 31 ----------------------------- +./testdata/grepinput:7:PATTERN at the start of a line. +./testdata/grepinput:8:In the middle of a line, PATTERN appears. +./testdata/grepinput-9- +./testdata/grepinput:10:This pattern is in lower case. +./testdata/grepinput-11- +./testdata/grepinput-12-Here follows a whole lot of stuff that makes the file over 24K long. +./testdata/grepinput-13- +-- +./testdata/grepinput:586:Check up on PATTERN near the end. +./testdata/grepinput-587- +./testdata/grepinput-588-This is the last line of this file. +-- +./testdata/grepinputx:3:Here is the pattern again. +./testdata/grepinputx-4- +./testdata/grepinputx:5:Pattern +./testdata/grepinputx-6-That time it was on a line by itself. +./testdata/grepinputx-7- +./testdata/grepinputx-8-To pat or not to pat, that is the question. +---------------------------- Test 32 ----------------------------- +./testdata/grepinputx +---------------------------- Test 33 ----------------------------- +pcregrep: Failed to open ./testdata/grepnonexist: No such file or directory +RC=2 +---------------------------- Test 34 ----------------------------- +RC=2 +---------------------------- Test 35 ----------------------------- +./testdata/grepinputx +RC=0 +---------------------------- Test 36 ----------------------------- +./testdata/grepinputx +RC=0 diff --git a/testdata/testinput1 b/testdata/testinput1 index c4b99c6..c1a05eb 100644 --- a/testdata/testinput1 +++ b/testdata/testinput1 @@ -3838,4 +3838,12 @@ /(?<=Z)X./ \x84XAZXB +/ab cd (?x) de fg/ + ab cd defg + +/ab cd(?x) de fg/ + ab cddefg + ** Failers + abcddefg + / End of testinput1 / diff --git a/testdata/testinput2 b/testdata/testinput2 index d118daf..f59ad22 100644 --- a/testdata/testinput2 +++ b/testdata/testinput2 @@ -1069,6 +1069,12 @@ /(?<=(abc)(?C))xyz/ abcxyz\C+ +/a(b+)(c*)(?C1)/ + abbbbbccc\C*1 + +/a(b+?)(c*?)(?C1)/ + abbbbbccc\C*1 + /(?C)abc/ /(?C)^abc/ @@ -1393,4 +1399,29 @@ /^a/IF +/line\nbreak/ + this is a line\nbreak + line one\nthis is a line\nbreak in the second line + +/line\nbreak/f + this is a line\nbreak + ** Failers + line one\nthis is a line\nbreak in the second line + +/line\nbreak/mf + this is a line\nbreak + ** Failers + line one\nthis is a line\nbreak in the second line + +/ab.cd/P + ab-cd + ab=cd + ** Failers + ab\ncd + +/ab.cd/Ps + ab-cd + ab=cd + ab\ncd + / End of testinput2 / diff --git a/testdata/testinput7 b/testdata/testinput7 new file mode 100644 index 0000000..4dd6037 --- /dev/null +++ b/testdata/testinput7 @@ -0,0 +1,4013 @@ +/abc/ + abc + +/ab*c/ + abc + abbbbc + ac + +/ab+c/ + abc + abbbbbbc + *** Failers + ac + ab + +/a*/ + a + aaaaaaaaaaaaaaaaa + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\F + +/(a|abcd|african)/ + a + abcd + african + +/^abc/ + abcdef + *** Failers + xyzabc + xyz\nabc + +/^abc/m + abcdef + xyz\nabc + *** Failers + xyzabc + +/\Aabc/ + abcdef + *** Failers + xyzabc + xyz\nabc + +/\Aabc/m + abcdef + *** Failers + xyzabc + xyz\nabc + +/\Gabc/ + abcdef + xyzabc\>3 + *** Failers + xyzabc + xyzabc\>2 + +/x\dy\Dz/ + x9yzz + x0y+z + *** Failers + xyz + xxy0z + +/x\sy\Sz/ + x yzz + x y+z + *** Failers + xyz + xxyyz + +/x\wy\Wz/ + xxy+z + *** Failers + xxy0z + x+y+z + +/x.y/ + x+y + x-y + *** Failers + x\ny + +/x.y/s + x+y + x-y + x\ny + +/(a.b(?s)c.d|x.y)p.q/ + a+bc+dp+q + a+bc\ndp+q + x\nyp+q + *** Failers + a\nbc\ndp+q + a+bc\ndp\nq + x\nyp\nq + +/a\d\z/ + ba0 + *** Failers + ba0\n + ba0\ncd + +/a\d\z/m + ba0 + *** Failers + ba0\n + ba0\ncd + +/a\d\Z/ + ba0 + ba0\n + *** Failers + ba0\ncd + +/a\d\Z/m + ba0 + ba0\n + *** Failers + ba0\ncd + +/a\d$/ + ba0 + ba0\n + *** Failers + ba0\ncd + +/a\d$/m + ba0 + ba0\n + ba0\ncd + *** Failers + +/abc/i + abc + aBc + ABC + +/[^a]/ + abcd + +/ab?\w/ + abz + abbz + azz + +/x{0,3}yz/ + ayzq + axyzq + axxyz + axxxyzq + axxxxyzq + *** Failers + ax + axx + +/x{3}yz/ + axxxyzq + axxxxyzq + *** Failers + ax + axx + ayzq + axyzq + axxyz + +/x{2,3}yz/ + axxyz + axxxyzq + axxxxyzq + *** Failers + ax + axx + ayzq + axyzq + +/[^a]+/ + bac + bcdefax + *** Failers + aaaaa + +/[^a]*/ + bac + bcdefax + *** Failers + aaaaa + +/[^a]{3,5}/ + xyz + awxyza + abcdefa + abcdefghijk + *** Failers + axya + axa + aaaaa + +/\d*/ + 1234b567 + xyz + +/\D*/ + a1234b567 + xyz + +/\d+/ + ab1234c56 + *** Failers + xyz + +/\D+/ + ab123c56 + *** Failers + 789 + +/\d?A/ + 045ABC + ABC + *** Failers + XYZ + +/\D?A/ + ABC + BAC + 9ABC + *** Failers + +/a+/ + aaaa + +/^.*xyz/ + xyz + ggggggggxyz + +/^.+xyz/ + abcdxyz + axyz + *** Failers + xyz + +/^.?xyz/ + xyz + cxyz + +/^\d{2,3}X/ + 12X + 123X + *** Failers + X + 1X + 1234X + +/^[abcd]\d/ + a45 + b93 + c99z + d04 + *** Failers + e45 + abcd + abcd1234 + 1234 + +/^[abcd]*\d/ + a45 + b93 + c99z + d04 + abcd1234 + 1234 + *** Failers + e45 + abcd + +/^[abcd]+\d/ + a45 + b93 + c99z + d04 + abcd1234 + *** Failers + 1234 + e45 + abcd + +/^a+X/ + aX + aaX + +/^[abcd]?\d/ + a45 + b93 + c99z + d04 + 1234 + *** Failers + abcd1234 + e45 + +/^[abcd]{2,3}\d/ + ab45 + bcd93 + *** Failers + 1234 + a36 + abcd1234 + ee45 + +/^(abc)*\d/ + abc45 + abcabcabc45 + 42xyz + *** Failers + +/^(abc)+\d/ + abc45 + abcabcabc45 + *** Failers + 42xyz + +/^(abc)?\d/ + abc45 + 42xyz + *** Failers + abcabcabc45 + +/^(abc){2,3}\d/ + abcabc45 + abcabcabc45 + *** Failers + abcabcabcabc45 + abc45 + 42xyz + +/1(abc|xyz)2(?1)3/ + 1abc2abc3456 + 1abc2xyz3456 + +/^(a*\w|ab)=(a*\w|ab)/ + ab=ab + +/^(a*\w|ab)=(?1)/ + ab=ab + +/^([^()]|\((?1)*\))*$/ + abc + a(b)c + a(b(c))d + *** Failers) + a(b(c)d + +/^>abc>([^()]|\((?1)*\))*<xyz<$/ + >abc>123<xyz< + >abc>1(2)3<xyz< + >abc>(1(2)3)<xyz< + +/^(?>a*)\d/ + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa9876 + *** Failers + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/< (?: (?(R) \d++ | [^<>]*+) | (?R)) * >/x + <> + <abcd> + <abc <123> hij> + <abc <def> hij> + <abc<>def> + <abc<> + *** Failers + <abc + +/^(?(?=abc)\w{3}:|\d\d)$/ + abc: + 12 + *** Failers + 123 + xyz + +/^(?(?!abc)\d\d|\w{3}:)$/ + abc: + 12 + *** Failers + 123 + xyz + +/^(?=abc)\w{5}:$/ + abcde: + *** Failers + abc.. + 123 + vwxyz + +/^(?!abc)\d\d$/ + 12 + *** Failers + abcde: + abc.. + 123 + vwxyz + +/(?<=abc|xy)123/ + abc12345 + wxy123z + *** Failers + 123abc + +/(?<!abc|xy)123/ + 123abc + mno123456 + *** Failers + abc12345 + wxy123z + +/abc(?C1)xyz/ + abcxyz + 123abcxyz999 + +/(ab|cd){3,4}/C + ababab + abcdabcd + abcdcdcdcdcd + +/^abc/ + abcdef + *** Failers + abcdef\B + +/^(a*|xyz)/ + bcd + aaabcd + xyz + xyz\N + *** Failers + bcd\N + +/xyz$/ + xyz + xyz\n + *** Failers + xyz\Z + xyz\n\Z + +/xyz$/m + xyz + xyz\n + abcxyz\npqr + abcxyz\npqr\Z + xyz\n\Z + *** Failers + xyz\Z + +/\Gabc/ + abcdef + defabcxyz\>3 + *** Failers + defabcxyz + +/^abcdef/ + ab\P + abcde\P + abcdef\P + *** Failers + abx\P + +/^a{2,4}\d+z/ + a\P + aa\P + aa2\P + aaa\P + aaa23\P + aaaa12345\P + aa0z\P + aaaa4444444444444z\P + *** Failers + az\P + aaaaa\P + a56\P + +/^abcdef/ + abc\P + def\R + +/(?<=foo)bar/ + xyzfo\P + foob\R\P\>2 + foobar...\R\P\>4 + xyzfo\P + foobar\R\>2 + *** Failers + xyzfo\P + obar\R + +/(ab*(cd|ef))+X/ + adfadadaklhlkalkajhlkjahdfasdfasdfladsfjkj\P\Z + lkjhlkjhlkjhlkjhabbbbbbcdaefabbbbbbbefa\P\B\Z + cdabbbbbbbb\P\R\B\Z + efabbbbbbbbbbbbbbbb\P\R\B\Z + bbbbbbbbbbbbcdXyasdfadf\P\R\B\Z + +/(a|b)/SF>testsavedregex +<testsavedregex + abc + ** Failers + def + +/the quick brown fox/ + the quick brown fox + The quick brown FOX + What do you know about the quick brown fox? + What do you know about THE QUICK BROWN FOX? + +/The quick brown fox/i + the quick brown fox + The quick brown FOX + What do you know about the quick brown fox? + What do you know about THE QUICK BROWN FOX? + +/abcd\t\n\r\f\a\e\071\x3b\$\\\?caxyz/ + abcd\t\n\r\f\a\e9;\$\\?caxyz + +/a*abc?xyz+pqr{3}ab{2,}xy{4,5}pq{0,6}AB{0,}zz/ + abxyzpqrrrabbxyyyypqAzz + abxyzpqrrrabbxyyyypqAzz + aabxyzpqrrrabbxyyyypqAzz + aaabxyzpqrrrabbxyyyypqAzz + aaaabxyzpqrrrabbxyyyypqAzz + abcxyzpqrrrabbxyyyypqAzz + aabcxyzpqrrrabbxyyyypqAzz + aaabcxyzpqrrrabbxyyyypAzz + aaabcxyzpqrrrabbxyyyypqAzz + aaabcxyzpqrrrabbxyyyypqqAzz + aaabcxyzpqrrrabbxyyyypqqqAzz + aaabcxyzpqrrrabbxyyyypqqqqAzz + aaabcxyzpqrrrabbxyyyypqqqqqAzz + aaabcxyzpqrrrabbxyyyypqqqqqqAzz + aaaabcxyzpqrrrabbxyyyypqAzz + abxyzzpqrrrabbxyyyypqAzz + aabxyzzzpqrrrabbxyyyypqAzz + aaabxyzzzzpqrrrabbxyyyypqAzz + aaaabxyzzzzpqrrrabbxyyyypqAzz + abcxyzzpqrrrabbxyyyypqAzz + aabcxyzzzpqrrrabbxyyyypqAzz + aaabcxyzzzzpqrrrabbxyyyypqAzz + aaaabcxyzzzzpqrrrabbxyyyypqAzz + aaaabcxyzzzzpqrrrabbbxyyyypqAzz + aaaabcxyzzzzpqrrrabbbxyyyyypqAzz + aaabcxyzpqrrrabbxyyyypABzz + aaabcxyzpqrrrabbxyyyypABBzz + >>>aaabxyzpqrrrabbxyyyypqAzz + >aaaabxyzpqrrrabbxyyyypqAzz + >>>>abcxyzpqrrrabbxyyyypqAzz + *** Failers + abxyzpqrrabbxyyyypqAzz + abxyzpqrrrrabbxyyyypqAzz + abxyzpqrrrabxyyyypqAzz + aaaabcxyzzzzpqrrrabbbxyyyyyypqAzz + aaaabcxyzzzzpqrrrabbbxyyypqAzz + aaabcxyzpqrrrabbxyyyypqqqqqqqAzz + +/^(abc){1,2}zz/ + abczz + abcabczz + *** Failers + zz + abcabcabczz + >>abczz + +/^(b+?|a){1,2}?c/ + bc + bbc + bbbc + bac + bbac + aac + abbbbbbbbbbbc + bbbbbbbbbbbac + *** Failers + aaac + abbbbbbbbbbbac + +/^(b+|a){1,2}c/ + bc + bbc + bbbc + bac + bbac + aac + abbbbbbbbbbbc + bbbbbbbbbbbac + *** Failers + aaac + abbbbbbbbbbbac + +/^(b+|a){1,2}?bc/ + bbc + +/^(b*|ba){1,2}?bc/ + babc + bbabc + bababc + *** Failers + bababbc + babababc + +/^(ba|b*){1,2}?bc/ + babc + bbabc + bababc + *** Failers + bababbc + babababc + +/^\ca\cA\c[\c{\c:/ + \x01\x01\e;z + +/^[ab\]cde]/ + athing + bthing + ]thing + cthing + dthing + ething + *** Failers + fthing + [thing + \\thing + +/^[]cde]/ + ]thing + cthing + dthing + ething + *** Failers + athing + fthing + +/^[^ab\]cde]/ + fthing + [thing + \\thing + *** Failers + athing + bthing + ]thing + cthing + dthing + ething + +/^[^]cde]/ + athing + fthing + *** Failers + ]thing + cthing + dthing + ething + +/^\/ + + +/^ÿ/ + ÿ + +/^[0-9]+$/ + 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 100 + *** Failers + abc + +/^.*nter/ + enter + inter + uponter + +/^xxx[0-9]+$/ + xxx0 + xxx1234 + *** Failers + xxx + +/^.+[0-9][0-9][0-9]$/ + x123 + xx123 + 123456 + *** Failers + 123 + x1234 + +/^.+?[0-9][0-9][0-9]$/ + x123 + xx123 + 123456 + *** Failers + 123 + x1234 + +/^([^!]+)!(.+)=apquxz\.ixr\.zzz\.ac\.uk$/ + abc!pqr=apquxz.ixr.zzz.ac.uk + *** Failers + !pqr=apquxz.ixr.zzz.ac.uk + abc!=apquxz.ixr.zzz.ac.uk + abc!pqr=apquxz:ixr.zzz.ac.uk + abc!pqr=apquxz.ixr.zzz.ac.ukk + +/:/ + Well, we need a colon: somewhere + *** Fail if we don't + +/([\da-f:]+)$/i + 0abc + abc + fed + E + :: + 5f03:12C0::932e + fed def + Any old stuff + *** Failers + 0zzz + gzzz + fed\x20 + Any old rubbish + +/^.*\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/ + .1.2.3 + A.12.123.0 + *** Failers + .1.2.3333 + 1.2.3 + 1234.2.3 + +/^(\d+)\s+IN\s+SOA\s+(\S+)\s+(\S+)\s*\(\s*$/ + 1 IN SOA non-sp1 non-sp2( + 1 IN SOA non-sp1 non-sp2 ( + *** Failers + 1IN SOA non-sp1 non-sp2( + +/^[a-zA-Z\d][a-zA-Z\d\-]*(\.[a-zA-Z\d][a-zA-z\d\-]*)*\.$/ + a. + Z. + 2. + ab-c.pq-r. + sxk.zzz.ac.uk. + x-.y-. + *** Failers + -abc.peq. + +/^\*\.[a-z]([a-z\-\d]*[a-z\d]+)?(\.[a-z]([a-z\-\d]*[a-z\d]+)?)*$/ + *.a + *.b0-a + *.c3-b.c + *.c-a.b-c + *** Failers + *.0 + *.a- + *.a-b.c- + *.c-a.0-c + +/^(?=ab(de))(abd)(e)/ + abde + +/^(?!(ab)de|x)(abd)(f)/ + abdf + +/^(?=(ab(cd)))(ab)/ + abcd + +/^[\da-f](\.[\da-f])*$/i + a.b.c.d + A.B.C.D + a.b.c.1.2.3.C + +/^\".*\"\s*(;.*)?$/ + \"1234\" + \"abcd\" ; + \"\" ; rhubarb + *** Failers + \"1234\" : things + +/^$/ + \ + *** Failers + +/ ^ a (?# begins with a) b\sc (?# then b c) $ (?# then end)/x + ab c + *** Failers + abc + ab cde + +/(?x) ^ a (?# begins with a) b\sc (?# then b c) $ (?# then end)/ + ab c + *** Failers + abc + ab cde + +/^ a\ b[c ]d $/x + a bcd + a b d + *** Failers + abcd + ab d + +/^(a(b(c)))(d(e(f)))(h(i(j)))(k(l(m)))$/ + abcdefhijklm + +/^(?:a(b(c)))(?:d(e(f)))(?:h(i(j)))(?:k(l(m)))$/ + abcdefhijklm + +/^[\w][\W][\s][\S][\d][\D][\b][\n][\c]][\022]/ + a+ Z0+\x08\n\x1d\x12 + +/^[.^$|()*+?{,}]+/ + .^\$(*+)|{?,?} + +/^a*\w/ + z + az + aaaz + a + aa + aaaa + a+ + aa+ + +/^a*?\w/ + z + az + aaaz + a + aa + aaaa + a+ + aa+ + +/^a+\w/ + az + aaaz + aa + aaaa + aa+ + +/^a+?\w/ + az + aaaz + aa + aaaa + aa+ + +/^\d{8}\w{2,}/ + 1234567890 + 12345678ab + 12345678__ + *** Failers + 1234567 + +/^[aeiou\d]{4,5}$/ + uoie + 1234 + 12345 + aaaaa + *** Failers + 123456 + +/^[aeiou\d]{4,5}?/ + uoie + 1234 + 12345 + aaaaa + 123456 + +/^From +([^ ]+) +[a-zA-Z][a-zA-Z][a-zA-Z] +[a-zA-Z][a-zA-Z][a-zA-Z] +[0-9]?[0-9] +[0-9][0-9]:[0-9][0-9]/ + From abcd Mon Sep 01 12:33:02 1997 + +/^From\s+\S+\s+([a-zA-Z]{3}\s+){2}\d{1,2}\s+\d\d:\d\d/ + From abcd Mon Sep 01 12:33:02 1997 + From abcd Mon Sep 1 12:33:02 1997 + *** Failers + From abcd Sep 01 12:33:02 1997 + +/^12.34/s + 12\n34 + 12\r34 + +/\w+(?=\t)/ + the quick brown\t fox + +/foo(?!bar)(.*)/ + foobar is foolish see? + +/(?:(?!foo)...|^.{0,2})bar(.*)/ + foobar crowbar etc + barrel + 2barrel + A barrel + +/^(\D*)(?=\d)(?!123)/ + abc456 + *** Failers + abc123 + +/^1234(?# test newlines + inside)/ + 1234 + +/^1234 #comment in extended re + /x + 1234 + +/#rhubarb + abcd/x + abcd + +/^abcd#rhubarb/x + abcd + +/(?!^)abc/ + the abc + *** Failers + abc + +/(?=^)abc/ + abc + *** Failers + the abc + +/^[ab]{1,3}(ab*|b)/ + aabbbbb + +/^[ab]{1,3}?(ab*|b)/ + aabbbbb + +/^[ab]{1,3}?(ab*?|b)/ + aabbbbb + +/^[ab]{1,3}(ab*?|b)/ + aabbbbb + +/ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional leading comment +(?: (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address +| # or +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # one word, optionally followed by.... +(?: +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] | # atom and space parts, or... +\( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) | # comments, or... + +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +# quoted strings +)* +< (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # leading < +(?: @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* + +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* , (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +)* # further okay, if led by comma +: # closing colon +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* )? # optional route +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address spec +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* > # trailing > +# name and address +) (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional trailing comment +/x + Alan Other <user\@dom.ain> + <user\@dom.ain> + user\@dom.ain + \"A. Other\" <user.1234\@dom.ain> (a comment) + A. Other <user.1234\@dom.ain> (a comment) + \"/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/\"\@x400-re.lay + A missing angle <user\@some.where + *** Failers + The quick brown fox + +/[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional leading comment +(?: +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# additional words +)* +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +# address +| # or +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +# leading word +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] * # "normal" atoms and or spaces +(?: +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +| +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +) # "special" comment or quoted string +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] * # more "normal" +)* +< +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# < +(?: +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +(?: , +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +)* # additional domains +: +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)? # optional route +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# additional words +)* +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +# address spec +> # > +# name and address +) +/x + Alan Other <user\@dom.ain> + <user\@dom.ain> + user\@dom.ain + \"A. Other\" <user.1234\@dom.ain> (a comment) + A. Other <user.1234\@dom.ain> (a comment) + \"/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/\"\@x400-re.lay + A missing angle <user\@some.where + *** Failers + The quick brown fox + +/abc\0def\00pqr\000xyz\0000AB/ + abc\0def\00pqr\000xyz\0000AB + abc456 abc\0def\00pqr\000xyz\0000ABCDE + +/abc\x0def\x00pqr\x000xyz\x0000AB/ + abc\x0def\x00pqr\x000xyz\x0000AB + abc456 abc\x0def\x00pqr\x000xyz\x0000ABCDE + +/^[\000-\037]/ + \0A + \01B + \037C + +/\0*/ + \0\0\0\0 + +/A\x0{2,3}Z/ + The A\x0\x0Z + An A\0\x0\0Z + *** Failers + A\0Z + A\0\x0\0\x0Z + +/^\s/ + \040abc + \x0cabc + \nabc + \rabc + \tabc + *** Failers + abc + +/^a b +
c/x + abc + +/ab{1,3}bc/ + abbbbc + abbbc + abbc + *** Failers + abc + abbbbbc + +/([^.]*)\.([^:]*):[T ]+(.*)/ + track1.title:TBlah blah blah + +/([^.]*)\.([^:]*):[T ]+(.*)/i + track1.title:TBlah blah blah + +/([^.]*)\.([^:]*):[t ]+(.*)/i + track1.title:TBlah blah blah + +/^[W-c]+$/ + WXY_^abc + *** Failers + wxy + +/^[W-c]+$/i + WXY_^abc + wxy_^ABC + +/^[\x3f-\x5F]+$/i + WXY_^abc + wxy_^ABC + +/^abc$/m + abc + qqq\nabc + abc\nzzz + qqq\nabc\nzzz + +/^abc$/ + abc + *** Failers + qqq\nabc + abc\nzzz + qqq\nabc\nzzz + +/\Aabc\Z/m + abc + abc\n + *** Failers + qqq\nabc + abc\nzzz + qqq\nabc\nzzz + +/\A(.)*\Z/s + abc\ndef + +/\A(.)*\Z/m + *** Failers + abc\ndef + +/(?:b)|(?::+)/ + b::c + c::b + +/[-az]+/ + az- + *** Failers + b + +/[az-]+/ + za- + *** Failers + b + +/[a\-z]+/ + a-z + *** Failers + b + +/[a-z]+/ + abcdxyz + +/[\d-]+/ + 12-34 + *** Failers + aaa + +/[\d-z]+/ + 12-34z + *** Failers + aaa + +/\x5c/ + \\ + +/\x20Z/ + the Zoo + *** Failers + Zulu + +/ab{3cd/ + ab{3cd + +/ab{3,cd/ + ab{3,cd + +/ab{3,4a}cd/ + ab{3,4a}cd + +/{4,5a}bc/ + {4,5a}bc + +/^a.b/ + a\rb + *** Failers + a\nb + +/abc$/ + abc + abc\n + *** Failers + abc\ndef + +/(abc)\123/ + abc\x53 + +/(abc)\223/ + abc\x93 + +/(abc)\323/ + abc\xd3 + +/(abc)\500/ + abc\x40 + abc\100 + +/(abc)\5000/ + abc\x400 + abc\x40\x30 + abc\1000 + abc\100\x30 + abc\100\060 + abc\100\60 + +/abc\81/ + abc\081 + abc\0\x38\x31 + +/abc\91/ + abc\091 + abc\0\x39\x31 + +/(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)\12\123/ + abcdefghijk\12S + +/ab\gdef/ + abgdef + +/a{0}bc/ + bc + +/(a|(bc)){0,0}?xyz/ + xyz + +/abc[\10]de/ + abc\010de + +/abc[\1]de/ + abc\1de + +/(abc)[\1]de/ + abc\1de + +/(?s)a.b/ + a\nb + +/^([^a])([^\b])([^c]*)([^d]{3,4})/ + baNOTccccd + baNOTcccd + baNOTccd + bacccd + *** Failers + anything + b\bc + baccd + +/[^a]/ + Abc + +/[^a]/i + Abc + +/[^a]+/ + AAAaAbc + +/[^a]+/i + AAAaAbc + +/[^a]+/ + bbb\nccc + +/[^k]$/ + abc + *** Failers + abk + +/[^k]{2,3}$/ + abc + kbc + kabc + *** Failers + abk + akb + akk + +/^\d{8,}\@.+[^k]$/ + 12345678\@a.b.c.d + 123456789\@x.y.z + *** Failers + 12345678\@x.y.uk + 1234567\@a.b.c.d + +/[^a]/ + aaaabcd + aaAabcd + +/[^a]/i + aaaabcd + aaAabcd + +/[^az]/ + aaaabcd + aaAabcd + +/[^az]/i + aaaabcd + aaAabcd + +/\000\001\002\003\004\005\006\007\010\011\012\013\014\015\016\017\020\021\022\023\024\025\026\027\030\031\032\033\034\035\036\037\040\041\042\043\044\045\046\047\050\051\052\053\054\055\056\057\060\061\062\063\064\065\066\067\070\071\072\073\074\075\076\077\100\101\102\103\104\105\106\107\110\111\112\113\114\115\116\117\120\121\122\123\124\125\126\127\130\131\132\133\134\135\136\137\140\141\142\143\144\145\146\147\150\151\152\153\154\155\156\157\160\161\162\163\164\165\166\167\170\171\172\173\174\175\176\177\200\201\202\203\204\205\206\207\210\211\212\213\214\215\216\217\220\221\222\223\224\225\226\227\230\231\232\233\234\235\236\237\240\241\242\243\244\245\246\247\250\251\252\253\254\255\256\257\260\261\262\263\264\265\266\267\270\271\272\273\274\275\276\277\300\301\302\303\304\305\306\307\310\311\312\313\314\315\316\317\320\321\322\323\324\325\326\327\330\331\332\333\334\335\336\337\340\341\342\343\344\345\346\347\350\351\352\353\354\355\356\357\360\361\362\363\364\365\366\367\370\371\372\373\374\375\376\377/ + \000\001\002\003\004\005\006\007\010\011\012\013\014\015\016\017\020\021\022\023\024\025\026\027\030\031\032\033\034\035\036\037\040\041\042\043\044\045\046\047\050\051\052\053\054\055\056\057\060\061\062\063\064\065\066\067\070\071\072\073\074\075\076\077\100\101\102\103\104\105\106\107\110\111\112\113\114\115\116\117\120\121\122\123\124\125\126\127\130\131\132\133\134\135\136\137\140\141\142\143\144\145\146\147\150\151\152\153\154\155\156\157\160\161\162\163\164\165\166\167\170\171\172\173\174\175\176\177\200\201\202\203\204\205\206\207\210\211\212\213\214\215\216\217\220\221\222\223\224\225\226\227\230\231\232\233\234\235\236\237\240\241\242\243\244\245\246\247\250\251\252\253\254\255\256\257\260\261\262\263\264\265\266\267\270\271\272\273\274\275\276\277\300\301\302\303\304\305\306\307\310\311\312\313\314\315\316\317\320\321\322\323\324\325\326\327\330\331\332\333\334\335\336\337\340\341\342\343\344\345\346\347\350\351\352\353\354\355\356\357\360\361\362\363\364\365\366\367\370\371\372\373\374\375\376\377 + +/P[^*]TAIRE[^*]{1,6}?LL/ + xxxxxxxxxxxPSTAIREISLLxxxxxxxxx + +/P[^*]TAIRE[^*]{1,}?LL/ + xxxxxxxxxxxPSTAIREISLLxxxxxxxxx + +/(\.\d\d[1-9]?)\d+/ + 1.230003938 + 1.875000282 + 1.235 + +/(\.\d\d((?=0)|\d(?=\d)))/ + 1.230003938 + 1.875000282 + *** Failers + 1.235 + +/a(?)b/ + ab + +/\b(foo)\s+(\w+)/i + Food is on the foo table + +/foo(.*)bar/ + The food is under the bar in the barn. + +/foo(.*?)bar/ + The food is under the bar in the barn. + +/(.*)(\d*)/ + I have 2 numbers: 53147 + +/(.*)(\d+)/ + I have 2 numbers: 53147 + +/(.*?)(\d*)/ + I have 2 numbers: 53147 + +/(.*?)(\d+)/ + I have 2 numbers: 53147 + +/(.*)(\d+)$/ + I have 2 numbers: 53147 + +/(.*?)(\d+)$/ + I have 2 numbers: 53147 + +/(.*)\b(\d+)$/ + I have 2 numbers: 53147 + +/(.*\D)(\d+)$/ + I have 2 numbers: 53147 + +/^\D*(?!123)/ + ABC123 + +/^(\D*)(?=\d)(?!123)/ + ABC445 + *** Failers + ABC123 + +/^[W-]46]/ + W46]789 + -46]789 + *** Failers + Wall + Zebra + 42 + [abcd] + ]abcd[ + +/^[W-\]46]/ + W46]789 + Wall + Zebra + Xylophone + 42 + [abcd] + ]abcd[ + \\backslash + *** Failers + -46]789 + well + +/\d\d\/\d\d\/\d\d\d\d/ + 01/01/2000 + +/word (?:[a-zA-Z0-9]+ ){0,10}otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark otherword + word cat dog elephant mussel cow horse canary baboon snake shark + +/word (?:[a-zA-Z0-9]+ ){0,300}otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark the quick brown fox and the lazy dog and several other words getting close to thirty by now I hope + +/^(a){0,0}/ + bcd + abc + aab + +/^(a){0,1}/ + bcd + abc + aab + +/^(a){0,2}/ + bcd + abc + aab + +/^(a){0,3}/ + bcd + abc + aab + aaa + +/^(a){0,}/ + bcd + abc + aab + aaa + aaaaaaaa + +/^(a){1,1}/ + bcd + abc + aab + +/^(a){1,2}/ + bcd + abc + aab + +/^(a){1,3}/ + bcd + abc + aab + aaa + +/^(a){1,}/ + bcd + abc + aab + aaa + aaaaaaaa + +/.*\.gif/ + borfle\nbib.gif\nno + +/.{0,}\.gif/ + borfle\nbib.gif\nno + +/.*\.gif/m + borfle\nbib.gif\nno + +/.*\.gif/s + borfle\nbib.gif\nno + +/.*\.gif/ms + borfle\nbib.gif\nno + +/.*$/ + borfle\nbib.gif\nno + +/.*$/m + borfle\nbib.gif\nno + +/.*$/s + borfle\nbib.gif\nno + +/.*$/ms + borfle\nbib.gif\nno + +/.*$/ + borfle\nbib.gif\nno\n + +/.*$/m + borfle\nbib.gif\nno\n + +/.*$/s + borfle\nbib.gif\nno\n + +/.*$/ms + borfle\nbib.gif\nno\n + +/(.*X|^B)/ + abcde\n1234Xyz + BarFoo + *** Failers + abcde\nBar + +/(.*X|^B)/m + abcde\n1234Xyz + BarFoo + abcde\nBar + +/(.*X|^B)/s + abcde\n1234Xyz + BarFoo + *** Failers + abcde\nBar + +/(.*X|^B)/ms + abcde\n1234Xyz + BarFoo + abcde\nBar + +/(?s)(.*X|^B)/ + abcde\n1234Xyz + BarFoo + *** Failers + abcde\nBar + +/(?s:.*X|^B)/ + abcde\n1234Xyz + BarFoo + *** Failers + abcde\nBar + +/^.*B/ + **** Failers + abc\nB + +/(?s)^.*B/ + abc\nB + +/(?m)^.*B/ + abc\nB + +/(?ms)^.*B/ + abc\nB + +/(?ms)^B/ + abc\nB + +/(?s)B$/ + B\n + +/^[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]/ + 123456654321 + +/^\d\d\d\d\d\d\d\d\d\d\d\d/ + 123456654321 + +/^[\d][\d][\d][\d][\d][\d][\d][\d][\d][\d][\d][\d]/ + 123456654321 + +/^[abc]{12}/ + abcabcabcabc + +/^[a-c]{12}/ + abcabcabcabc + +/^(a|b|c){12}/ + abcabcabcabc + +/^[abcdefghijklmnopqrstuvwxy0123456789]/ + n + *** Failers + z + +/abcde{0,0}/ + abcd + *** Failers + abce + +/ab[cd]{0,0}e/ + abe + *** Failers + abcde + +/ab(c){0,0}d/ + abd + *** Failers + abcd + +/a(b*)/ + a + ab + abbbb + *** Failers + bbbbb + +/ab\d{0}e/ + abe + *** Failers + ab1e + +/"([^\\"]+|\\.)*"/ + the \"quick\" brown fox + \"the \\\"quick\\\" brown fox\" + +/.*?/g+ + abc + +/\b/g+ + abc + +/\b/+g + abc + +//g + abc + +/<tr([\w\W\s\d][^<>]{0,})><TD([\w\W\s\d][^<>]{0,})>([\d]{0,}\.)(.*)((<BR>([\w\W\s\d][^<>]{0,})|[\s]{0,}))<\/a><\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><\/TR>/is + <TR BGCOLOR='#DBE9E9'><TD align=left valign=top>43.<a href='joblist.cfm?JobID=94 6735&Keyword='>Word Processor<BR>(N-1286)</a></TD><TD align=left valign=top>Lega lstaff.com</TD><TD align=left valign=top>CA - Statewide</TD></TR> + +/a[^a]b/ + acb + a\nb + +/a.b/ + acb + *** Failers + a\nb + +/a[^a]b/s + acb + a\nb + +/a.b/s + acb + a\nb + +/^(b+?|a){1,2}?c/ + bac + bbac + bbbac + bbbbac + bbbbbac + +/^(b+|a){1,2}?c/ + bac + bbac + bbbac + bbbbac + bbbbbac + +/(?!\A)x/m + x\nb\n + a\bx\n + +/\x0{ab}/ + \0{ab} + +/(A|B)*?CD/ + CD + +/(A|B)*CD/ + CD + +/(?<!bar)foo/ + foo + catfood + arfootle + rfoosh + *** Failers + barfoo + towbarfoo + +/\w{3}(?<!bar)foo/ + catfood + *** Failers + foo + barfoo + towbarfoo + +/(?<=(foo)a)bar/ + fooabar + *** Failers + bar + foobbar + +/\Aabc\z/m + abc + *** Failers + abc\n + qqq\nabc + abc\nzzz + qqq\nabc\nzzz + +"(?>.*/)foo" + /this/is/a/very/long/line/in/deed/with/very/many/slashes/in/it/you/see/ + +"(?>.*/)foo" + /this/is/a/very/long/line/in/deed/with/very/many/slashes/in/and/foo + +/(?>(\.\d\d[1-9]?))\d+/ + 1.230003938 + 1.875000282 + *** Failers + 1.235 + +/^((?>\w+)|(?>\s+))*$/ + now is the time for all good men to come to the aid of the party + *** Failers + this is not a line with only words and spaces! + +/(\d+)(\w)/ + 12345a + 12345+ + +/((?>\d+))(\w)/ + 12345a + *** Failers + 12345+ + +/(?>a+)b/ + aaab + +/((?>a+)b)/ + aaab + +/(?>(a+))b/ + aaab + +/(?>b)+/ + aaabbbccc + +/(?>a+|b+|c+)*c/ + aaabbbbccccd + +/(a+|b+|c+)*c/ + aaabbbbccccd + +/((?>[^()]+)|\([^()]*\))+/ + ((abc(ade)ufh()()x + +/\(((?>[^()]+)|\([^()]+\))+\)/ + (abc) + (abc(def)xyz) + *** Failers + ((()aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/a(?-i)b/i + ab + Ab + *** Failers + aB + AB + +/(a (?x)b c)d e/ + a bcd e + *** Failers + a b cd e + abcd e + a bcde + +/(a b(?x)c d (?-x)e f)/ + a bcde f + *** Failers + abcdef + +/(a(?i)b)c/ + abc + aBc + *** Failers + abC + aBC + Abc + ABc + ABC + AbC + +/a(?i:b)c/ + abc + aBc + *** Failers + ABC + abC + aBC + +/a(?i:b)*c/ + aBc + aBBc + *** Failers + aBC + aBBC + +/a(?=b(?i)c)\w\wd/ + abcd + abCd + *** Failers + aBCd + abcD + +/(?s-i:more.*than).*million/i + more than million + more than MILLION + more \n than Million + *** Failers + MORE THAN MILLION + more \n than \n million + +/(?:(?s-i)more.*than).*million/i + more than million + more than MILLION + more \n than Million + *** Failers + MORE THAN MILLION + more \n than \n million + +/(?>a(?i)b+)+c/ + abc + aBbc + aBBc + *** Failers + Abc + abAb + abbC + +/(?=a(?i)b)\w\wc/ + abc + aBc + *** Failers + Ab + abC + aBC + +/(?<=a(?i)b)(\w\w)c/ + abxxc + aBxxc + *** Failers + Abxxc + ABxxc + abxxC + +/^(?(?=abc)\w{3}:|\d\d)$/ + abc: + 12 + *** Failers + 123 + xyz + +/^(?(?!abc)\d\d|\w{3}:)$/ + abc: + 12 + *** Failers + 123 + xyz + +/(?(?<=foo)bar|cat)/ + foobar + cat + fcat + focat + *** Failers + foocat + +/(?(?<!foo)cat|bar)/ + foobar + cat + fcat + focat + *** Failers + foocat + +/(?>a*)*/ + a + aa + aaaa + +/(abc|)+/ + abc + abcabc + abcabcabc + xyz + +/([a]*)*/ + a + aaaaa + +/([ab]*)*/ + a + b + ababab + aaaabcde + bbbb + +/([^a]*)*/ + b + bbbb + aaa + +/([^ab]*)*/ + cccc + abab + +/([a]*?)*/ + a + aaaa + +/([ab]*?)*/ + a + b + abab + baba + +/([^a]*?)*/ + b + bbbb + aaa + +/([^ab]*?)*/ + c + cccc + baba + +/(?>a*)*/ + a + aaabcde + +/((?>a*))*/ + aaaaa + aabbaa + +/((?>a*?))*/ + aaaaa + aabbaa + +/(?(?=[^a-z]+[a-z]) \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) /x + 12-sep-98 + 12-09-98 + *** Failers + sep-12-98 + +/(?i:saturday|sunday)/ + saturday + sunday + Saturday + Sunday + SATURDAY + SUNDAY + SunDay + +/(a(?i)bc|BB)x/ + abcx + aBCx + bbx + BBx + *** Failers + abcX + aBCX + bbX + BBX + +/^([ab](?i)[cd]|[ef])/ + ac + aC + bD + elephant + Europe + frog + France + *** Failers + Africa + +/^(ab|a(?i)[b-c](?m-i)d|x(?i)y|z)/ + ab + aBd + xy + xY + zebra + Zambesi + *** Failers + aCD + XY + +/(?<=foo\n)^bar/m + foo\nbar + *** Failers + bar + baz\nbar + +/(?<=(?<!foo)bar)baz/ + barbaz + barbarbaz + koobarbaz + *** Failers + baz + foobarbaz + +/The following tests are taken from the Perl 5.005 test suite; some of them/ +/are compatible with 5.004, but I'd rather not have to sort them out./ + +/abc/ + abc + xabcy + ababc + *** Failers + xbc + axc + abx + +/ab*c/ + abc + +/ab*bc/ + abc + abbc + abbbbc + +/.{1}/ + abbbbc + +/.{3,4}/ + abbbbc + +/ab{0,}bc/ + abbbbc + +/ab+bc/ + abbc + *** Failers + abc + abq + +/ab{1,}bc/ + +/ab+bc/ + abbbbc + +/ab{1,}bc/ + abbbbc + +/ab{1,3}bc/ + abbbbc + +/ab{3,4}bc/ + abbbbc + +/ab{4,5}bc/ + *** Failers + abq + abbbbc + +/ab?bc/ + abbc + abc + +/ab{0,1}bc/ + abc + +/ab?bc/ + +/ab?c/ + abc + +/ab{0,1}c/ + abc + +/^abc$/ + abc + *** Failers + abbbbc + abcc + +/^abc/ + abcc + +/^abc$/ + +/abc$/ + aabc + *** Failers + aabc + aabcd + +/^/ + abc + +/$/ + abc + +/a.c/ + abc + axc + +/a.*c/ + axyzc + +/a[bc]d/ + abd + *** Failers + axyzd + abc + +/a[b-d]e/ + ace + +/a[b-d]/ + aac + +/a[-b]/ + a- + +/a[b-]/ + a- + +/a]/ + a] + +/a[]]b/ + a]b + +/a[^bc]d/ + aed + *** Failers + abd + abd + +/a[^-b]c/ + adc + +/a[^]b]c/ + adc + *** Failers + a-c + a]c + +/\ba\b/ + a- + -a + -a- + +/\by\b/ + *** Failers + xy + yz + xyz + +/\Ba\B/ + *** Failers + a- + -a + -a- + +/\By\b/ + xy + +/\by\B/ + yz + +/\By\B/ + xyz + +/\w/ + a + +/\W/ + - + *** Failers + - + a + +/a\sb/ + a b + +/a\Sb/ + a-b + *** Failers + a-b + a b + +/\d/ + 1 + +/\D/ + - + *** Failers + - + 1 + +/[\w]/ + a + +/[\W]/ + - + *** Failers + - + a + +/a[\s]b/ + a b + +/a[\S]b/ + a-b + *** Failers + a-b + a b + +/[\d]/ + 1 + +/[\D]/ + - + *** Failers + - + 1 + +/ab|cd/ + abc + abcd + +/()ef/ + def + +/$b/ + +/a\(b/ + a(b + +/a\(*b/ + ab + a((b + +/a\\b/ + a\b + +/((a))/ + abc + +/(a)b(c)/ + abc + +/a+b+c/ + aabbabc + +/a{1,}b{1,}c/ + aabbabc + +/a.+?c/ + abcabc + +/(a+|b)*/ + ab + +/(a+|b){0,}/ + ab + +/(a+|b)+/ + ab + +/(a+|b){1,}/ + ab + +/(a+|b)?/ + ab + +/(a+|b){0,1}/ + ab + +/[^ab]*/ + cde + +/abc/ + *** Failers + b + + +/a*/ + + +/([abc])*d/ + abbbcd + +/([abc])*bcd/ + abcd + +/a|b|c|d|e/ + e + +/(a|b|c|d|e)f/ + ef + +/abcd*efg/ + abcdefg + +/ab*/ + xabyabbbz + xayabbbz + +/(ab|cd)e/ + abcde + +/[abhgefdc]ij/ + hij + +/^(ab|cd)e/ + +/(abc|)ef/ + abcdef + +/(a|b)c*d/ + abcd + +/(ab|ab*)bc/ + abc + +/a([bc]*)c*/ + abc + +/a([bc]*)(c*d)/ + abcd + +/a([bc]+)(c*d)/ + abcd + +/a([bc]*)(c+d)/ + abcd + +/a[bcd]*dcdcde/ + adcdcde + +/a[bcd]+dcdcde/ + *** Failers + abcde + adcdcde + +/(ab|a)b*c/ + abc + +/((a)(b)c)(d)/ + abcd + +/[a-zA-Z_][a-zA-Z0-9_]*/ + alpha + +/^a(bc+|b[eh])g|.h$/ + abh + +/(bc+d$|ef*g.|h?i(j|k))/ + effgz + ij + reffgz + *** Failers + effg + bcdd + +/((((((((((a))))))))))/ + a + +/(((((((((a)))))))))/ + a + +/multiple words of text/ + *** Failers + aa + uh-uh + +/multiple words/ + multiple words, yeah + +/(.*)c(.*)/ + abcde + +/\((.*), (.*)\)/ + (a, b) + +/[k]/ + +/abcd/ + abcd + +/a(bc)d/ + abcd + +/a[-]?c/ + ac + +/abc/i + ABC + XABCY + ABABC + *** Failers + aaxabxbaxbbx + XBC + AXC + ABX + +/ab*c/i + ABC + +/ab*bc/i + ABC + ABBC + +/ab*?bc/i + ABBBBC + +/ab{0,}?bc/i + ABBBBC + +/ab+?bc/i + ABBC + +/ab+bc/i + *** Failers + ABC + ABQ + +/ab{1,}bc/i + +/ab+bc/i + ABBBBC + +/ab{1,}?bc/i + ABBBBC + +/ab{1,3}?bc/i + ABBBBC + +/ab{3,4}?bc/i + ABBBBC + +/ab{4,5}?bc/i + *** Failers + ABQ + ABBBBC + +/ab??bc/i + ABBC + ABC + +/ab{0,1}?bc/i + ABC + +/ab??bc/i + +/ab??c/i + ABC + +/ab{0,1}?c/i + ABC + +/^abc$/i + ABC + *** Failers + ABBBBC + ABCC + +/^abc/i + ABCC + +/^abc$/i + +/abc$/i + AABC + +/^/i + ABC + +/$/i + ABC + +/a.c/i + ABC + AXC + +/a.*?c/i + AXYZC + +/a.*c/i + *** Failers + AABC + AXYZD + +/a[bc]d/i + ABD + +/a[b-d]e/i + ACE + *** Failers + ABC + ABD + +/a[b-d]/i + AAC + +/a[-b]/i + A- + +/a[b-]/i + A- + +/a]/i + A] + +/a[]]b/i + A]B + +/a[^bc]d/i + AED + +/a[^-b]c/i + ADC + *** Failers + ABD + A-C + +/a[^]b]c/i + ADC + +/ab|cd/i + ABC + ABCD + +/()ef/i + DEF + +/$b/i + *** Failers + A]C + B + +/a\(b/i + A(B + +/a\(*b/i + AB + A((B + +/a\\b/i + A\B + +/((a))/i + ABC + +/(a)b(c)/i + ABC + +/a+b+c/i + AABBABC + +/a{1,}b{1,}c/i + AABBABC + +/a.+?c/i + ABCABC + +/a.*?c/i + ABCABC + +/a.{0,5}?c/i + ABCABC + +/(a+|b)*/i + AB + +/(a+|b){0,}/i + AB + +/(a+|b)+/i + AB + +/(a+|b){1,}/i + AB + +/(a+|b)?/i + AB + +/(a+|b){0,1}/i + AB + +/(a+|b){0,1}?/i + AB + +/[^ab]*/i + CDE + +/abc/i + +/a*/i + + +/([abc])*d/i + ABBBCD + +/([abc])*bcd/i + ABCD + +/a|b|c|d|e/i + E + +/(a|b|c|d|e)f/i + EF + +/abcd*efg/i + ABCDEFG + +/ab*/i + XABYABBBZ + XAYABBBZ + +/(ab|cd)e/i + ABCDE + +/[abhgefdc]ij/i + HIJ + +/^(ab|cd)e/i + ABCDE + +/(abc|)ef/i + ABCDEF + +/(a|b)c*d/i + ABCD + +/(ab|ab*)bc/i + ABC + +/a([bc]*)c*/i + ABC + +/a([bc]*)(c*d)/i + ABCD + +/a([bc]+)(c*d)/i + ABCD + +/a([bc]*)(c+d)/i + ABCD + +/a[bcd]*dcdcde/i + ADCDCDE + +/a[bcd]+dcdcde/i + +/(ab|a)b*c/i + ABC + +/((a)(b)c)(d)/i + ABCD + +/[a-zA-Z_][a-zA-Z0-9_]*/i + ALPHA + +/^a(bc+|b[eh])g|.h$/i + ABH + +/(bc+d$|ef*g.|h?i(j|k))/i + EFFGZ + IJ + REFFGZ + *** Failers + ADCDCDE + EFFG + BCDD + +/((((((((((a))))))))))/i + A + +/(((((((((a)))))))))/i + A + +/(?:(?:(?:(?:(?:(?:(?:(?:(?:(a))))))))))/i + A + +/(?:(?:(?:(?:(?:(?:(?:(?:(?:(a|b|c))))))))))/i + C + +/multiple words of text/i + *** Failers + AA + UH-UH + +/multiple words/i + MULTIPLE WORDS, YEAH + +/(.*)c(.*)/i + ABCDE + +/\((.*), (.*)\)/i + (A, B) + +/[k]/i + +/abcd/i + ABCD + +/a(bc)d/i + ABCD + +/a[-]?c/i + AC + +/a(?!b)./ + abad + +/a(?=d)./ + abad + +/a(?=c|d)./ + abad + +/a(?:b|c|d)(.)/ + ace + +/a(?:b|c|d)*(.)/ + ace + +/a(?:b|c|d)+?(.)/ + ace + acdbcdbe + +/a(?:b|c|d)+(.)/ + acdbcdbe + +/a(?:b|c|d){2}(.)/ + acdbcdbe + +/a(?:b|c|d){4,5}(.)/ + acdbcdbe + +/a(?:b|c|d){4,5}?(.)/ + acdbcdbe + +/((foo)|(bar))*/ + foobar + +/a(?:b|c|d){6,7}(.)/ + acdbcdbe + +/a(?:b|c|d){6,7}?(.)/ + acdbcdbe + +/a(?:b|c|d){5,6}(.)/ + acdbcdbe + +/a(?:b|c|d){5,6}?(.)/ + acdbcdbe + +/a(?:b|c|d){5,7}(.)/ + acdbcdbe + +/a(?:b|c|d){5,7}?(.)/ + acdbcdbe + +/a(?:b|(c|e){1,2}?|d)+?(.)/ + ace + +/^(.+)?B/ + AB + +/^([^a-z])|(\^)$/ + . + +/^[<>]&/ + <&OUT + +/(?:(f)(o)(o)|(b)(a)(r))*/ + foobar + +/(?<=a)b/ + ab + *** Failers + cb + b + +/(?<!c)b/ + ab + b + b + +/(?:..)*a/ + aba + +/(?:..)*?a/ + aba + +/^(){3,5}/ + abc + +/^(a+)*ax/ + aax + +/^((a|b)+)*ax/ + aax + +/^((a|bc)+)*ax/ + aax + +/(a|x)*ab/ + cab + +/(a)*ab/ + cab + +/(?:(?i)a)b/ + ab + +/((?i)a)b/ + ab + +/(?:(?i)a)b/ + Ab + +/((?i)a)b/ + Ab + +/(?:(?i)a)b/ + *** Failers + cb + aB + +/((?i)a)b/ + +/(?i:a)b/ + ab + +/((?i:a))b/ + ab + +/(?i:a)b/ + Ab + +/((?i:a))b/ + Ab + +/(?i:a)b/ + *** Failers + aB + aB + +/((?i:a))b/ + +/(?:(?-i)a)b/i + ab + +/((?-i)a)b/i + ab + +/(?:(?-i)a)b/i + aB + +/((?-i)a)b/i + aB + +/(?:(?-i)a)b/i + *** Failers + aB + Ab + +/((?-i)a)b/i + +/(?:(?-i)a)b/i + aB + +/((?-i)a)b/i + aB + +/(?:(?-i)a)b/i + *** Failers + Ab + AB + +/((?-i)a)b/i + +/(?-i:a)b/i + ab + +/((?-i:a))b/i + ab + +/(?-i:a)b/i + aB + +/((?-i:a))b/i + aB + +/(?-i:a)b/i + *** Failers + AB + Ab + +/((?-i:a))b/i + +/(?-i:a)b/i + aB + +/((?-i:a))b/i + aB + +/(?-i:a)b/i + *** Failers + Ab + AB + +/((?-i:a))b/i + +/((?-i:a.))b/i + *** Failers + AB + a\nB + +/((?s-i:a.))b/i + a\nB + +/(?:c|d)(?:)(?:a(?:)(?:b)(?:b(?:))(?:b(?:)(?:b)))/ + cabbbb + +/(?:c|d)(?:)(?:aaaaaaaa(?:)(?:bbbbbbbb)(?:bbbbbbbb(?:))(?:bbbbbbbb(?:)(?:bbbbbbbb)))/ + caaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + +/foo\w*\d{4}baz/ + foobar1234baz + +/x(~~)*(?:(?:F)?)?/ + x~~ + +/^a(?#xxx){3}c/ + aaac + +/^a (?#xxx) (?#yyy) {3}c/x + aaac + +/(?<![cd])b/ + *** Failers + B\nB + dbcb + +/(?<![cd])[ab]/ + dbaacb + +/(?<!(c|d))b/ + +/(?<!(c|d))[ab]/ + dbaacb + +/(?<!cd)[ab]/ + cdaccb + +/^(?:a?b?)*$/ + *** Failers + dbcb + a-- + +/((?s)^a(.))((?m)^b$)/ + a\nb\nc\n + +/((?m)^b$)/ + a\nb\nc\n + +/(?m)^b/ + a\nb\n + +/(?m)^(b)/ + a\nb\n + +/((?m)^b)/ + a\nb\n + +/\n((?m)^b)/ + a\nb\n + +/((?s).)c(?!.)/ + a\nb\nc\n + a\nb\nc\n + +/((?s)b.)c(?!.)/ + a\nb\nc\n + a\nb\nc\n + +/^b/ + +/()^b/ + *** Failers + a\nb\nc\n + a\nb\nc\n + +/((?m)^b)/ + a\nb\nc\n + +/(?(?!a)a|b)/ + +/(?(?!a)b|a)/ + a + +/(?(?=a)b|a)/ + *** Failers + a + a + +/(?(?=a)a|b)/ + a + +/(\w+:)+/ + one: + +/$(?<=^(a))/ + a + +/([\w:]+::)?(\w+)$/ + abcd + xy:z:::abcd + +/^[^bcd]*(c+)/ + aexycd + +/(a*)b+/ + caab + +/([\w:]+::)?(\w+)$/ + abcd + xy:z:::abcd + *** Failers + abcd: + abcd: + +/^[^bcd]*(c+)/ + aexycd + +/(>a+)ab/ + +/(?>a+)b/ + aaab + +/([[:]+)/ + a:[b]: + +/([[=]+)/ + a=[b]= + +/([[.]+)/ + a.[b]. + +/((?>a+)b)/ + aaab + +/(?>(a+))b/ + aaab + +/((?>[^()]+)|\([^()]*\))+/ + ((abc(ade)ufh()()x + +/a\Z/ + *** Failers + aaab + a\nb\n + +/b\Z/ + a\nb\n + +/b\z/ + +/b\Z/ + a\nb + +/b\z/ + a\nb + *** Failers + +/(?>.*)(?<=(abcd|wxyz))/ + alphabetabcd + endingwxyz + *** Failers + a rather long string that doesn't end with one of them + +/word (?>(?:(?!otherword)[a-zA-Z0-9]+ ){0,30})otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark otherword + word cat dog elephant mussel cow horse canary baboon snake shark + +/word (?>[a-zA-Z0-9]+ ){0,30}otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark the quick brown fox and the lazy dog and several other words getting close to thirty by now I hope + +/(?<=\d{3}(?!999))foo/ + 999foo + 123999foo + *** Failers + 123abcfoo + +/(?<=(?!...999)\d{3})foo/ + 999foo + 123999foo + *** Failers + 123abcfoo + +/(?<=\d{3}(?!999)...)foo/ + 123abcfoo + 123456foo + *** Failers + 123999foo + +/(?<=\d{3}...)(?<!999)foo/ + 123abcfoo + 123456foo + *** Failers + 123999foo + +/((Z)+|A)*/ + ZABCDEFG + +/(Z()|A)*/ + ZABCDEFG + +/(Z(())|A)*/ + ZABCDEFG + +/((?>Z)+|A)*/ + ZABCDEFG + +/((?>)+|A)*/ + ZABCDEFG + +/a*/g + abbab + +/^[a-\d]/ + abcde + -things + 0digit + *** Failers + bcdef + +/^[\d-a]/ + abcde + -things + 0digit + *** Failers + bcdef + +/[[:space:]]+/ + > \x09\x0a\x0c\x0d\x0b< + +/[[:blank:]]+/ + > \x09\x0a\x0c\x0d\x0b< + +/[\s]+/ + > \x09\x0a\x0c\x0d\x0b< + +/\s+/ + > \x09\x0a\x0c\x0d\x0b< + +/a b/x + ab + +/(?!\A)x/m + a\nxb\n + +/(?!^)x/m + a\nxb\n + +/abc\Qabc\Eabc/ + abcabcabc + +/abc\Q(*+|\Eabc/ + abc(*+|abc + +/ abc\Q abc\Eabc/x + abc abcabc + *** Failers + abcabcabc + +/abc#comment + \Q#not comment + literal\E/x + abc#not comment\n literal + +/abc#comment + \Q#not comment + literal/x + abc#not comment\n literal + +/abc#comment + \Q#not comment + literal\E #more comment + /x + abc#not comment\n literal + +/abc#comment + \Q#not comment + literal\E #more comment/x + abc#not comment\n literal + +/\Qabc\$xyz\E/ + abc\\\$xyz + +/\Qabc\E\$\Qxyz\E/ + abc\$xyz + +/\Gabc/ + abc + *** Failers + xyzabc + +/\Gabc./g + abc1abc2xyzabc3 + +/abc./g + abc1abc2xyzabc3 + +/a(?x: b c )d/ + XabcdY + *** Failers + Xa b c d Y + +/((?x)x y z | a b c)/ + XabcY + AxyzB + +/(?i)AB(?-i)C/ + XabCY + *** Failers + XabcY + +/((?i)AB(?-i)C|D)E/ + abCE + DE + *** Failers + abcE + abCe + dE + De + +/[z\Qa-d]\E]/ + z + a + - + d + ] + *** Failers + b + +/[\z\C]/ + z + C + +/\M/ + M + +/(a+)*b/ + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/(?i)reg(?:ul(?:[aä]|ae)r|ex)/ + REGular + regulaer + Regex + regulär + +/Åæåä[à-ÿÀ-ß]+/ + Åæåäà + Åæåäÿ + ÅæåäÀ + Åæåäß + +/(?<=Z)X./ + \x84XAZXB + +/^(?(2)a|(1)(2))+$/ + 123a + +/(?<=a|bbbb)c/ + ac + bbbbc + +/abc/>testsavedregex +<testsavedregex + abc + *** Failers + bca + +/abc/F>testsavedregex +<testsavedregex + abc + *** Failers + bca + +/(a|b)/S>testsavedregex +<testsavedregex + abc + *** Failers + def + +/(a|b)/SF>testsavedregex +<testsavedregex + abc + *** Failers + def + +/line\nbreak/ + this is a line\nbreak + line one\nthis is a line\nbreak in the second line + +/line\nbreak/f + this is a line\nbreak + ** Failers + line one\nthis is a line\nbreak in the second line + +/line\nbreak/mf + this is a line\nbreak + ** Failers + line one\nthis is a line\nbreak in the second line + +/ End of testinput7 / diff --git a/testdata/testinput8 b/testdata/testinput8 new file mode 100644 index 0000000..36db243 --- /dev/null +++ b/testdata/testinput8 @@ -0,0 +1,540 @@ +/-- Do not use the \x{} construct except with patterns that have the --/ +/-- /8 option set, because PCRE doesn't recognize them as UTF-8 unless --/ +/-- that option is set. However, the latest Perls recognize them always. --/ + +/\x{100}ab/8 + \x{100}ab + +/a\x{100}*b/8 + ab + a\x{100}b + a\x{100}\x{100}b + +/a\x{100}+b/8 + a\x{100}b + a\x{100}\x{100}b + *** Failers + ab + +/\bX/8 + Xoanon + +Xoanon + \x{300}Xoanon + *** Failers + YXoanon + +/\BX/8 + YXoanon + *** Failers + Xoanon + +Xoanon + \x{300}Xoanon + +/X\b/8 + X+oanon + ZX\x{300}oanon + FAX + *** Failers + Xoanon + +/X\B/8 + Xoanon + *** Failers + X+oanon + ZX\x{300}oanon + FAX + +/[^a]/8 + abcd + a\x{100} + +/^[abc\x{123}\x{400}-\x{402}]{2,3}\d/8 + ab99 + \x{123}\x{123}45 + \x{400}\x{401}\x{402}6 + *** Failers + d99 + \x{123}\x{122}4 + \x{400}\x{403}6 + \x{400}\x{401}\x{402}\x{402}6 + +/abc/8 + Ã] + à + ÃÃà + ÃÃÃ\? + +/a.b/8 + acb + a\x7fb + a\x{100}b + *** Failers + a\nb + +/a(.{3})b/8 + a\x{4000}xyb + a\x{4000}\x7fyb + a\x{4000}\x{100}yb + *** Failers + a\x{4000}b + ac\ncb + +/a(.*?)(.)/ + a\xc0\x88b + +/a(.*?)(.)/8 + a\x{100}b + +/a(.*)(.)/ + a\xc0\x88b + +/a(.*)(.)/8 + a\x{100}b + +/a(.)(.)/ + a\xc0\x92bcd + +/a(.)(.)/8 + a\x{240}bcd + +/a(.?)(.)/ + a\xc0\x92bcd + +/a(.?)(.)/8 + a\x{240}bcd + +/a(.??)(.)/ + a\xc0\x92bcd + +/a(.??)(.)/8 + a\x{240}bcd + +/a(.{3})b/8 + a\x{1234}xyb + a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + *** Failers + a\x{1234}b + ac\ncb + +/a(.{3,})b/8 + a\x{1234}xyb + a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + a\x{1234}\x{4321}\x{3412}\x{3421}b + *** Failers + a\x{1234}b + +/a(.{3,}?)b/8 + a\x{1234}xyb + a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + a\x{1234}\x{4321}\x{3412}\x{3421}b + *** Failers + a\x{1234}b + +/a(.{3,5})b/8 + a\x{1234}xyb + a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + a\x{1234}\x{4321}\x{3412}\x{3421}b + axbxxbcdefghijb + axxxxxbcdefghijb + *** Failers + a\x{1234}b + axxxxxxbcdefghijb + +/a(.{3,5}?)b/8 + a\x{1234}xyb + a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + a\x{1234}\x{4321}\x{3412}\x{3421}b + axbxxbcdefghijb + axxxxxbcdefghijb + *** Failers + a\x{1234}b + axxxxxxbcdefghijb + +/^[a\x{c0}]/8 + *** Failers + \x{100} + +/(?<=aXb)cd/8 + aXbcd + +/(?<=a\x{100}b)cd/8 + a\x{100}bcd + +/(?<=a\x{100000}b)cd/8 + a\x{100000}bcd + +/(?:\x{100}){3}b/8 + \x{100}\x{100}\x{100}b + *** Failers + \x{100}\x{100}b + +/\x{ab}/8 + \x{ab} + \xc2\xab + *** Failers + \x00{ab} + +/(?<=(.))X/8 + WXYZ + \x{256}XYZ + *** Failers + XYZ + +/[^a]+/8g + bcd + \x{100}aY\x{256}Z + +/^[^a]{2}/8 + \x{100}bc + +/^[^a]{2,}/8 + \x{100}bcAa + +/^[^a]{2,}?/8 + \x{100}bca + +/[^a]+/8ig + bcd + \x{100}aY\x{256}Z + +/^[^a]{2}/8i + \x{100}bc + +/^[^a]{2,}/8i + \x{100}bcAa + +/^[^a]{2,}?/8i + \x{100}bca + +/\x{100}{0,0}/8 + abcd + +/\x{100}?/8 + abcd + \x{100}\x{100} + +/\x{100}{0,3}/8 + \x{100}\x{100} + \x{100}\x{100}\x{100}\x{100} + +/\x{100}*/8 + abce + \x{100}\x{100}\x{100}\x{100} + +/\x{100}{1,1}/8 + abcd\x{100}\x{100}\x{100}\x{100} + +/\x{100}{1,3}/8 + abcd\x{100}\x{100}\x{100}\x{100} + +/\x{100}+/8 + abcd\x{100}\x{100}\x{100}\x{100} + +/\x{100}{3}/8 + abcd\x{100}\x{100}\x{100}XX + +/\x{100}{3,5}/8 + abcd\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}XX + +/\x{100}{3,}/8 + abcd\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}XX + +/(?<=a\x{100}{2}b)X/8 + Xyyya\x{100}\x{100}bXzzz + +/\D*/8 + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/\D*/8 + \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + +/\D/8 + 1X2 + 1\x{100}2 + +/>\S/8 + > >X Y + > >\x{100} Y + +/\d/8 + \x{100}3 + +/\s/8 + \x{100} X + +/\D+/8 + 12abcd34 + *** Failers + 1234 + +/\D{2,3}/8 + 12abcd34 + 12ab34 + *** Failers + 1234 + 12a34 + +/\D{2,3}?/8 + 12abcd34 + 12ab34 + *** Failers + 1234 + 12a34 + +/\d+/8 + 12abcd34 + *** Failers + +/\d{2,3}/8 + 12abcd34 + 1234abcd + *** Failers + 1.4 + +/\d{2,3}?/8 + 12abcd34 + 1234abcd + *** Failers + 1.4 + +/\S+/8 + 12abcd34 + *** Failers + \ \ + +/\S{2,3}/8 + 12abcd34 + 1234abcd + *** Failers + \ \ + +/\S{2,3}?/8 + 12abcd34 + 1234abcd + *** Failers + \ \ + +/>\s+</8 + 12> <34 + *** Failers + +/>\s{2,3}</8 + ab> <cd + ab> <ce + *** Failers + ab> <cd + +/>\s{2,3}?</8 + ab> <cd + ab> <ce + *** Failers + ab> <cd + +/\w+/8 + 12 34 + *** Failers + +++=*! + +/\w{2,3}/8 + ab cd + abcd ce + *** Failers + a.b.c + +/\w{2,3}?/8 + ab cd + abcd ce + *** Failers + a.b.c + +/\W+/8 + 12====34 + *** Failers + abcd + +/\W{2,3}/8 + ab====cd + ab==cd + *** Failers + a.b.c + +/\W{2,3}?/8 + ab====cd + ab==cd + *** Failers + a.b.c + +/[\x{100}]/8 + \x{100} + Z\x{100} + \x{100}Z + *** Failers + +/[Z\x{100}]/8 + Z\x{100} + \x{100} + \x{100}Z + *** Failers + +/[\x{100}\x{200}]/8 + ab\x{100}cd + ab\x{200}cd + *** Failers + +/[\x{100}-\x{200}]/8 + ab\x{100}cd + ab\x{200}cd + ab\x{111}cd + *** Failers + +/[z-\x{200}]/8 + ab\x{100}cd + ab\x{200}cd + ab\x{111}cd + abzcd + ab|cd + *** Failers + +/[Q\x{100}\x{200}]/8 + ab\x{100}cd + ab\x{200}cd + Q? + *** Failers + +/[Q\x{100}-\x{200}]/8 + ab\x{100}cd + ab\x{200}cd + ab\x{111}cd + Q? + *** Failers + +/[Qz-\x{200}]/8 + ab\x{100}cd + ab\x{200}cd + ab\x{111}cd + abzcd + ab|cd + Q? + *** Failers + +/[\x{100}\x{200}]{1,3}/8 + ab\x{100}cd + ab\x{200}cd + ab\x{200}\x{100}\x{200}\x{100}cd + *** Failers + +/[\x{100}\x{200}]{1,3}?/8 + ab\x{100}cd + ab\x{200}cd + ab\x{200}\x{100}\x{200}\x{100}cd + *** Failers + +/[Q\x{100}\x{200}]{1,3}/8 + ab\x{100}cd + ab\x{200}cd + ab\x{200}\x{100}\x{200}\x{100}cd + *** Failers + +/[Q\x{100}\x{200}]{1,3}?/8 + ab\x{100}cd + ab\x{200}cd + ab\x{200}\x{100}\x{200}\x{100}cd + *** Failers + +/(?<=[\x{100}\x{200}])X/8 + abc\x{200}X + abc\x{100}X + *** Failers + X + +/(?<=[Q\x{100}\x{200}])X/8 + abc\x{200}X + abc\x{100}X + abQX + *** Failers + X + +/(?<=[\x{100}\x{200}]{3})X/8 + abc\x{100}\x{200}\x{100}X + *** Failers + abc\x{200}X + X + +/[^\x{100}\x{200}]X/8 + AX + \x{150}X + \x{500}X + *** Failers + \x{100}X + \x{200}X + +/[^Q\x{100}\x{200}]X/8 + AX + \x{150}X + \x{500}X + *** Failers + \x{100}X + \x{200}X + QX + +/[^\x{100}-\x{200}]X/8 + AX + \x{500}X + *** Failers + \x{100}X + \x{150}X + \x{200}X + +/[z-\x{100}]/8i + z + Z + \x{100} + *** Failers + \x{102} + y + +/[\xFF]/ + >\xff< + +/[\xff]/8 + >\x{ff}< + +/[^\xFF]/ + XYZ + +/[^\xff]/8 + XYZ + \x{123} + +/^[ac]*b/8 + xb + +/^[ac\x{100}]*b/8 + xb + +/^[^x]*b/8i + xb + +/^[^x]*b/8 + xb + +/^\d*b/8 + xb + +/(|a)/g8 + catac + a\x{256}a + +/^\x{85}$/8i + \x{85} + +/ End of testinput 8 / diff --git a/testdata/testinput9 b/testdata/testinput9 new file mode 100644 index 0000000..f18dd12 --- /dev/null +++ b/testdata/testinput9 @@ -0,0 +1,599 @@ +/\pL\P{Nd}/8 + AB + *** Failers + A0 + 00 + +/\X./8 + AB + A\x{300}BC + A\x{300}\x{301}\x{302}BC + *** Failers + \x{300} + +/\X\X/8 + ABC + A\x{300}B\x{300}\x{301}C + A\x{300}\x{301}\x{302}BC + *** Failers + \x{300} + +/^\pL+/8 + abcd + a + *** Failers + +/^\PL+/8 + 1234 + = + *** Failers + abcd + +/^\X+/8 + abcdA\x{300}\x{301}\x{302} + A\x{300}\x{301}\x{302} + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302} + a + *** Failers + \x{300}\x{301}\x{302} + +/\X?abc/8 + abc + A\x{300}abc + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz + \x{300}abc + *** Failers + +/^\X?abc/8 + abc + A\x{300}abc + *** Failers + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz + \x{300}abc + +/\X*abc/8 + abc + A\x{300}abc + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz + \x{300}abc + *** Failers + +/^\X*abc/8 + abc + A\x{300}abc + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz + *** Failers + \x{300}abc + +/^\pL?=./8 + A=b + =c + *** Failers + 1=2 + AAAA=b + +/^\pL*=./8 + AAAA=b + =c + *** Failers + 1=2 + +/^\X{2,3}X/8 + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X + *** Failers + X + A\x{300}\x{301}\x{302}X + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X + +/^\pC\pL\pM\pN\pP\pS\pZ</8 + \x7f\x{c0}\x{30f}\x{660}\x{66c}\x{f01}\x{1680}< + \np\x{300}9!\$ < + ** Failers + ap\x{300}9!\$ < + +/^\PC/8 + X + ** Failers + \x7f + +/^\PL/8 + 9 + ** Failers + \x{c0} + +/^\PM/8 + X + ** Failers + \x{30f} + +/^\PN/8 + X + ** Failers + \x{660} + +/^\PP/8 + X + ** Failers + \x{66c} + +/^\PS/8 + X + ** Failers + \x{f01} + +/^\PZ/8 + X + ** Failers + \x{1680} + +/^\p{Cc}/8 + \x{017} + \x{09f} + ** Failers + \x{0600} + +/^\p{Cf}/8 + \x{601} + ** Failers + \x{09f} + +/^\p{Cn}/8 + ** Failers + \x{09f} + +/^\p{Co}/8 + \x{f8ff} + ** Failers + \x{09f} + +/^\p{Cs}/8 + \x{dfff} + ** Failers + \x{09f} + +/^\p{Ll}/8 + a + ** Failers + Z + \x{dfff} + +/^\p{Lm}/8 + \x{2b0} + ** Failers + a + +/^\p{Lo}/8 + \x{1bb} + ** Failers + a + \x{2b0} + +/^\p{Lt}/8 + \x{1c5} + ** Failers + a + \x{2b0} + +/^\p{Lu}/8 + A + ** Failers + \x{2b0} + +/^\p{Mc}/8 + \x{903} + ** Failers + X + \x{300} + +/^\p{Me}/8 + \x{488} + ** Failers + X + \x{903} + \x{300} + +/^\p{Mn}/8 + \x{300} + ** Failers + X + \x{903} + +/^\p{Nd}+/8 + 0123456789\x{660}\x{661}\x{662}\x{663}\x{664}\x{665}\x{666}\x{667}\x{668}\x{669}\x{66a} + \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4}\x{6f5}\x{6f6}\x{6f7}\x{6f8}\x{6f9}\x{6fa} + \x{966}\x{967}\x{968}\x{969}\x{96a}\x{96b}\x{96c}\x{96d}\x{96e}\x{96f}\x{970} + ** Failers + X + +/^\p{Nl}/8 + \x{16ee} + ** Failers + X + \x{966} + +/^\p{No}/8 + \x{b2} + \x{b3} + ** Failers + X + \x{16ee} + +/^\p{Pc}/8 + \x5f + \x{203f} + ** Failers + X + - + \x{58a} + +/^\p{Pd}/8 + - + \x{58a} + ** Failers + X + \x{203f} + +/^\p{Pe}/8 + ) + ] + } + \x{f3b} + ** Failers + X + \x{203f} + ( + [ + { + \x{f3c} + +/^\p{Pf}/8 + \x{bb} + \x{2019} + ** Failers + X + \x{203f} + +/^\p{Pi}/8 + \x{ab} + \x{2018} + ** Failers + X + \x{203f} + +/^\p{Po}/8 + ! + \x{37e} + ** Failers + X + \x{203f} + +/^\p{Ps}/8 + ( + [ + { + \x{f3c} + ** Failers + X + ) + ] + } + \x{f3b} + +/^\p{Sc}+/8 + $\x{a2}\x{a3}\x{a4}\x{a5}\x{a6} + \x{9f2} + ** Failers + X + \x{2c2} + +/^\p{Sk}/8 + \x{2c2} + ** Failers + X + \x{9f2} + +/^\p{Sm}+/8 + +<|~\x{ac}\x{2044} + ** Failers + X + \x{9f2} + +/^\p{So}/8 + \x{a6} + \x{482} + ** Failers + X + \x{9f2} + +/^\p{Zl}/8 + \x{2028} + ** Failers + X + \x{2029} + +/^\p{Zp}/8 + \x{2029} + ** Failers + X + \x{2028} + +/^\p{Zs}/8 + \ \ + \x{a0} + \x{1680} + \x{180e} + \x{2000} + \x{2001} + ** Failers + \x{2028} + \x{200d} + +/\p{Nd}+(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}+?(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}{2,}(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}{2,}?(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}*(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}*?(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}{2}(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}{2,3}(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}{2,3}?(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}?(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}??(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}*+(..)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}*+(...)/8 + \x{660}\x{661}\x{662}ABC + +/\p{Nd}*+(....)/8 + ** Failers + \x{660}\x{661}\x{662}ABC + +/\p{Lu}/8i + A + a\x{10a0}B + ** Failers + a + \x{1d00} + +/\p{^Lu}/8i + 1234 + ** Failers + ABC + +/\P{Lu}/8i + 1234 + ** Failers + ABC + +/(?<=A\p{Nd})XYZ/8 + A2XYZ + 123A5XYZPQR + ABA\x{660}XYZpqr + ** Failers + AXYZ + XYZ + +/(?<!\pL)XYZ/8 + 1XYZ + AB=XYZ.. + XYZ + ** Failers + WXYZ + +/[\p{Nd}]/8 + 1234 + +/[\p{Nd}+-]+/8 + 1234 + 12-34 + 12+\x{661}-34 + ** Failers + abcd + +/[\P{Nd}]+/8 + abcd + ** Failers + 1234 + +/\D+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/\P{Nd}+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/[\D]+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/[\P{Nd}]+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/[\D\P{Nd}]+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/\pL/8 + a + A + +/\pL/8i + a + A + +/\p{Lu}/8 + A + aZ + ** Failers + abc + +/\p{Lu}/8i + A + aZ + ** Failers + abc + +/\p{Ll}/8 + a + Az + ** Failers + ABC + +/\p{Ll}/8i + a + Az + ** Failers + ABC + +/^\x{c0}$/8i + \x{c0} + \x{e0} + +/^\x{e0}$/8i + \x{c0} + \x{e0} + +/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8 + A\x{391}\x{10427}\x{ff3a}\x{1fb0} + ** Failers + a\x{391}\x{10427}\x{ff3a}\x{1fb0} + A\x{3b1}\x{10427}\x{ff3a}\x{1fb0} + A\x{391}\x{1044F}\x{ff3a}\x{1fb0} + A\x{391}\x{10427}\x{ff5a}\x{1fb0} + A\x{391}\x{10427}\x{ff3a}\x{1fb8} + +/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8i + A\x{391}\x{10427}\x{ff3a}\x{1fb0} + a\x{391}\x{10427}\x{ff3a}\x{1fb0} + A\x{3b1}\x{10427}\x{ff3a}\x{1fb0} + A\x{391}\x{1044F}\x{ff3a}\x{1fb0} + A\x{391}\x{10427}\x{ff5a}\x{1fb0} + A\x{391}\x{10427}\x{ff3a}\x{1fb8} + +/\x{391}+/8i + \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} + +/\x{391}{3,5}(.)/8i + \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391}X + +/\x{391}{3,5}?(.)/8i + \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391}X + +/[\x{391}\x{ff3a}]/8i + \x{391} + \x{ff3a} + \x{3b1} + \x{ff5a} + +/[\x{c0}\x{391}]/8i + \x{c0} + \x{e0} + +/[\x{105}-\x{109}]/8i + \x{104} + \x{105} + \x{109} + ** Failers + \x{100} + \x{10a} + +/[z-\x{100}]/8i + Z + z + \x{39c} + \x{178} + | + \x{80} + \x{ff} + \x{100} + \x{101} + ** Failers + \x{102} + Y + y + +/[z-\x{100}]/8i + +/^\X/8 + A + A\x{300}BC + A\x{300}\x{301}\x{302}BC + *** Failers + \x{300} + +/^[\X]/8 + X123 + *** Failers + AXYZ + +/^(\X*)C/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + +/^(\X*?)C/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + +/^(\X*)(.)/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + +/^(\X*?)(.)/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + +/^\X(.)/8 + *** Failers + A\x{300}\x{301}\x{302} + +/^\X{2,3}(.)/8 + A\x{300}\x{301}B\x{300}X + A\x{300}\x{301}B\x{300}C\x{300}\x{301} + A\x{300}\x{301}B\x{300}C\x{300}\x{301}X + A\x{300}\x{301}B\x{300}C\x{300}\x{301}DA\x{300}X + +/^\X{2,3}?(.)/8 + A\x{300}\x{301}B\x{300}X + A\x{300}\x{301}B\x{300}C\x{300}\x{301} + A\x{300}\x{301}B\x{300}C\x{300}\x{301}X + A\x{300}\x{301}B\x{300}C\x{300}\x{301}DA\x{300}X + +/^\pN{2,3}X/ + 12X + 123X + *** Failers + X + 1X + 1234X + +/\x{100}/i8 + \x{100} + \x{101} + +/ End / diff --git a/testdata/testoutput1 b/testdata/testoutput1 index 07e7376..dc0575e 100644 --- a/testdata/testoutput1 +++ b/testdata/testoutput1 @@ -1,4 +1,4 @@ -PCRE version 5.0 13-Sep-2004 +PCRE version 6.0 07-Jun-2005 /the quick brown fox/ the quick brown fox @@ -6271,4 +6271,16 @@ No match \x84XAZXB 0: XB +/ab cd (?x) de fg/ + ab cd defg + 0: ab cd defg + +/ab cd(?x) de fg/ + ab cddefg + 0: ab cddefg + ** Failers +No match + abcddefg +No match + / End of testinput1 / diff --git a/testdata/testoutput2 b/testdata/testoutput2 index f2e7d54..63abb14 100644 --- a/testdata/testoutput2 +++ b/testdata/testoutput2 @@ -1,4 +1,4 @@ -PCRE version 5.0 13-Sep-2004 +PCRE version 6.0 07-Jun-2005 /(a)b|/ Capturing subpattern count = 1 @@ -3841,6 +3841,58 @@ Callout 0: last capture = 1 0: xyz 1: abc +/a(b+)(c*)(?C1)/ +Capturing subpattern count = 2 +Partial matching not supported +No options +First char = 'a' +Need char = 'b' + abbbbbccc\C*1 +--->abbbbbccc + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 +No match + +/a(b+?)(c*?)(?C1)/ +Capturing subpattern count = 2 +Partial matching not supported +No options +First char = 'a' +Need char = 'b' + abbbbbccc\C*1 +--->abbbbbccc + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 + 1 ^ ^ +Callout data = 1 +No match + /(?C)abc/ Capturing subpattern count = 0 No options @@ -5600,6 +5652,58 @@ Options: anchored No first char No need char +/line\nbreak/ +Capturing subpattern count = 0 +No options +First char = 'l' +Need char = 'k' + this is a line\nbreak + 0: line\x0abreak + line one\nthis is a line\nbreak in the second line + 0: line\x0abreak + +/line\nbreak/f +Capturing subpattern count = 0 +Options: firstline +First char = 'l' +Need char = 'k' + this is a line\nbreak + 0: line\x0abreak + ** Failers +No match + line one\nthis is a line\nbreak in the second line +No match + +/line\nbreak/mf +Capturing subpattern count = 0 +Options: multiline firstline +First char = 'l' +Need char = 'k' + this is a line\nbreak + 0: line\x0abreak + ** Failers +No match + line one\nthis is a line\nbreak in the second line +No match + +/ab.cd/P + ab-cd + 0: ab-cd + ab=cd + 0: ab=cd + ** Failers +No match: POSIX code 17: match failed + ab\ncd +No match: POSIX code 17: match failed + +/ab.cd/Ps + ab-cd + 0: ab-cd + ab=cd + 0: ab=cd + ab\ncd + 0: ab\x0acd + / End of testinput2 / Capturing subpattern count = 0 No options diff --git a/testdata/testoutput3 b/testdata/testoutput3 index 225bd5b..d025d7a 100644 --- a/testdata/testoutput3 +++ b/testdata/testoutput3 @@ -1,4 +1,4 @@ -PCRE version 5.0 13-Sep-2004 +PCRE version 6.0 07-Jun-2005 /^[\w]+/ *** Failers diff --git a/testdata/testoutput4 b/testdata/testoutput4 index e8d2603..51fc085 100644 --- a/testdata/testoutput4 +++ b/testdata/testoutput4 @@ -1,4 +1,4 @@ -PCRE version 5.0 13-Sep-2004 +PCRE version 6.0 07-Jun-2005 /-- Do not use the \x{} construct except with patterns that have the --/ /-- /8 option set, because PCRE doesn't recognize them as UTF-8 unless --/ diff --git a/testdata/testoutput5 b/testdata/testoutput5 index b010957..ec31cf9 100644 --- a/testdata/testoutput5 +++ b/testdata/testoutput5 @@ -1,4 +1,4 @@ -PCRE version 5.0 13-Sep-2004 +PCRE version 6.0 07-Jun-2005 /\x{100}/8DM Memory allocation (code space): 10 diff --git a/testdata/testoutput6 b/testdata/testoutput6 index 732e5aa..3a75a7f 100644 --- a/testdata/testoutput6 +++ b/testdata/testoutput6 @@ -1,4 +1,4 @@ -PCRE version 5.0 13-Sep-2004 +PCRE version 6.0 07-Jun-2005 /^\pC\pL\pM\pN\pP\pS\pZ</8 \x7f\x{c0}\x{30f}\x{660}\x{66c}\x{f01}\x{1680}< diff --git a/testdata/testoutput7 b/testdata/testoutput7 new file mode 100644 index 0000000..24ec615 --- /dev/null +++ b/testdata/testoutput7 @@ -0,0 +1,6516 @@ +PCRE version 6.0 07-Jun-2005 + +/abc/ + abc + 0: abc + +/ab*c/ + abc + 0: abc + abbbbc + 0: abbbbc + ac + 0: ac + +/ab+c/ + abc + 0: abc + abbbbbbc + 0: abbbbbbc + *** Failers +No match + ac +No match + ab +No match + +/a*/ + a + 0: a + 1: + aaaaaaaaaaaaaaaaa + 0: aaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaa + 4: aaaaaaaaaaaaa + 5: aaaaaaaaaaaa + 6: aaaaaaaaaaa + 7: aaaaaaaaaa + 8: aaaaaaaaa + 9: aaaaaaaa +10: aaaaaaa +11: aaaaaa +12: aaaaa +13: aaaa +14: aaa +15: aa +16: a +17: + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +Matched, but too many subsidiary matches + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaaaaaaaaaaaaaaa + 4: aaaaaaaaaaaaaaaaaaaaaaaaaa + 5: aaaaaaaaaaaaaaaaaaaaaaaaa + 6: aaaaaaaaaaaaaaaaaaaaaaaa + 7: aaaaaaaaaaaaaaaaaaaaaaa + 8: aaaaaaaaaaaaaaaaaaaaaa + 9: aaaaaaaaaaaaaaaaaaaaa +10: aaaaaaaaaaaaaaaaaaaa +11: aaaaaaaaaaaaaaaaaaa +12: aaaaaaaaaaaaaaaaaa +13: aaaaaaaaaaaaaaaaa +14: aaaaaaaaaaaaaaaa +15: aaaaaaaaaaaaaaa +16: aaaaaaaaaaaaaa +17: aaaaaaaaaaaaa +18: aaaaaaaaaaaa +19: aaaaaaaaaaa +20: aaaaaaaaaa +21: aaaaaaaaa + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\F + 0: + +/(a|abcd|african)/ + a + 0: a + abcd + 0: abcd + 1: a + african + 0: african + 1: a + +/^abc/ + abcdef + 0: abc + *** Failers +No match + xyzabc +No match + xyz\nabc +No match + +/^abc/m + abcdef + 0: abc + xyz\nabc + 0: abc + *** Failers +No match + xyzabc +No match + +/\Aabc/ + abcdef + 0: abc + *** Failers +No match + xyzabc +No match + xyz\nabc +No match + +/\Aabc/m + abcdef + 0: abc + *** Failers +No match + xyzabc +No match + xyz\nabc +No match + +/\Gabc/ + abcdef + 0: abc + xyzabc\>3 + 0: abc + *** Failers +No match + xyzabc +No match + xyzabc\>2 +No match + +/x\dy\Dz/ + x9yzz + 0: x9yzz + x0y+z + 0: x0y+z + *** Failers +No match + xyz +No match + xxy0z +No match + +/x\sy\Sz/ + x yzz + 0: x yzz + x y+z + 0: x y+z + *** Failers +No match + xyz +No match + xxyyz +No match + +/x\wy\Wz/ + xxy+z + 0: xxy+z + *** Failers +No match + xxy0z +No match + x+y+z +No match + +/x.y/ + x+y + 0: x+y + x-y + 0: x-y + *** Failers +No match + x\ny +No match + +/x.y/s + x+y + 0: x+y + x-y + 0: x-y + x\ny + 0: x\x0ay + +/(a.b(?s)c.d|x.y)p.q/ + a+bc+dp+q + 0: a+bc+dp+q + a+bc\ndp+q + 0: a+bc\x0adp+q + x\nyp+q + 0: x\x0ayp+q + *** Failers +No match + a\nbc\ndp+q +No match + a+bc\ndp\nq +No match + x\nyp\nq +No match + +/a\d\z/ + ba0 + 0: a0 + *** Failers +No match + ba0\n +No match + ba0\ncd +No match + +/a\d\z/m + ba0 + 0: a0 + *** Failers +No match + ba0\n +No match + ba0\ncd +No match + +/a\d\Z/ + ba0 + 0: a0 + ba0\n + 0: a0 + *** Failers +No match + ba0\ncd +No match + +/a\d\Z/m + ba0 + 0: a0 + ba0\n + 0: a0 + *** Failers +No match + ba0\ncd +No match + +/a\d$/ + ba0 + 0: a0 + ba0\n + 0: a0 + *** Failers +No match + ba0\ncd +No match + +/a\d$/m + ba0 + 0: a0 + ba0\n + 0: a0 + ba0\ncd + 0: a0 + *** Failers +No match + +/abc/i + abc + 0: abc + aBc + 0: aBc + ABC + 0: ABC + +/[^a]/ + abcd + 0: b + +/ab?\w/ + abz + 0: abz + 1: ab + abbz + 0: abb + 1: ab + azz + 0: az + +/x{0,3}yz/ + ayzq + 0: yz + axyzq + 0: xyz + axxyz + 0: xxyz + axxxyzq + 0: xxxyz + axxxxyzq + 0: xxxyz + *** Failers +No match + ax +No match + axx +No match + +/x{3}yz/ + axxxyzq + 0: xxxyz + axxxxyzq + 0: xxxyz + *** Failers +No match + ax +No match + axx +No match + ayzq +No match + axyzq +No match + axxyz +No match + +/x{2,3}yz/ + axxyz + 0: xxyz + axxxyzq + 0: xxxyz + axxxxyzq + 0: xxxyz + *** Failers +No match + ax +No match + axx +No match + ayzq +No match + axyzq +No match + +/[^a]+/ + bac + 0: b + bcdefax + 0: bcdef + 1: bcde + 2: bcd + 3: bc + 4: b + *** Failers + 0: *** F + 1: *** + 2: *** + 3: ** + 4: * + aaaaa +No match + +/[^a]*/ + bac + 0: b + 1: + bcdefax + 0: bcdef + 1: bcde + 2: bcd + 3: bc + 4: b + 5: + *** Failers + 0: *** F + 1: *** + 2: *** + 3: ** + 4: * + 5: + aaaaa + 0: + +/[^a]{3,5}/ + xyz + 0: xyz + awxyza + 0: wxyz + 1: wxy + abcdefa + 0: bcdef + 1: bcde + 2: bcd + abcdefghijk + 0: bcdef + 1: bcde + 2: bcd + *** Failers + 0: *** F + 1: *** + 2: *** + axya +No match + axa +No match + aaaaa +No match + +/\d*/ + 1234b567 + 0: 1234 + 1: 123 + 2: 12 + 3: 1 + 4: + xyz + 0: + +/\D*/ + a1234b567 + 0: a + 1: + xyz + 0: xyz + 1: xy + 2: x + 3: + +/\d+/ + ab1234c56 + 0: 1234 + 1: 123 + 2: 12 + 3: 1 + *** Failers +No match + xyz +No match + +/\D+/ + ab123c56 + 0: ab + 1: a + *** Failers + 0: *** Failers + 1: *** Failer + 2: *** Faile + 3: *** Fail + 4: *** Fai + 5: *** Fa + 6: *** F + 7: *** + 8: *** + 9: ** +10: * + 789 +No match + +/\d?A/ + 045ABC + 0: 5A + ABC + 0: A + *** Failers +No match + XYZ +No match + +/\D?A/ + ABC + 0: A + BAC + 0: BA + 9ABC + 0: A + *** Failers +No match + +/a+/ + aaaa + 0: aaaa + 1: aaa + 2: aa + 3: a + +/^.*xyz/ + xyz + 0: xyz + ggggggggxyz + 0: ggggggggxyz + +/^.+xyz/ + abcdxyz + 0: abcdxyz + axyz + 0: axyz + *** Failers +No match + xyz +No match + +/^.?xyz/ + xyz + 0: xyz + cxyz + 0: cxyz + +/^\d{2,3}X/ + 12X + 0: 12X + 123X + 0: 123X + *** Failers +No match + X +No match + 1X +No match + 1234X +No match + +/^[abcd]\d/ + a45 + 0: a4 + b93 + 0: b9 + c99z + 0: c9 + d04 + 0: d0 + *** Failers +No match + e45 +No match + abcd +No match + abcd1234 +No match + 1234 +No match + +/^[abcd]*\d/ + a45 + 0: a4 + b93 + 0: b9 + c99z + 0: c9 + d04 + 0: d0 + abcd1234 + 0: abcd1 + 1234 + 0: 1 + *** Failers +No match + e45 +No match + abcd +No match + +/^[abcd]+\d/ + a45 + 0: a4 + b93 + 0: b9 + c99z + 0: c9 + d04 + 0: d0 + abcd1234 + 0: abcd1 + *** Failers +No match + 1234 +No match + e45 +No match + abcd +No match + +/^a+X/ + aX + 0: aX + aaX + 0: aaX + +/^[abcd]?\d/ + a45 + 0: a4 + b93 + 0: b9 + c99z + 0: c9 + d04 + 0: d0 + 1234 + 0: 1 + *** Failers +No match + abcd1234 +No match + e45 +No match + +/^[abcd]{2,3}\d/ + ab45 + 0: ab4 + bcd93 + 0: bcd9 + *** Failers +No match + 1234 +No match + a36 +No match + abcd1234 +No match + ee45 +No match + +/^(abc)*\d/ + abc45 + 0: abc4 + abcabcabc45 + 0: abcabcabc4 + 42xyz + 0: 4 + *** Failers +No match + +/^(abc)+\d/ + abc45 + 0: abc4 + abcabcabc45 + 0: abcabcabc4 + *** Failers +No match + 42xyz +No match + +/^(abc)?\d/ + abc45 + 0: abc4 + 42xyz + 0: 4 + *** Failers +No match + abcabcabc45 +No match + +/^(abc){2,3}\d/ + abcabc45 + 0: abcabc4 + abcabcabc45 + 0: abcabcabc4 + *** Failers +No match + abcabcabcabc45 +No match + abc45 +No match + 42xyz +No match + +/1(abc|xyz)2(?1)3/ + 1abc2abc3456 + 0: 1abc2abc3 + 1abc2xyz3456 + 0: 1abc2xyz3 + +/^(a*\w|ab)=(a*\w|ab)/ + ab=ab + 0: ab=ab + 1: ab=a + +/^(a*\w|ab)=(?1)/ + ab=ab + 0: ab=ab + 1: ab=a + +/^([^()]|\((?1)*\))*$/ + abc + 0: abc + a(b)c + 0: a(b)c + a(b(c))d + 0: a(b(c))d + *** Failers) +No match + a(b(c)d +No match + +/^>abc>([^()]|\((?1)*\))*<xyz<$/ + >abc>123<xyz< + 0: >abc>123<xyz< + >abc>1(2)3<xyz< + 0: >abc>1(2)3<xyz< + >abc>(1(2)3)<xyz< + 0: >abc>(1(2)3)<xyz< + +/^(?>a*)\d/ + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa9876 + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa9 + *** Failers +No match + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +No match + +/< (?: (?(R) \d++ | [^<>]*+) | (?R)) * >/x + <> + 0: <> + <abcd> + 0: <abcd> + <abc <123> hij> + 0: <abc <123> hij> + <abc <def> hij> + 0: <def> + <abc<>def> + 0: <abc<>def> + <abc<> + 0: <> + *** Failers +No match + <abc +No match + +/^(?(?=abc)\w{3}:|\d\d)$/ + abc: + 0: abc: + 12 + 0: 12 + *** Failers +No match + 123 +No match + xyz +No match + +/^(?(?!abc)\d\d|\w{3}:)$/ + abc: + 0: abc: + 12 + 0: 12 + *** Failers +No match + 123 +No match + xyz +No match + +/^(?=abc)\w{5}:$/ + abcde: + 0: abcde: + *** Failers +No match + abc.. +No match + 123 +No match + vwxyz +No match + +/^(?!abc)\d\d$/ + 12 + 0: 12 + *** Failers +No match + abcde: +No match + abc.. +No match + 123 +No match + vwxyz +No match + +/(?<=abc|xy)123/ + abc12345 + 0: 123 + wxy123z + 0: 123 + *** Failers +No match + 123abc +No match + +/(?<!abc|xy)123/ + 123abc + 0: 123 + mno123456 + 0: 123 + *** Failers +No match + abc12345 +No match + wxy123z +No match + +/abc(?C1)xyz/ + abcxyz +--->abcxyz + 1 ^ ^ x + 0: abcxyz + 123abcxyz999 +--->123abcxyz999 + 1 ^ ^ x + 0: abcxyz + +/(ab|cd){3,4}/C + ababab +--->ababab + +0 ^ (ab|cd){3,4} + +1 ^ a + +4 ^ c + +2 ^^ b + +3 ^ ^ | + +1 ^ ^ a + +4 ^ ^ c + +2 ^ ^ b + +3 ^ ^ | + +1 ^ ^ a + +4 ^ ^ c + +2 ^ ^ b + +3 ^ ^ | ++12 ^ ^ + +1 ^ ^ a + +4 ^ ^ c + 0: ababab + abcdabcd +--->abcdabcd + +0 ^ (ab|cd){3,4} + +1 ^ a + +4 ^ c + +2 ^^ b + +3 ^ ^ | + +1 ^ ^ a + +4 ^ ^ c + +5 ^ ^ d + +6 ^ ^ ) + +1 ^ ^ a + +4 ^ ^ c + +2 ^ ^ b + +3 ^ ^ | ++12 ^ ^ + +1 ^ ^ a + +4 ^ ^ c + +5 ^ ^ d + +6 ^ ^ ) ++12 ^ ^ + 0: abcdabcd + 1: abcdab + abcdcdcdcdcd +--->abcdcdcdcdcd + +0 ^ (ab|cd){3,4} + +1 ^ a + +4 ^ c + +2 ^^ b + +3 ^ ^ | + +1 ^ ^ a + +4 ^ ^ c + +5 ^ ^ d + +6 ^ ^ ) + +1 ^ ^ a + +4 ^ ^ c + +5 ^ ^ d + +6 ^ ^ ) ++12 ^ ^ + +1 ^ ^ a + +4 ^ ^ c + +5 ^ ^ d + +6 ^ ^ ) ++12 ^ ^ + 0: abcdcdcd + 1: abcdcd + +/^abc/ + abcdef + 0: abc + *** Failers +No match + abcdef\B +No match + +/^(a*|xyz)/ + bcd + 0: + aaabcd + 0: aaa + 1: aa + 2: a + 3: + xyz + 0: xyz + 1: + xyz\N + 0: xyz + *** Failers + 0: + bcd\N +No match + +/xyz$/ + xyz + 0: xyz + xyz\n + 0: xyz + *** Failers +No match + xyz\Z +No match + xyz\n\Z +No match + +/xyz$/m + xyz + 0: xyz + xyz\n + 0: xyz + abcxyz\npqr + 0: xyz + abcxyz\npqr\Z + 0: xyz + xyz\n\Z + 0: xyz + *** Failers +No match + xyz\Z +No match + +/\Gabc/ + abcdef + 0: abc + defabcxyz\>3 + 0: abc + *** Failers +No match + defabcxyz +No match + +/^abcdef/ + ab\P +Partial match: ab + abcde\P +Partial match: abcde + abcdef\P + 0: abcdef + *** Failers +No match + abx\P +No match + +/^a{2,4}\d+z/ + a\P +Partial match: a + aa\P +Partial match: aa + aa2\P +Partial match: aa2 + aaa\P +Partial match: aaa + aaa23\P +Partial match: aaa23 + aaaa12345\P +Partial match: aaaa12345 + aa0z\P + 0: aa0z + aaaa4444444444444z\P + 0: aaaa4444444444444z + *** Failers +No match + az\P +No match + aaaaa\P +No match + a56\P +No match + +/^abcdef/ + abc\P +Partial match: abc + def\R + 0: def + +/(?<=foo)bar/ + xyzfo\P +No match + foob\R\P\>2 +Partial match: b + foobar...\R\P\>4 + 0: ar + xyzfo\P +No match + foobar\R\>2 + 0: bar + *** Failers +No match + xyzfo\P +No match + obar\R +No match + +/(ab*(cd|ef))+X/ + adfadadaklhlkalkajhlkjahdfasdfasdfladsfjkj\P\Z +No match + lkjhlkjhlkjhlkjhabbbbbbcdaefabbbbbbbefa\P\B\Z +Partial match: abbbbbbcdaefabbbbbbbefa + cdabbbbbbbb\P\R\B\Z +Partial match: cdabbbbbbbb + efabbbbbbbbbbbbbbbb\P\R\B\Z +Partial match: efabbbbbbbbbbbbbbbb + bbbbbbbbbbbbcdXyasdfadf\P\R\B\Z + 0: bbbbbbbbbbbbcdX + +/(a|b)/SF>testsavedregex +Compiled regex written to testsavedregex +Study data written to testsavedregex +<testsavedregex +Compiled regex (byte-inverted) loaded from testsavedregex +Study data loaded from testsavedregex + abc + 0: a + ** Failers + 0: a + def +No match + +/the quick brown fox/ + the quick brown fox + 0: the quick brown fox + The quick brown FOX +No match + What do you know about the quick brown fox? + 0: the quick brown fox + What do you know about THE QUICK BROWN FOX? +No match + +/The quick brown fox/i + the quick brown fox + 0: the quick brown fox + The quick brown FOX + 0: The quick brown FOX + What do you know about the quick brown fox? + 0: the quick brown fox + What do you know about THE QUICK BROWN FOX? + 0: THE QUICK BROWN FOX + +/abcd\t\n\r\f\a\e\071\x3b\$\\\?caxyz/ + abcd\t\n\r\f\a\e9;\$\\?caxyz + 0: abcd\x09\x0a\x0d\x0c\x07\x1b9;$\?caxyz + +/a*abc?xyz+pqr{3}ab{2,}xy{4,5}pq{0,6}AB{0,}zz/ + abxyzpqrrrabbxyyyypqAzz + 0: abxyzpqrrrabbxyyyypqAzz + abxyzpqrrrabbxyyyypqAzz + 0: abxyzpqrrrabbxyyyypqAzz + aabxyzpqrrrabbxyyyypqAzz + 0: aabxyzpqrrrabbxyyyypqAzz + aaabxyzpqrrrabbxyyyypqAzz + 0: aaabxyzpqrrrabbxyyyypqAzz + aaaabxyzpqrrrabbxyyyypqAzz + 0: aaaabxyzpqrrrabbxyyyypqAzz + abcxyzpqrrrabbxyyyypqAzz + 0: abcxyzpqrrrabbxyyyypqAzz + aabcxyzpqrrrabbxyyyypqAzz + 0: aabcxyzpqrrrabbxyyyypqAzz + aaabcxyzpqrrrabbxyyyypAzz + 0: aaabcxyzpqrrrabbxyyyypAzz + aaabcxyzpqrrrabbxyyyypqAzz + 0: aaabcxyzpqrrrabbxyyyypqAzz + aaabcxyzpqrrrabbxyyyypqqAzz + 0: aaabcxyzpqrrrabbxyyyypqqAzz + aaabcxyzpqrrrabbxyyyypqqqAzz + 0: aaabcxyzpqrrrabbxyyyypqqqAzz + aaabcxyzpqrrrabbxyyyypqqqqAzz + 0: aaabcxyzpqrrrabbxyyyypqqqqAzz + aaabcxyzpqrrrabbxyyyypqqqqqAzz + 0: aaabcxyzpqrrrabbxyyyypqqqqqAzz + aaabcxyzpqrrrabbxyyyypqqqqqqAzz + 0: aaabcxyzpqrrrabbxyyyypqqqqqqAzz + aaaabcxyzpqrrrabbxyyyypqAzz + 0: aaaabcxyzpqrrrabbxyyyypqAzz + abxyzzpqrrrabbxyyyypqAzz + 0: abxyzzpqrrrabbxyyyypqAzz + aabxyzzzpqrrrabbxyyyypqAzz + 0: aabxyzzzpqrrrabbxyyyypqAzz + aaabxyzzzzpqrrrabbxyyyypqAzz + 0: aaabxyzzzzpqrrrabbxyyyypqAzz + aaaabxyzzzzpqrrrabbxyyyypqAzz + 0: aaaabxyzzzzpqrrrabbxyyyypqAzz + abcxyzzpqrrrabbxyyyypqAzz + 0: abcxyzzpqrrrabbxyyyypqAzz + aabcxyzzzpqrrrabbxyyyypqAzz + 0: aabcxyzzzpqrrrabbxyyyypqAzz + aaabcxyzzzzpqrrrabbxyyyypqAzz + 0: aaabcxyzzzzpqrrrabbxyyyypqAzz + aaaabcxyzzzzpqrrrabbxyyyypqAzz + 0: aaaabcxyzzzzpqrrrabbxyyyypqAzz + aaaabcxyzzzzpqrrrabbbxyyyypqAzz + 0: aaaabcxyzzzzpqrrrabbbxyyyypqAzz + aaaabcxyzzzzpqrrrabbbxyyyyypqAzz + 0: aaaabcxyzzzzpqrrrabbbxyyyyypqAzz + aaabcxyzpqrrrabbxyyyypABzz + 0: aaabcxyzpqrrrabbxyyyypABzz + aaabcxyzpqrrrabbxyyyypABBzz + 0: aaabcxyzpqrrrabbxyyyypABBzz + >>>aaabxyzpqrrrabbxyyyypqAzz + 0: aaabxyzpqrrrabbxyyyypqAzz + >aaaabxyzpqrrrabbxyyyypqAzz + 0: aaaabxyzpqrrrabbxyyyypqAzz + >>>>abcxyzpqrrrabbxyyyypqAzz + 0: abcxyzpqrrrabbxyyyypqAzz + *** Failers +No match + abxyzpqrrabbxyyyypqAzz +No match + abxyzpqrrrrabbxyyyypqAzz +No match + abxyzpqrrrabxyyyypqAzz +No match + aaaabcxyzzzzpqrrrabbbxyyyyyypqAzz +No match + aaaabcxyzzzzpqrrrabbbxyyypqAzz +No match + aaabcxyzpqrrrabbxyyyypqqqqqqqAzz +No match + +/^(abc){1,2}zz/ + abczz + 0: abczz + abcabczz + 0: abcabczz + *** Failers +No match + zz +No match + abcabcabczz +No match + >>abczz +No match + +/^(b+?|a){1,2}?c/ + bc + 0: bc + bbc + 0: bbc + bbbc + 0: bbbc + bac + 0: bac + bbac + 0: bbac + aac + 0: aac + abbbbbbbbbbbc + 0: abbbbbbbbbbbc + bbbbbbbbbbbac + 0: bbbbbbbbbbbac + *** Failers +No match + aaac +No match + abbbbbbbbbbbac +No match + +/^(b+|a){1,2}c/ + bc + 0: bc + bbc + 0: bbc + bbbc + 0: bbbc + bac + 0: bac + bbac + 0: bbac + aac + 0: aac + abbbbbbbbbbbc + 0: abbbbbbbbbbbc + bbbbbbbbbbbac + 0: bbbbbbbbbbbac + *** Failers +No match + aaac +No match + abbbbbbbbbbbac +No match + +/^(b+|a){1,2}?bc/ + bbc + 0: bbc + +/^(b*|ba){1,2}?bc/ + babc + 0: babc + bbabc + 0: bbabc + bababc + 0: bababc + *** Failers +No match + bababbc +No match + babababc +No match + +/^(ba|b*){1,2}?bc/ + babc + 0: babc + bbabc + 0: bbabc + bababc + 0: bababc + *** Failers +No match + bababbc +No match + babababc +No match + +/^\ca\cA\c[\c{\c:/ + \x01\x01\e;z + 0: \x01\x01\x1b;z + +/^[ab\]cde]/ + athing + 0: a + bthing + 0: b + ]thing + 0: ] + cthing + 0: c + dthing + 0: d + ething + 0: e + *** Failers +No match + fthing +No match + [thing +No match + \\thing +No match + +/^[]cde]/ + ]thing + 0: ] + cthing + 0: c + dthing + 0: d + ething + 0: e + *** Failers +No match + athing +No match + fthing +No match + +/^[^ab\]cde]/ + fthing + 0: f + [thing + 0: [ + \\thing + 0: \ + *** Failers + 0: * + athing +No match + bthing +No match + ]thing +No match + cthing +No match + dthing +No match + ething +No match + +/^[^]cde]/ + athing + 0: a + fthing + 0: f + *** Failers + 0: * + ]thing +No match + cthing +No match + dthing +No match + ething +No match + +/^\/ + + 0: \x81 + +/^ÿ/ + ÿ + 0: \xff + +/^[0-9]+$/ + 0 + 0: 0 + 1 + 0: 1 + 2 + 0: 2 + 3 + 0: 3 + 4 + 0: 4 + 5 + 0: 5 + 6 + 0: 6 + 7 + 0: 7 + 8 + 0: 8 + 9 + 0: 9 + 10 + 0: 10 + 100 + 0: 100 + *** Failers +No match + abc +No match + +/^.*nter/ + enter + 0: enter + inter + 0: inter + uponter + 0: uponter + +/^xxx[0-9]+$/ + xxx0 + 0: xxx0 + xxx1234 + 0: xxx1234 + *** Failers +No match + xxx +No match + +/^.+[0-9][0-9][0-9]$/ + x123 + 0: x123 + xx123 + 0: xx123 + 123456 + 0: 123456 + *** Failers +No match + 123 +No match + x1234 + 0: x1234 + +/^.+?[0-9][0-9][0-9]$/ + x123 + 0: x123 + xx123 + 0: xx123 + 123456 + 0: 123456 + *** Failers +No match + 123 +No match + x1234 + 0: x1234 + +/^([^!]+)!(.+)=apquxz\.ixr\.zzz\.ac\.uk$/ + abc!pqr=apquxz.ixr.zzz.ac.uk + 0: abc!pqr=apquxz.ixr.zzz.ac.uk + *** Failers +No match + !pqr=apquxz.ixr.zzz.ac.uk +No match + abc!=apquxz.ixr.zzz.ac.uk +No match + abc!pqr=apquxz:ixr.zzz.ac.uk +No match + abc!pqr=apquxz.ixr.zzz.ac.ukk +No match + +/:/ + Well, we need a colon: somewhere + 0: : + *** Fail if we don't +No match + +/([\da-f:]+)$/i + 0abc + 0: 0abc + abc + 0: abc + fed + 0: fed + E + 0: E + :: + 0: :: + 5f03:12C0::932e + 0: 5f03:12C0::932e + fed def + 0: def + Any old stuff + 0: ff + *** Failers +No match + 0zzz +No match + gzzz +No match + fed\x20 +No match + Any old rubbish +No match + +/^.*\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/ + .1.2.3 + 0: .1.2.3 + A.12.123.0 + 0: A.12.123.0 + *** Failers +No match + .1.2.3333 +No match + 1.2.3 +No match + 1234.2.3 +No match + +/^(\d+)\s+IN\s+SOA\s+(\S+)\s+(\S+)\s*\(\s*$/ + 1 IN SOA non-sp1 non-sp2( + 0: 1 IN SOA non-sp1 non-sp2( + 1 IN SOA non-sp1 non-sp2 ( + 0: 1 IN SOA non-sp1 non-sp2 ( + *** Failers +No match + 1IN SOA non-sp1 non-sp2( +No match + +/^[a-zA-Z\d][a-zA-Z\d\-]*(\.[a-zA-Z\d][a-zA-z\d\-]*)*\.$/ + a. + 0: a. + Z. + 0: Z. + 2. + 0: 2. + ab-c.pq-r. + 0: ab-c.pq-r. + sxk.zzz.ac.uk. + 0: sxk.zzz.ac.uk. + x-.y-. + 0: x-.y-. + *** Failers +No match + -abc.peq. +No match + +/^\*\.[a-z]([a-z\-\d]*[a-z\d]+)?(\.[a-z]([a-z\-\d]*[a-z\d]+)?)*$/ + *.a + 0: *.a + *.b0-a + 0: *.b0-a + *.c3-b.c + 0: *.c3-b.c + *.c-a.b-c + 0: *.c-a.b-c + *** Failers +No match + *.0 +No match + *.a- +No match + *.a-b.c- +No match + *.c-a.0-c +No match + +/^(?=ab(de))(abd)(e)/ + abde + 0: abde + +/^(?!(ab)de|x)(abd)(f)/ + abdf + 0: abdf + +/^(?=(ab(cd)))(ab)/ + abcd + 0: ab + +/^[\da-f](\.[\da-f])*$/i + a.b.c.d + 0: a.b.c.d + A.B.C.D + 0: A.B.C.D + a.b.c.1.2.3.C + 0: a.b.c.1.2.3.C + +/^\".*\"\s*(;.*)?$/ + \"1234\" + 0: "1234" + \"abcd\" ; + 0: "abcd" ; + \"\" ; rhubarb + 0: "" ; rhubarb + *** Failers +No match + \"1234\" : things +No match + +/^$/ + \ + 0: + *** Failers +No match + +/ ^ a (?# begins with a) b\sc (?# then b c) $ (?# then end)/x + ab c + 0: ab c + *** Failers +No match + abc +No match + ab cde +No match + +/(?x) ^ a (?# begins with a) b\sc (?# then b c) $ (?# then end)/ + ab c + 0: ab c + *** Failers +No match + abc +No match + ab cde +No match + +/^ a\ b[c ]d $/x + a bcd + 0: a bcd + a b d + 0: a b d + *** Failers +No match + abcd +No match + ab d +No match + +/^(a(b(c)))(d(e(f)))(h(i(j)))(k(l(m)))$/ + abcdefhijklm + 0: abcdefhijklm + +/^(?:a(b(c)))(?:d(e(f)))(?:h(i(j)))(?:k(l(m)))$/ + abcdefhijklm + 0: abcdefhijklm + +/^[\w][\W][\s][\S][\d][\D][\b][\n][\c]][\022]/ + a+ Z0+\x08\n\x1d\x12 + 0: a+ Z0+\x08\x0a\x1d\x12 + +/^[.^$|()*+?{,}]+/ + .^\$(*+)|{?,?} + 0: .^$(*+)|{?,?} + 1: .^$(*+)|{?,? + 2: .^$(*+)|{?, + 3: .^$(*+)|{? + 4: .^$(*+)|{ + 5: .^$(*+)| + 6: .^$(*+) + 7: .^$(*+ + 8: .^$(* + 9: .^$( +10: .^$ +11: .^ +12: . + +/^a*\w/ + z + 0: z + az + 0: az + 1: a + aaaz + 0: aaaz + 1: aaa + 2: aa + 3: a + a + 0: a + aa + 0: aa + 1: a + aaaa + 0: aaaa + 1: aaa + 2: aa + 3: a + a+ + 0: a + aa+ + 0: aa + 1: a + +/^a*?\w/ + z + 0: z + az + 0: az + 1: a + aaaz + 0: aaaz + 1: aaa + 2: aa + 3: a + a + 0: a + aa + 0: aa + 1: a + aaaa + 0: aaaa + 1: aaa + 2: aa + 3: a + a+ + 0: a + aa+ + 0: aa + 1: a + +/^a+\w/ + az + 0: az + aaaz + 0: aaaz + 1: aaa + 2: aa + aa + 0: aa + aaaa + 0: aaaa + 1: aaa + 2: aa + aa+ + 0: aa + +/^a+?\w/ + az + 0: az + aaaz + 0: aaaz + 1: aaa + 2: aa + aa + 0: aa + aaaa + 0: aaaa + 1: aaa + 2: aa + aa+ + 0: aa + +/^\d{8}\w{2,}/ + 1234567890 + 0: 1234567890 + 12345678ab + 0: 12345678ab + 12345678__ + 0: 12345678__ + *** Failers +No match + 1234567 +No match + +/^[aeiou\d]{4,5}$/ + uoie + 0: uoie + 1234 + 0: 1234 + 12345 + 0: 12345 + aaaaa + 0: aaaaa + *** Failers +No match + 123456 +No match + +/^[aeiou\d]{4,5}?/ + uoie + 0: uoie + 1234 + 0: 1234 + 12345 + 0: 12345 + 1: 1234 + aaaaa + 0: aaaaa + 1: aaaa + 123456 + 0: 12345 + 1: 1234 + +/^From +([^ ]+) +[a-zA-Z][a-zA-Z][a-zA-Z] +[a-zA-Z][a-zA-Z][a-zA-Z] +[0-9]?[0-9] +[0-9][0-9]:[0-9][0-9]/ + From abcd Mon Sep 01 12:33:02 1997 + 0: From abcd Mon Sep 01 12:33 + +/^From\s+\S+\s+([a-zA-Z]{3}\s+){2}\d{1,2}\s+\d\d:\d\d/ + From abcd Mon Sep 01 12:33:02 1997 + 0: From abcd Mon Sep 01 12:33 + From abcd Mon Sep 1 12:33:02 1997 + 0: From abcd Mon Sep 1 12:33 + *** Failers +No match + From abcd Sep 01 12:33:02 1997 +No match + +/^12.34/s + 12\n34 + 0: 12\x0a34 + 12\r34 + 0: 12\x0d34 + +/\w+(?=\t)/ + the quick brown\t fox + 0: brown + +/foo(?!bar)(.*)/ + foobar is foolish see? + 0: foolish see? + 1: foolish see + 2: foolish se + 3: foolish s + 4: foolish + 5: foolish + 6: foolis + 7: fooli + 8: fool + 9: foo + +/(?:(?!foo)...|^.{0,2})bar(.*)/ + foobar crowbar etc + 0: rowbar etc + 1: rowbar et + 2: rowbar e + 3: rowbar + 4: rowbar + barrel + 0: barrel + 1: barre + 2: barr + 3: bar + 2barrel + 0: 2barrel + 1: 2barre + 2: 2barr + 3: 2bar + A barrel + 0: A barrel + 1: A barre + 2: A barr + 3: A bar + +/^(\D*)(?=\d)(?!123)/ + abc456 + 0: abc + *** Failers +No match + abc123 +No match + +/^1234(?# test newlines + inside)/ + 1234 + 0: 1234 + +/^1234 #comment in extended re + /x + 1234 + 0: 1234 + +/#rhubarb + abcd/x + abcd + 0: abcd + +/^abcd#rhubarb/x + abcd + 0: abcd + +/(?!^)abc/ + the abc + 0: abc + *** Failers +No match + abc +No match + +/(?=^)abc/ + abc + 0: abc + *** Failers +No match + the abc +No match + +/^[ab]{1,3}(ab*|b)/ + aabbbbb + 0: aabbbbb + 1: aabbbb + 2: aabbb + 3: aabb + 4: aab + 5: aa + +/^[ab]{1,3}?(ab*|b)/ + aabbbbb + 0: aabbbbb + 1: aabbbb + 2: aabbb + 3: aabb + 4: aab + 5: aa + +/^[ab]{1,3}?(ab*?|b)/ + aabbbbb + 0: aabbbbb + 1: aabbbb + 2: aabbb + 3: aabb + 4: aab + 5: aa + +/^[ab]{1,3}(ab*?|b)/ + aabbbbb + 0: aabbbbb + 1: aabbbb + 2: aabbb + 3: aabb + 4: aab + 5: aa + +/ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional leading comment +(?: (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address +| # or +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # one word, optionally followed by.... +(?: +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] | # atom and space parts, or... +\( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) | # comments, or... + +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +# quoted strings +)* +< (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # leading < +(?: @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* + +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* , (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +)* # further okay, if led by comma +: # closing colon +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* )? # optional route +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) # initial word +(?: (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +" (?: # opening quote... +[^\\\x80-\xff\n\015"] # Anything except backslash and quote +| # or +\\ [^\x80-\xff] # Escaped something (something != CR) +)* " # closing quote +) )* # further okay, if led by a period +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* @ (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # initial subdomain +(?: # +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* \. # if led by a period... +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* (?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| \[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) # ...further okay +)* +# address spec +(?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* > # trailing > +# name and address +) (?: [\040\t] | \( +(?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] | \( (?: [^\\\x80-\xff\n\015()] | \\ [^\x80-\xff] )* \) )* +\) )* # optional trailing comment +/x + Alan Other <user\@dom.ain> + 0: Alan Other <user@dom.ain> + <user\@dom.ain> + 0: user@dom.ain + 1: user@dom + user\@dom.ain + 0: user@dom.ain + 1: user@dom + \"A. Other\" <user.1234\@dom.ain> (a comment) + 0: "A. Other" <user.1234@dom.ain> (a comment) + 1: "A. Other" <user.1234@dom.ain> + 2: "A. Other" <user.1234@dom.ain> + A. Other <user.1234\@dom.ain> (a comment) + 0: Other <user.1234@dom.ain> (a comment) + 1: Other <user.1234@dom.ain> + 2: Other <user.1234@dom.ain> + \"/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/\"\@x400-re.lay + 0: "/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/"@x400-re.lay + 1: "/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/"@x400-re + A missing angle <user\@some.where + 0: user@some.where + 1: user@some + *** Failers +No match + The quick brown fox +No match + +/[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional leading comment +(?: +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# additional words +)* +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +# address +| # or +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +# leading word +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] * # "normal" atoms and or spaces +(?: +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +| +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +) # "special" comment or quoted string +[^()<>@,;:".\\\[\]\x80-\xff\000-\010\012-\037] * # more "normal" +)* +< +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# < +(?: +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +(?: , +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +)* # additional domains +: +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)? # optional route +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +# Atom +| # or +" # " +[^\\\x80-\xff\n\015"] * # normal +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015"] * )* # ( special normal* )* +" # " +# Quoted string +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# additional words +)* +@ +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +(?: +\. +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +(?: +[^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]+ # some number of atom characters... +(?![^(\040)<>@,;:".\\\[\]\000-\037\x80-\xff]) # ..not followed by something that could be part of an atom +| +\[ # [ +(?: [^\\\x80-\xff\n\015\[\]] | \\ [^\x80-\xff] )* # stuff +\] # ] +) +[\040\t]* # Nab whitespace. +(?: +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: # ( +(?: \\ [^\x80-\xff] | +\( # ( +[^\\\x80-\xff\n\015()] * # normal* +(?: \\ [^\x80-\xff] [^\\\x80-\xff\n\015()] * )* # (special normal*)* +\) # ) +) # special +[^\\\x80-\xff\n\015()] * # normal* +)* # )* +\) # ) +[\040\t]* )* # If comment found, allow more spaces. +# optional trailing comments +)* +# address spec +> # > +# name and address +) +/x + Alan Other <user\@dom.ain> + 0: Alan Other <user@dom.ain> + <user\@dom.ain> + 0: user@dom.ain + 1: user@dom + user\@dom.ain + 0: user@dom.ain + 1: user@dom + \"A. Other\" <user.1234\@dom.ain> (a comment) + 0: "A. Other" <user.1234@dom.ain> + A. Other <user.1234\@dom.ain> (a comment) + 0: Other <user.1234@dom.ain> + \"/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/\"\@x400-re.lay + 0: "/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/"@x400-re.lay + 1: "/s=user/ou=host/o=place/prmd=uu.yy/admd= /c=gb/"@x400-re + A missing angle <user\@some.where + 0: user@some.where + 1: user@some + *** Failers +No match + The quick brown fox +No match + +/abc\0def\00pqr\000xyz\0000AB/ + abc\0def\00pqr\000xyz\0000AB + 0: abc\x00def\x00pqr\x00xyz\x000AB + abc456 abc\0def\00pqr\000xyz\0000ABCDE + 0: abc\x00def\x00pqr\x00xyz\x000AB + +/abc\x0def\x00pqr\x000xyz\x0000AB/ + abc\x0def\x00pqr\x000xyz\x0000AB + 0: abc\x0def\x00pqr\x000xyz\x0000AB + abc456 abc\x0def\x00pqr\x000xyz\x0000ABCDE + 0: abc\x0def\x00pqr\x000xyz\x0000AB + +/^[\000-\037]/ + \0A + 0: \x00 + \01B + 0: \x01 + \037C + 0: \x1f + +/\0*/ + \0\0\0\0 + 0: \x00\x00\x00\x00 + 1: \x00\x00\x00 + 2: \x00\x00 + 3: \x00 + 4: + +/A\x0{2,3}Z/ + The A\x0\x0Z + 0: A\x00\x00Z + An A\0\x0\0Z + 0: A\x00\x00\x00Z + *** Failers +No match + A\0Z +No match + A\0\x0\0\x0Z +No match + +/^\s/ + \040abc + 0: + \x0cabc + 0: \x0c + \nabc + 0: \x0a + \rabc + 0: \x0d + \tabc + 0: \x09 + *** Failers +No match + abc +No match + +/^a b +
c/x + abc + 0: abc + +/ab{1,3}bc/ + abbbbc + 0: abbbbc + abbbc + 0: abbbc + abbc + 0: abbc + *** Failers +No match + abc +No match + abbbbbc +No match + +/([^.]*)\.([^:]*):[T ]+(.*)/ + track1.title:TBlah blah blah + 0: track1.title:TBlah blah blah + 1: track1.title:TBlah blah bla + 2: track1.title:TBlah blah bl + 3: track1.title:TBlah blah b + 4: track1.title:TBlah blah + 5: track1.title:TBlah blah + 6: track1.title:TBlah bla + 7: track1.title:TBlah bl + 8: track1.title:TBlah b + 9: track1.title:TBlah +10: track1.title:TBlah +11: track1.title:TBla +12: track1.title:TBl +13: track1.title:TB +14: track1.title:T + +/([^.]*)\.([^:]*):[T ]+(.*)/i + track1.title:TBlah blah blah + 0: track1.title:TBlah blah blah + 1: track1.title:TBlah blah bla + 2: track1.title:TBlah blah bl + 3: track1.title:TBlah blah b + 4: track1.title:TBlah blah + 5: track1.title:TBlah blah + 6: track1.title:TBlah bla + 7: track1.title:TBlah bl + 8: track1.title:TBlah b + 9: track1.title:TBlah +10: track1.title:TBlah +11: track1.title:TBla +12: track1.title:TBl +13: track1.title:TB +14: track1.title:T + +/([^.]*)\.([^:]*):[t ]+(.*)/i + track1.title:TBlah blah blah + 0: track1.title:TBlah blah blah + 1: track1.title:TBlah blah bla + 2: track1.title:TBlah blah bl + 3: track1.title:TBlah blah b + 4: track1.title:TBlah blah + 5: track1.title:TBlah blah + 6: track1.title:TBlah bla + 7: track1.title:TBlah bl + 8: track1.title:TBlah b + 9: track1.title:TBlah +10: track1.title:TBlah +11: track1.title:TBla +12: track1.title:TBl +13: track1.title:TB +14: track1.title:T + +/^[W-c]+$/ + WXY_^abc + 0: WXY_^abc + *** Failers +No match + wxy +No match + +/^[W-c]+$/i + WXY_^abc + 0: WXY_^abc + wxy_^ABC + 0: wxy_^ABC + +/^[\x3f-\x5F]+$/i + WXY_^abc + 0: WXY_^abc + wxy_^ABC + 0: wxy_^ABC + +/^abc$/m + abc + 0: abc + qqq\nabc + 0: abc + abc\nzzz + 0: abc + qqq\nabc\nzzz + 0: abc + +/^abc$/ + abc + 0: abc + *** Failers +No match + qqq\nabc +No match + abc\nzzz +No match + qqq\nabc\nzzz +No match + +/\Aabc\Z/m + abc + 0: abc + abc\n + 0: abc + *** Failers +No match + qqq\nabc +No match + abc\nzzz +No match + qqq\nabc\nzzz +No match + +/\A(.)*\Z/s + abc\ndef + 0: abc\x0adef + +/\A(.)*\Z/m + *** Failers + 0: *** Failers + abc\ndef +No match + +/(?:b)|(?::+)/ + b::c + 0: b + c::b + 0: :: + 1: : + +/[-az]+/ + az- + 0: az- + 1: az + 2: a + *** Failers + 0: a + b +No match + +/[az-]+/ + za- + 0: za- + 1: za + 2: z + *** Failers + 0: a + b +No match + +/[a\-z]+/ + a-z + 0: a-z + 1: a- + 2: a + *** Failers + 0: a + b +No match + +/[a-z]+/ + abcdxyz + 0: abcdxyz + 1: abcdxy + 2: abcdx + 3: abcd + 4: abc + 5: ab + 6: a + +/[\d-]+/ + 12-34 + 0: 12-34 + 1: 12-3 + 2: 12- + 3: 12 + 4: 1 + *** Failers +No match + aaa +No match + +/[\d-z]+/ + 12-34z + 0: 12-34z + 1: 12-34 + 2: 12-3 + 3: 12- + 4: 12 + 5: 1 + *** Failers +No match + aaa +No match + +/\x5c/ + \\ + 0: \ + +/\x20Z/ + the Zoo + 0: Z + *** Failers +No match + Zulu +No match + +/ab{3cd/ + ab{3cd + 0: ab{3cd + +/ab{3,cd/ + ab{3,cd + 0: ab{3,cd + +/ab{3,4a}cd/ + ab{3,4a}cd + 0: ab{3,4a}cd + +/{4,5a}bc/ + {4,5a}bc + 0: {4,5a}bc + +/^a.b/ + a\rb + 0: a\x0db + *** Failers +No match + a\nb +No match + +/abc$/ + abc + 0: abc + abc\n + 0: abc + *** Failers +No match + abc\ndef +No match + +/(abc)\123/ + abc\x53 + 0: abcS + +/(abc)\223/ + abc\x93 + 0: abc\x93 + +/(abc)\323/ + abc\xd3 + 0: abc\xd3 + +/(abc)\500/ + abc\x40 + 0: abc@ + abc\100 + 0: abc@ + +/(abc)\5000/ + abc\x400 + 0: abc@0 + abc\x40\x30 + 0: abc@0 + abc\1000 + 0: abc@0 + abc\100\x30 + 0: abc@0 + abc\100\060 + 0: abc@0 + abc\100\60 + 0: abc@0 + +/abc\81/ + abc\081 + 0: abc\x0081 + abc\0\x38\x31 + 0: abc\x0081 + +/abc\91/ + abc\091 + 0: abc\x0091 + abc\0\x39\x31 + 0: abc\x0091 + +/(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)\12\123/ + abcdefghijk\12S + 0: abcdefghijk\x0aS + +/ab\gdef/ + abgdef + 0: abgdef + +/a{0}bc/ + bc + 0: bc + +/(a|(bc)){0,0}?xyz/ + xyz + 0: xyz + +/abc[\10]de/ + abc\010de + 0: abc\x08de + +/abc[\1]de/ + abc\1de + 0: abc\x01de + +/(abc)[\1]de/ + abc\1de + 0: abc\x01de + +/(?s)a.b/ + a\nb + 0: a\x0ab + +/^([^a])([^\b])([^c]*)([^d]{3,4})/ + baNOTccccd + 0: baNOTcccc + 1: baNOTccc + 2: baNOTcc + 3: baNOTc + 4: baNOT + baNOTcccd + 0: baNOTccc + 1: baNOTcc + 2: baNOTc + 3: baNOT + baNOTccd + 0: baNOTcc + 1: baNOTc + 2: baNOT + bacccd + 0: baccc + *** Failers + 0: *** Failers + 1: *** Failer + 2: *** Faile + 3: *** Fail + 4: *** Fai + 5: *** Fa + 6: *** F + anything +No match + b\bc +No match + baccd +No match + +/[^a]/ + Abc + 0: A + +/[^a]/i + Abc + 0: b + +/[^a]+/ + AAAaAbc + 0: AAA + 1: AA + 2: A + +/[^a]+/i + AAAaAbc + 0: bc + 1: b + +/[^a]+/ + bbb\nccc + 0: bbb\x0accc + 1: bbb\x0acc + 2: bbb\x0ac + 3: bbb\x0a + 4: bbb + 5: bb + 6: b + +/[^k]$/ + abc + 0: c + *** Failers + 0: s + abk +No match + +/[^k]{2,3}$/ + abc + 0: abc + kbc + 0: bc + kabc + 0: abc + *** Failers + 0: ers + abk +No match + akb +No match + akk +No match + +/^\d{8,}\@.+[^k]$/ + 12345678\@a.b.c.d + 0: 12345678@a.b.c.d + 123456789\@x.y.z + 0: 123456789@x.y.z + *** Failers +No match + 12345678\@x.y.uk +No match + 1234567\@a.b.c.d +No match + +/[^a]/ + aaaabcd + 0: b + aaAabcd + 0: A + +/[^a]/i + aaaabcd + 0: b + aaAabcd + 0: b + +/[^az]/ + aaaabcd + 0: b + aaAabcd + 0: A + +/[^az]/i + aaaabcd + 0: b + aaAabcd + 0: b + +/\000\001\002\003\004\005\006\007\010\011\012\013\014\015\016\017\020\021\022\023\024\025\026\027\030\031\032\033\034\035\036\037\040\041\042\043\044\045\046\047\050\051\052\053\054\055\056\057\060\061\062\063\064\065\066\067\070\071\072\073\074\075\076\077\100\101\102\103\104\105\106\107\110\111\112\113\114\115\116\117\120\121\122\123\124\125\126\127\130\131\132\133\134\135\136\137\140\141\142\143\144\145\146\147\150\151\152\153\154\155\156\157\160\161\162\163\164\165\166\167\170\171\172\173\174\175\176\177\200\201\202\203\204\205\206\207\210\211\212\213\214\215\216\217\220\221\222\223\224\225\226\227\230\231\232\233\234\235\236\237\240\241\242\243\244\245\246\247\250\251\252\253\254\255\256\257\260\261\262\263\264\265\266\267\270\271\272\273\274\275\276\277\300\301\302\303\304\305\306\307\310\311\312\313\314\315\316\317\320\321\322\323\324\325\326\327\330\331\332\333\334\335\336\337\340\341\342\343\344\345\346\347\350\351\352\353\354\355\356\357\360\361\362\363\364\365\366\367\370\371\372\373\374\375\376\377/ + \000\001\002\003\004\005\006\007\010\011\012\013\014\015\016\017\020\021\022\023\024\025\026\027\030\031\032\033\034\035\036\037\040\041\042\043\044\045\046\047\050\051\052\053\054\055\056\057\060\061\062\063\064\065\066\067\070\071\072\073\074\075\076\077\100\101\102\103\104\105\106\107\110\111\112\113\114\115\116\117\120\121\122\123\124\125\126\127\130\131\132\133\134\135\136\137\140\141\142\143\144\145\146\147\150\151\152\153\154\155\156\157\160\161\162\163\164\165\166\167\170\171\172\173\174\175\176\177\200\201\202\203\204\205\206\207\210\211\212\213\214\215\216\217\220\221\222\223\224\225\226\227\230\231\232\233\234\235\236\237\240\241\242\243\244\245\246\247\250\251\252\253\254\255\256\257\260\261\262\263\264\265\266\267\270\271\272\273\274\275\276\277\300\301\302\303\304\305\306\307\310\311\312\313\314\315\316\317\320\321\322\323\324\325\326\327\330\331\332\333\334\335\336\337\340\341\342\343\344\345\346\347\350\351\352\353\354\355\356\357\360\361\362\363\364\365\366\367\370\371\372\373\374\375\376\377 + 0: \x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff + +/P[^*]TAIRE[^*]{1,6}?LL/ + xxxxxxxxxxxPSTAIREISLLxxxxxxxxx + 0: PSTAIREISLL + +/P[^*]TAIRE[^*]{1,}?LL/ + xxxxxxxxxxxPSTAIREISLLxxxxxxxxx + 0: PSTAIREISLL + +/(\.\d\d[1-9]?)\d+/ + 1.230003938 + 0: .230003938 + 1: .23000393 + 2: .2300039 + 3: .230003 + 4: .23000 + 5: .2300 + 6: .230 + 1.875000282 + 0: .875000282 + 1: .87500028 + 2: .8750002 + 3: .875000 + 4: .87500 + 5: .8750 + 6: .875 + 1.235 + 0: .235 + +/(\.\d\d((?=0)|\d(?=\d)))/ + 1.230003938 + 0: .230 + 1: .23 + 1.875000282 + 0: .875 + *** Failers +No match + 1.235 +No match + +/a(?)b/ + ab + 0: ab + +/\b(foo)\s+(\w+)/i + Food is on the foo table + 0: foo table + 1: foo tabl + 2: foo tab + 3: foo ta + 4: foo t + +/foo(.*)bar/ + The food is under the bar in the barn. + 0: food is under the bar in the bar + 1: food is under the bar + +/foo(.*?)bar/ + The food is under the bar in the barn. + 0: food is under the bar in the bar + 1: food is under the bar + +/(.*)(\d*)/ + I have 2 numbers: 53147 +Matched, but too many subsidiary matches + 0: I have 2 numbers: 53147 + 1: I have 2 numbers: 5314 + 2: I have 2 numbers: 531 + 3: I have 2 numbers: 53 + 4: I have 2 numbers: 5 + 5: I have 2 numbers: + 6: I have 2 numbers: + 7: I have 2 numbers + 8: I have 2 number + 9: I have 2 numbe +10: I have 2 numb +11: I have 2 num +12: I have 2 nu +13: I have 2 n +14: I have 2 +15: I have 2 +16: I have +17: I have +18: I hav +19: I ha +20: I h +21: I + +/(.*)(\d+)/ + I have 2 numbers: 53147 + 0: I have 2 numbers: 53147 + 1: I have 2 numbers: 5314 + 2: I have 2 numbers: 531 + 3: I have 2 numbers: 53 + 4: I have 2 numbers: 5 + 5: I have 2 + +/(.*?)(\d*)/ + I have 2 numbers: 53147 +Matched, but too many subsidiary matches + 0: I have 2 numbers: 53147 + 1: I have 2 numbers: 5314 + 2: I have 2 numbers: 531 + 3: I have 2 numbers: 53 + 4: I have 2 numbers: 5 + 5: I have 2 numbers: + 6: I have 2 numbers: + 7: I have 2 numbers + 8: I have 2 number + 9: I have 2 numbe +10: I have 2 numb +11: I have 2 num +12: I have 2 nu +13: I have 2 n +14: I have 2 +15: I have 2 +16: I have +17: I have +18: I hav +19: I ha +20: I h +21: I + +/(.*?)(\d+)/ + I have 2 numbers: 53147 + 0: I have 2 numbers: 53147 + 1: I have 2 numbers: 5314 + 2: I have 2 numbers: 531 + 3: I have 2 numbers: 53 + 4: I have 2 numbers: 5 + 5: I have 2 + +/(.*)(\d+)$/ + I have 2 numbers: 53147 + 0: I have 2 numbers: 53147 + +/(.*?)(\d+)$/ + I have 2 numbers: 53147 + 0: I have 2 numbers: 53147 + +/(.*)\b(\d+)$/ + I have 2 numbers: 53147 + 0: I have 2 numbers: 53147 + +/(.*\D)(\d+)$/ + I have 2 numbers: 53147 + 0: I have 2 numbers: 53147 + +/^\D*(?!123)/ + ABC123 + 0: AB + 1: A + 2: + +/^(\D*)(?=\d)(?!123)/ + ABC445 + 0: ABC + *** Failers +No match + ABC123 +No match + +/^[W-]46]/ + W46]789 + 0: W46] + -46]789 + 0: -46] + *** Failers +No match + Wall +No match + Zebra +No match + 42 +No match + [abcd] +No match + ]abcd[ +No match + +/^[W-\]46]/ + W46]789 + 0: W + Wall + 0: W + Zebra + 0: Z + Xylophone + 0: X + 42 + 0: 4 + [abcd] + 0: [ + ]abcd[ + 0: ] + \\backslash + 0: \ + *** Failers +No match + -46]789 +No match + well +No match + +/\d\d\/\d\d\/\d\d\d\d/ + 01/01/2000 + 0: 01/01/2000 + +/word (?:[a-zA-Z0-9]+ ){0,10}otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark otherword + 0: word cat dog elephant mussel cow horse canary baboon snake shark otherword + word cat dog elephant mussel cow horse canary baboon snake shark +No match + +/word (?:[a-zA-Z0-9]+ ){0,300}otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark the quick brown fox and the lazy dog and several other words getting close to thirty by now I hope +No match + +/^(a){0,0}/ + bcd + 0: + abc + 0: + aab + 0: + +/^(a){0,1}/ + bcd + 0: + abc + 0: a + 1: + aab + 0: a + 1: + +/^(a){0,2}/ + bcd + 0: + abc + 0: a + 1: + aab + 0: aa + 1: a + 2: + +/^(a){0,3}/ + bcd + 0: + abc + 0: a + 1: + aab + 0: aa + 1: a + 2: + aaa + 0: aaa + 1: aa + 2: a + 3: + +/^(a){0,}/ + bcd + 0: + abc + 0: a + 1: + aab + 0: aa + 1: a + 2: + aaa + 0: aaa + 1: aa + 2: a + 3: + aaaaaaaa + 0: aaaaaaaa + 1: aaaaaaa + 2: aaaaaa + 3: aaaaa + 4: aaaa + 5: aaa + 6: aa + 7: a + 8: + +/^(a){1,1}/ + bcd +No match + abc + 0: a + aab + 0: a + +/^(a){1,2}/ + bcd +No match + abc + 0: a + aab + 0: aa + 1: a + +/^(a){1,3}/ + bcd +No match + abc + 0: a + aab + 0: aa + 1: a + aaa + 0: aaa + 1: aa + 2: a + +/^(a){1,}/ + bcd +No match + abc + 0: a + aab + 0: aa + 1: a + aaa + 0: aaa + 1: aa + 2: a + aaaaaaaa + 0: aaaaaaaa + 1: aaaaaaa + 2: aaaaaa + 3: aaaaa + 4: aaaa + 5: aaa + 6: aa + 7: a + +/.*\.gif/ + borfle\nbib.gif\nno + 0: bib.gif + +/.{0,}\.gif/ + borfle\nbib.gif\nno + 0: bib.gif + +/.*\.gif/m + borfle\nbib.gif\nno + 0: bib.gif + +/.*\.gif/s + borfle\nbib.gif\nno + 0: borfle\x0abib.gif + +/.*\.gif/ms + borfle\nbib.gif\nno + 0: borfle\x0abib.gif + +/.*$/ + borfle\nbib.gif\nno + 0: no + +/.*$/m + borfle\nbib.gif\nno + 0: borfle + +/.*$/s + borfle\nbib.gif\nno + 0: borfle\x0abib.gif\x0ano + +/.*$/ms + borfle\nbib.gif\nno + 0: borfle\x0abib.gif\x0ano + 1: borfle\x0abib.gif + 2: borfle + +/.*$/ + borfle\nbib.gif\nno\n + 0: no + +/.*$/m + borfle\nbib.gif\nno\n + 0: borfle + +/.*$/s + borfle\nbib.gif\nno\n + 0: borfle\x0abib.gif\x0ano\x0a + 1: borfle\x0abib.gif\x0ano + +/.*$/ms + borfle\nbib.gif\nno\n + 0: borfle\x0abib.gif\x0ano\x0a + 1: borfle\x0abib.gif\x0ano + 2: borfle\x0abib.gif + 3: borfle + +/(.*X|^B)/ + abcde\n1234Xyz + 0: 1234X + BarFoo + 0: B + *** Failers +No match + abcde\nBar +No match + +/(.*X|^B)/m + abcde\n1234Xyz + 0: 1234X + BarFoo + 0: B + abcde\nBar + 0: B + +/(.*X|^B)/s + abcde\n1234Xyz + 0: abcde\x0a1234X + BarFoo + 0: B + *** Failers +No match + abcde\nBar +No match + +/(.*X|^B)/ms + abcde\n1234Xyz + 0: abcde\x0a1234X + BarFoo + 0: B + abcde\nBar + 0: B + +/(?s)(.*X|^B)/ + abcde\n1234Xyz + 0: abcde\x0a1234X + BarFoo + 0: B + *** Failers +No match + abcde\nBar +No match + +/(?s:.*X|^B)/ + abcde\n1234Xyz + 0: abcde\x0a1234X + BarFoo + 0: B + *** Failers +No match + abcde\nBar +No match + +/^.*B/ + **** Failers +No match + abc\nB +No match + +/(?s)^.*B/ + abc\nB + 0: abc\x0aB + +/(?m)^.*B/ + abc\nB + 0: B + +/(?ms)^.*B/ + abc\nB + 0: abc\x0aB + +/(?ms)^B/ + abc\nB + 0: B + +/(?s)B$/ + B\n + 0: B + +/^[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]/ + 123456654321 + 0: 123456654321 + +/^\d\d\d\d\d\d\d\d\d\d\d\d/ + 123456654321 + 0: 123456654321 + +/^[\d][\d][\d][\d][\d][\d][\d][\d][\d][\d][\d][\d]/ + 123456654321 + 0: 123456654321 + +/^[abc]{12}/ + abcabcabcabc + 0: abcabcabcabc + +/^[a-c]{12}/ + abcabcabcabc + 0: abcabcabcabc + +/^(a|b|c){12}/ + abcabcabcabc + 0: abcabcabcabc + +/^[abcdefghijklmnopqrstuvwxy0123456789]/ + n + 0: n + *** Failers +No match + z +No match + +/abcde{0,0}/ + abcd + 0: abcd + *** Failers +No match + abce +No match + +/ab[cd]{0,0}e/ + abe + 0: abe + *** Failers +No match + abcde +No match + +/ab(c){0,0}d/ + abd + 0: abd + *** Failers +No match + abcd +No match + +/a(b*)/ + a + 0: a + ab + 0: ab + 1: a + abbbb + 0: abbbb + 1: abbb + 2: abb + 3: ab + 4: a + *** Failers + 0: a + bbbbb +No match + +/ab\d{0}e/ + abe + 0: abe + *** Failers +No match + ab1e +No match + +/"([^\\"]+|\\.)*"/ + the \"quick\" brown fox + 0: "quick" + \"the \\\"quick\\\" brown fox\" + 0: "the \"quick\" brown fox" + +/.*?/g+ + abc + 0: abc + 0+ + 1: ab + 2: a + 3: + 0: + 0+ + +/\b/g+ + abc + 0: + 0+ abc + 0: + 0+ + +/\b/+g + abc + 0: + 0+ abc + 0: + 0+ + +//g + abc + 0: + 0: + 0: + 0: + +/<tr([\w\W\s\d][^<>]{0,})><TD([\w\W\s\d][^<>]{0,})>([\d]{0,}\.)(.*)((<BR>([\w\W\s\d][^<>]{0,})|[\s]{0,}))<\/a><\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><TD([\w\W\s\d][^<>]{0,})>([\w\W\s\d][^<>]{0,})<\/TD><\/TR>/is + <TR BGCOLOR='#DBE9E9'><TD align=left valign=top>43.<a href='joblist.cfm?JobID=94 6735&Keyword='>Word Processor<BR>(N-1286)</a></TD><TD align=left valign=top>Lega lstaff.com</TD><TD align=left valign=top>CA - Statewide</TD></TR> + 0: <TR BGCOLOR='#DBE9E9'><TD align=left valign=top>43.<a href='joblist.cfm?JobID=94 6735&Keyword='>Word Processor<BR>(N-1286)</a></TD><TD align=left valign=top>Lega lstaff.com</TD><TD align=left valign=top>CA - Statewide</TD></TR> + +/a[^a]b/ + acb + 0: acb + a\nb + 0: a\x0ab + +/a.b/ + acb + 0: acb + *** Failers +No match + a\nb +No match + +/a[^a]b/s + acb + 0: acb + a\nb + 0: a\x0ab + +/a.b/s + acb + 0: acb + a\nb + 0: a\x0ab + +/^(b+?|a){1,2}?c/ + bac + 0: bac + bbac + 0: bbac + bbbac + 0: bbbac + bbbbac + 0: bbbbac + bbbbbac + 0: bbbbbac + +/^(b+|a){1,2}?c/ + bac + 0: bac + bbac + 0: bbac + bbbac + 0: bbbac + bbbbac + 0: bbbbac + bbbbbac + 0: bbbbbac + +/(?!\A)x/m + x\nb\n +No match + a\bx\n + 0: x + +/\x0{ab}/ + \0{ab} + 0: \x00{ab} + +/(A|B)*?CD/ + CD + 0: CD + +/(A|B)*CD/ + CD + 0: CD + +/(?<!bar)foo/ + foo + 0: foo + catfood + 0: foo + arfootle + 0: foo + rfoosh + 0: foo + *** Failers +No match + barfoo +No match + towbarfoo +No match + +/\w{3}(?<!bar)foo/ + catfood + 0: catfoo + *** Failers +No match + foo +No match + barfoo +No match + towbarfoo +No match + +/(?<=(foo)a)bar/ + fooabar + 0: bar + *** Failers +No match + bar +No match + foobbar +No match + +/\Aabc\z/m + abc + 0: abc + *** Failers +No match + abc\n +No match + qqq\nabc +No match + abc\nzzz +No match + qqq\nabc\nzzz +No match + +"(?>.*/)foo" + /this/is/a/very/long/line/in/deed/with/very/many/slashes/in/it/you/see/ +No match + +"(?>.*/)foo" + /this/is/a/very/long/line/in/deed/with/very/many/slashes/in/and/foo + 0: /this/is/a/very/long/line/in/deed/with/very/many/slashes/in/and/foo + +/(?>(\.\d\d[1-9]?))\d+/ + 1.230003938 + 0: .230003938 + 1: .23000393 + 2: .2300039 + 3: .230003 + 4: .23000 + 5: .2300 + 6: .230 + 1.875000282 + 0: .875000282 + 1: .87500028 + 2: .8750002 + 3: .875000 + 4: .87500 + 5: .8750 + *** Failers +No match + 1.235 +No match + +/^((?>\w+)|(?>\s+))*$/ + now is the time for all good men to come to the aid of the party + 0: now is the time for all good men to come to the aid of the party + *** Failers +No match + this is not a line with only words and spaces! +No match + +/(\d+)(\w)/ + 12345a + 0: 12345a + 1: 12345 + 2: 1234 + 3: 123 + 4: 12 + 12345+ + 0: 12345 + 1: 1234 + 2: 123 + 3: 12 + +/((?>\d+))(\w)/ + 12345a + 0: 12345a + *** Failers +No match + 12345+ +No match + +/(?>a+)b/ + aaab + 0: aaab + +/((?>a+)b)/ + aaab + 0: aaab + +/(?>(a+))b/ + aaab + 0: aaab + +/(?>b)+/ + aaabbbccc + 0: bbb + 1: bb + 2: b + +/(?>a+|b+|c+)*c/ + aaabbbbccccd + 0: aaabbbbcccc + 1: aaabbbbc + +/(a+|b+|c+)*c/ + aaabbbbccccd + 0: aaabbbbcccc + 1: aaabbbbccc + 2: aaabbbbcc + 3: aaabbbbc + +/((?>[^()]+)|\([^()]*\))+/ + ((abc(ade)ufh()()x + 0: abc(ade)ufh()()x + 1: abc(ade)ufh()() + 2: abc(ade)ufh() + 3: abc(ade)ufh + 4: abc(ade) + 5: abc + +/\(((?>[^()]+)|\([^()]+\))+\)/ + (abc) + 0: (abc) + (abc(def)xyz) + 0: (abc(def)xyz) + *** Failers +No match + ((()aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +No match + +/a(?-i)b/i + ab + 0: ab + Ab + 0: Ab + *** Failers +No match + aB +No match + AB +No match + +/(a (?x)b c)d e/ + a bcd e + 0: a bcd e + *** Failers +No match + a b cd e +No match + abcd e +No match + a bcde +No match + +/(a b(?x)c d (?-x)e f)/ + a bcde f + 0: a bcde f + *** Failers +No match + abcdef +No match + +/(a(?i)b)c/ + abc + 0: abc + aBc + 0: aBc + *** Failers +No match + abC +No match + aBC +No match + Abc +No match + ABc +No match + ABC +No match + AbC +No match + +/a(?i:b)c/ + abc + 0: abc + aBc + 0: aBc + *** Failers +No match + ABC +No match + abC +No match + aBC +No match + +/a(?i:b)*c/ + aBc + 0: aBc + aBBc + 0: aBBc + *** Failers +No match + aBC +No match + aBBC +No match + +/a(?=b(?i)c)\w\wd/ + abcd + 0: abcd + abCd + 0: abCd + *** Failers +No match + aBCd +No match + abcD +No match + +/(?s-i:more.*than).*million/i + more than million + 0: more than million + more than MILLION + 0: more than MILLION + more \n than Million + 0: more \x0a than Million + *** Failers +No match + MORE THAN MILLION +No match + more \n than \n million +No match + +/(?:(?s-i)more.*than).*million/i + more than million + 0: more than million + more than MILLION + 0: more than MILLION + more \n than Million + 0: more \x0a than Million + *** Failers +No match + MORE THAN MILLION +No match + more \n than \n million +No match + +/(?>a(?i)b+)+c/ + abc + 0: abc + aBbc + 0: aBbc + aBBc + 0: aBBc + *** Failers +No match + Abc +No match + abAb +No match + abbC +No match + +/(?=a(?i)b)\w\wc/ + abc + 0: abc + aBc + 0: aBc + *** Failers +No match + Ab +No match + abC +No match + aBC +No match + +/(?<=a(?i)b)(\w\w)c/ + abxxc + 0: xxc + aBxxc + 0: xxc + *** Failers +No match + Abxxc +No match + ABxxc +No match + abxxC +No match + +/^(?(?=abc)\w{3}:|\d\d)$/ + abc: + 0: abc: + 12 + 0: 12 + *** Failers +No match + 123 +No match + xyz +No match + +/^(?(?!abc)\d\d|\w{3}:)$/ + abc: + 0: abc: + 12 + 0: 12 + *** Failers +No match + 123 +No match + xyz +No match + +/(?(?<=foo)bar|cat)/ + foobar + 0: bar + cat + 0: cat + fcat + 0: cat + focat + 0: cat + *** Failers +No match + foocat +No match + +/(?(?<!foo)cat|bar)/ + foobar + 0: bar + cat + 0: cat + fcat + 0: cat + focat + 0: cat + *** Failers +No match + foocat +No match + +/(?>a*)*/ + a + 0: a + 1: + aa + 0: aa + 1: + aaaa + 0: aaaa + 1: + +/(abc|)+/ + abc + 0: abc + 1: + abcabc + 0: abcabc + 1: abc + 2: + abcabcabc + 0: abcabcabc + 1: abcabc + 2: abc + 3: + xyz + 0: + +/([a]*)*/ + a + 0: a + 1: + aaaaa + 0: aaaaa + 1: aaaa + 2: aaa + 3: aa + 4: a + 5: + +/([ab]*)*/ + a + 0: a + 1: + b + 0: b + 1: + ababab + 0: ababab + 1: ababa + 2: abab + 3: aba + 4: ab + 5: a + 6: + aaaabcde + 0: aaaab + 1: aaaa + 2: aaa + 3: aa + 4: a + 5: + bbbb + 0: bbbb + 1: bbb + 2: bb + 3: b + 4: + +/([^a]*)*/ + b + 0: b + 1: + bbbb + 0: bbbb + 1: bbb + 2: bb + 3: b + 4: + aaa + 0: + +/([^ab]*)*/ + cccc + 0: cccc + 1: ccc + 2: cc + 3: c + 4: + abab + 0: + +/([a]*?)*/ + a + 0: a + 1: + aaaa + 0: aaaa + 1: aaa + 2: aa + 3: a + 4: + +/([ab]*?)*/ + a + 0: a + 1: + b + 0: b + 1: + abab + 0: abab + 1: aba + 2: ab + 3: a + 4: + baba + 0: baba + 1: bab + 2: ba + 3: b + 4: + +/([^a]*?)*/ + b + 0: b + 1: + bbbb + 0: bbbb + 1: bbb + 2: bb + 3: b + 4: + aaa + 0: + +/([^ab]*?)*/ + c + 0: c + 1: + cccc + 0: cccc + 1: ccc + 2: cc + 3: c + 4: + baba + 0: + +/(?>a*)*/ + a + 0: a + 1: + aaabcde + 0: aaa + 1: + +/((?>a*))*/ + aaaaa + 0: aaaaa + 1: + aabbaa + 0: aa + 1: + +/((?>a*?))*/ + aaaaa + 0: aaaaa + 1: + aabbaa + 0: aa + 1: + +/(?(?=[^a-z]+[a-z]) \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} ) /x + 12-sep-98 + 0: 12-sep-98 + 12-09-98 + 0: 12-09-98 + *** Failers +No match + sep-12-98 +No match + +/(?i:saturday|sunday)/ + saturday + 0: saturday + sunday + 0: sunday + Saturday + 0: Saturday + Sunday + 0: Sunday + SATURDAY + 0: SATURDAY + SUNDAY + 0: SUNDAY + SunDay + 0: SunDay + +/(a(?i)bc|BB)x/ + abcx + 0: abcx + aBCx + 0: aBCx + bbx + 0: bbx + BBx + 0: BBx + *** Failers +No match + abcX +No match + aBCX +No match + bbX +No match + BBX +No match + +/^([ab](?i)[cd]|[ef])/ + ac + 0: ac + aC + 0: aC + bD + 0: bD + elephant + 0: e + Europe + 0: E + frog + 0: f + France + 0: F + *** Failers +No match + Africa +No match + +/^(ab|a(?i)[b-c](?m-i)d|x(?i)y|z)/ + ab + 0: ab + aBd + 0: aBd + xy + 0: xy + xY + 0: xY + zebra + 0: z + Zambesi + 0: Z + *** Failers +No match + aCD +No match + XY +No match + +/(?<=foo\n)^bar/m + foo\nbar + 0: bar + *** Failers +No match + bar +No match + baz\nbar +No match + +/(?<=(?<!foo)bar)baz/ + barbaz + 0: baz + barbarbaz + 0: baz + koobarbaz + 0: baz + *** Failers +No match + baz +No match + foobarbaz +No match + +/The following tests are taken from the Perl 5.005 test suite; some of them/ +/are compatible with 5.004, but I'd rather not have to sort them out./ +No match + +/abc/ + abc + 0: abc + xabcy + 0: abc + ababc + 0: abc + *** Failers +No match + xbc +No match + axc +No match + abx +No match + +/ab*c/ + abc + 0: abc + +/ab*bc/ + abc + 0: abc + abbc + 0: abbc + abbbbc + 0: abbbbc + +/.{1}/ + abbbbc + 0: a + +/.{3,4}/ + abbbbc + 0: abbb + 1: abb + +/ab{0,}bc/ + abbbbc + 0: abbbbc + +/ab+bc/ + abbc + 0: abbc + *** Failers +No match + abc +No match + abq +No match + +/ab{1,}bc/ + +/ab+bc/ + abbbbc + 0: abbbbc + +/ab{1,}bc/ + abbbbc + 0: abbbbc + +/ab{1,3}bc/ + abbbbc + 0: abbbbc + +/ab{3,4}bc/ + abbbbc + 0: abbbbc + +/ab{4,5}bc/ + *** Failers +No match + abq +No match + abbbbc +No match + +/ab?bc/ + abbc + 0: abbc + abc + 0: abc + +/ab{0,1}bc/ + abc + 0: abc + +/ab?bc/ + +/ab?c/ + abc + 0: abc + +/ab{0,1}c/ + abc + 0: abc + +/^abc$/ + abc + 0: abc + *** Failers +No match + abbbbc +No match + abcc +No match + +/^abc/ + abcc + 0: abc + +/^abc$/ + +/abc$/ + aabc + 0: abc + *** Failers +No match + aabc + 0: abc + aabcd +No match + +/^/ + abc + 0: + +/$/ + abc + 0: + +/a.c/ + abc + 0: abc + axc + 0: axc + +/a.*c/ + axyzc + 0: axyzc + +/a[bc]d/ + abd + 0: abd + *** Failers +No match + axyzd +No match + abc +No match + +/a[b-d]e/ + ace + 0: ace + +/a[b-d]/ + aac + 0: ac + +/a[-b]/ + a- + 0: a- + +/a[b-]/ + a- + 0: a- + +/a]/ + a] + 0: a] + +/a[]]b/ + a]b + 0: a]b + +/a[^bc]d/ + aed + 0: aed + *** Failers +No match + abd +No match + abd +No match + +/a[^-b]c/ + adc + 0: adc + +/a[^]b]c/ + adc + 0: adc + *** Failers +No match + a-c + 0: a-c + a]c +No match + +/\ba\b/ + a- + 0: a + -a + 0: a + -a- + 0: a + +/\by\b/ + *** Failers +No match + xy +No match + yz +No match + xyz +No match + +/\Ba\B/ + *** Failers + 0: a + a- +No match + -a +No match + -a- +No match + +/\By\b/ + xy + 0: y + +/\by\B/ + yz + 0: y + +/\By\B/ + xyz + 0: y + +/\w/ + a + 0: a + +/\W/ + - + 0: - + *** Failers + 0: * + - + 0: - + a +No match + +/a\sb/ + a b + 0: a b + +/a\Sb/ + a-b + 0: a-b + *** Failers +No match + a-b + 0: a-b + a b +No match + +/\d/ + 1 + 0: 1 + +/\D/ + - + 0: - + *** Failers + 0: * + - + 0: - + 1 +No match + +/[\w]/ + a + 0: a + +/[\W]/ + - + 0: - + *** Failers + 0: * + - + 0: - + a +No match + +/a[\s]b/ + a b + 0: a b + +/a[\S]b/ + a-b + 0: a-b + *** Failers +No match + a-b + 0: a-b + a b +No match + +/[\d]/ + 1 + 0: 1 + +/[\D]/ + - + 0: - + *** Failers + 0: * + - + 0: - + 1 +No match + +/ab|cd/ + abc + 0: ab + abcd + 0: ab + +/()ef/ + def + 0: ef + +/$b/ + +/a\(b/ + a(b + 0: a(b + +/a\(*b/ + ab + 0: ab + a((b + 0: a((b + +/a\\b/ + a\b +No match + +/((a))/ + abc + 0: a + +/(a)b(c)/ + abc + 0: abc + +/a+b+c/ + aabbabc + 0: abc + +/a{1,}b{1,}c/ + aabbabc + 0: abc + +/a.+?c/ + abcabc + 0: abcabc + 1: abc + +/(a+|b)*/ + ab + 0: ab + 1: a + 2: + +/(a+|b){0,}/ + ab + 0: ab + 1: a + 2: + +/(a+|b)+/ + ab + 0: ab + 1: a + +/(a+|b){1,}/ + ab + 0: ab + 1: a + +/(a+|b)?/ + ab + 0: a + 1: + +/(a+|b){0,1}/ + ab + 0: a + 1: + +/[^ab]*/ + cde + 0: cde + 1: cd + 2: c + 3: + +/abc/ + *** Failers +No match + b +No match + + +/a*/ + + +/([abc])*d/ + abbbcd + 0: abbbcd + +/([abc])*bcd/ + abcd + 0: abcd + +/a|b|c|d|e/ + e + 0: e + +/(a|b|c|d|e)f/ + ef + 0: ef + +/abcd*efg/ + abcdefg + 0: abcdefg + +/ab*/ + xabyabbbz + 0: ab + 1: a + xayabbbz + 0: a + +/(ab|cd)e/ + abcde + 0: cde + +/[abhgefdc]ij/ + hij + 0: hij + +/^(ab|cd)e/ + +/(abc|)ef/ + abcdef + 0: ef + +/(a|b)c*d/ + abcd + 0: bcd + +/(ab|ab*)bc/ + abc + 0: abc + +/a([bc]*)c*/ + abc + 0: abc + 1: ab + 2: a + +/a([bc]*)(c*d)/ + abcd + 0: abcd + +/a([bc]+)(c*d)/ + abcd + 0: abcd + +/a([bc]*)(c+d)/ + abcd + 0: abcd + +/a[bcd]*dcdcde/ + adcdcde + 0: adcdcde + +/a[bcd]+dcdcde/ + *** Failers +No match + abcde +No match + adcdcde +No match + +/(ab|a)b*c/ + abc + 0: abc + +/((a)(b)c)(d)/ + abcd + 0: abcd + +/[a-zA-Z_][a-zA-Z0-9_]*/ + alpha + 0: alpha + 1: alph + 2: alp + 3: al + 4: a + +/^a(bc+|b[eh])g|.h$/ + abh + 0: bh + +/(bc+d$|ef*g.|h?i(j|k))/ + effgz + 0: effgz + ij + 0: ij + reffgz + 0: effgz + *** Failers +No match + effg +No match + bcdd +No match + +/((((((((((a))))))))))/ + a + 0: a + +/(((((((((a)))))))))/ + a + 0: a + +/multiple words of text/ + *** Failers +No match + aa +No match + uh-uh +No match + +/multiple words/ + multiple words, yeah + 0: multiple words + +/(.*)c(.*)/ + abcde + 0: abcde + 1: abcd + 2: abc + +/\((.*), (.*)\)/ + (a, b) + 0: (a, b) + +/[k]/ + +/abcd/ + abcd + 0: abcd + +/a(bc)d/ + abcd + 0: abcd + +/a[-]?c/ + ac + 0: ac + +/abc/i + ABC + 0: ABC + XABCY + 0: ABC + ABABC + 0: ABC + *** Failers +No match + aaxabxbaxbbx +No match + XBC +No match + AXC +No match + ABX +No match + +/ab*c/i + ABC + 0: ABC + +/ab*bc/i + ABC + 0: ABC + ABBC + 0: ABBC + +/ab*?bc/i + ABBBBC + 0: ABBBBC + +/ab{0,}?bc/i + ABBBBC + 0: ABBBBC + +/ab+?bc/i + ABBC + 0: ABBC + +/ab+bc/i + *** Failers +No match + ABC +No match + ABQ +No match + +/ab{1,}bc/i + +/ab+bc/i + ABBBBC + 0: ABBBBC + +/ab{1,}?bc/i + ABBBBC + 0: ABBBBC + +/ab{1,3}?bc/i + ABBBBC + 0: ABBBBC + +/ab{3,4}?bc/i + ABBBBC + 0: ABBBBC + +/ab{4,5}?bc/i + *** Failers +No match + ABQ +No match + ABBBBC +No match + +/ab??bc/i + ABBC + 0: ABBC + ABC + 0: ABC + +/ab{0,1}?bc/i + ABC + 0: ABC + +/ab??bc/i + +/ab??c/i + ABC + 0: ABC + +/ab{0,1}?c/i + ABC + 0: ABC + +/^abc$/i + ABC + 0: ABC + *** Failers +No match + ABBBBC +No match + ABCC +No match + +/^abc/i + ABCC + 0: ABC + +/^abc$/i + +/abc$/i + AABC + 0: ABC + +/^/i + ABC + 0: + +/$/i + ABC + 0: + +/a.c/i + ABC + 0: ABC + AXC + 0: AXC + +/a.*?c/i + AXYZC + 0: AXYZC + +/a.*c/i + *** Failers +No match + AABC + 0: AABC + AXYZD +No match + +/a[bc]d/i + ABD + 0: ABD + +/a[b-d]e/i + ACE + 0: ACE + *** Failers +No match + ABC +No match + ABD +No match + +/a[b-d]/i + AAC + 0: AC + +/a[-b]/i + A- + 0: A- + +/a[b-]/i + A- + 0: A- + +/a]/i + A] + 0: A] + +/a[]]b/i + A]B + 0: A]B + +/a[^bc]d/i + AED + 0: AED + +/a[^-b]c/i + ADC + 0: ADC + *** Failers +No match + ABD +No match + A-C +No match + +/a[^]b]c/i + ADC + 0: ADC + +/ab|cd/i + ABC + 0: AB + ABCD + 0: AB + +/()ef/i + DEF + 0: EF + +/$b/i + *** Failers +No match + A]C +No match + B +No match + +/a\(b/i + A(B + 0: A(B + +/a\(*b/i + AB + 0: AB + A((B + 0: A((B + +/a\\b/i + A\B +No match + +/((a))/i + ABC + 0: A + +/(a)b(c)/i + ABC + 0: ABC + +/a+b+c/i + AABBABC + 0: ABC + +/a{1,}b{1,}c/i + AABBABC + 0: ABC + +/a.+?c/i + ABCABC + 0: ABCABC + 1: ABC + +/a.*?c/i + ABCABC + 0: ABCABC + 1: ABC + +/a.{0,5}?c/i + ABCABC + 0: ABCABC + 1: ABC + +/(a+|b)*/i + AB + 0: AB + 1: A + 2: + +/(a+|b){0,}/i + AB + 0: AB + 1: A + 2: + +/(a+|b)+/i + AB + 0: AB + 1: A + +/(a+|b){1,}/i + AB + 0: AB + 1: A + +/(a+|b)?/i + AB + 0: A + 1: + +/(a+|b){0,1}/i + AB + 0: A + 1: + +/(a+|b){0,1}?/i + AB + 0: A + 1: + +/[^ab]*/i + CDE + 0: CDE + 1: CD + 2: C + 3: + +/abc/i + +/a*/i + + +/([abc])*d/i + ABBBCD + 0: ABBBCD + +/([abc])*bcd/i + ABCD + 0: ABCD + +/a|b|c|d|e/i + E + 0: E + +/(a|b|c|d|e)f/i + EF + 0: EF + +/abcd*efg/i + ABCDEFG + 0: ABCDEFG + +/ab*/i + XABYABBBZ + 0: AB + 1: A + XAYABBBZ + 0: A + +/(ab|cd)e/i + ABCDE + 0: CDE + +/[abhgefdc]ij/i + HIJ + 0: HIJ + +/^(ab|cd)e/i + ABCDE +No match + +/(abc|)ef/i + ABCDEF + 0: EF + +/(a|b)c*d/i + ABCD + 0: BCD + +/(ab|ab*)bc/i + ABC + 0: ABC + +/a([bc]*)c*/i + ABC + 0: ABC + 1: AB + 2: A + +/a([bc]*)(c*d)/i + ABCD + 0: ABCD + +/a([bc]+)(c*d)/i + ABCD + 0: ABCD + +/a([bc]*)(c+d)/i + ABCD + 0: ABCD + +/a[bcd]*dcdcde/i + ADCDCDE + 0: ADCDCDE + +/a[bcd]+dcdcde/i + +/(ab|a)b*c/i + ABC + 0: ABC + +/((a)(b)c)(d)/i + ABCD + 0: ABCD + +/[a-zA-Z_][a-zA-Z0-9_]*/i + ALPHA + 0: ALPHA + 1: ALPH + 2: ALP + 3: AL + 4: A + +/^a(bc+|b[eh])g|.h$/i + ABH + 0: BH + +/(bc+d$|ef*g.|h?i(j|k))/i + EFFGZ + 0: EFFGZ + IJ + 0: IJ + REFFGZ + 0: EFFGZ + *** Failers +No match + ADCDCDE +No match + EFFG +No match + BCDD +No match + +/((((((((((a))))))))))/i + A + 0: A + +/(((((((((a)))))))))/i + A + 0: A + +/(?:(?:(?:(?:(?:(?:(?:(?:(?:(a))))))))))/i + A + 0: A + +/(?:(?:(?:(?:(?:(?:(?:(?:(?:(a|b|c))))))))))/i + C + 0: C + +/multiple words of text/i + *** Failers +No match + AA +No match + UH-UH +No match + +/multiple words/i + MULTIPLE WORDS, YEAH + 0: MULTIPLE WORDS + +/(.*)c(.*)/i + ABCDE + 0: ABCDE + 1: ABCD + 2: ABC + +/\((.*), (.*)\)/i + (A, B) + 0: (A, B) + +/[k]/i + +/abcd/i + ABCD + 0: ABCD + +/a(bc)d/i + ABCD + 0: ABCD + +/a[-]?c/i + AC + 0: AC + +/a(?!b)./ + abad + 0: ad + +/a(?=d)./ + abad + 0: ad + +/a(?=c|d)./ + abad + 0: ad + +/a(?:b|c|d)(.)/ + ace + 0: ace + +/a(?:b|c|d)*(.)/ + ace + 0: ace + 1: ac + +/a(?:b|c|d)+?(.)/ + ace + 0: ace + acdbcdbe + 0: acdbcdbe + 1: acdbcdb + 2: acdbcd + 3: acdbc + 4: acdb + 5: acd + +/a(?:b|c|d)+(.)/ + acdbcdbe + 0: acdbcdbe + 1: acdbcdb + 2: acdbcd + 3: acdbc + 4: acdb + 5: acd + +/a(?:b|c|d){2}(.)/ + acdbcdbe + 0: acdb + +/a(?:b|c|d){4,5}(.)/ + acdbcdbe + 0: acdbcdb + 1: acdbcd + +/a(?:b|c|d){4,5}?(.)/ + acdbcdbe + 0: acdbcdb + 1: acdbcd + +/((foo)|(bar))*/ + foobar + 0: foobar + 1: foo + 2: + +/a(?:b|c|d){6,7}(.)/ + acdbcdbe + 0: acdbcdbe + +/a(?:b|c|d){6,7}?(.)/ + acdbcdbe + 0: acdbcdbe + +/a(?:b|c|d){5,6}(.)/ + acdbcdbe + 0: acdbcdbe + 1: acdbcdb + +/a(?:b|c|d){5,6}?(.)/ + acdbcdbe + 0: acdbcdbe + 1: acdbcdb + +/a(?:b|c|d){5,7}(.)/ + acdbcdbe + 0: acdbcdbe + 1: acdbcdb + +/a(?:b|c|d){5,7}?(.)/ + acdbcdbe + 0: acdbcdbe + 1: acdbcdb + +/a(?:b|(c|e){1,2}?|d)+?(.)/ + ace + 0: ace + +/^(.+)?B/ + AB + 0: AB + +/^([^a-z])|(\^)$/ + . + 0: . + +/^[<>]&/ + <&OUT + 0: <& + +/(?:(f)(o)(o)|(b)(a)(r))*/ + foobar + 0: foobar + 1: foo + 2: + +/(?<=a)b/ + ab + 0: b + *** Failers +No match + cb +No match + b +No match + +/(?<!c)b/ + ab + 0: b + b + 0: b + b + 0: b + +/(?:..)*a/ + aba + 0: aba + 1: a + +/(?:..)*?a/ + aba + 0: aba + 1: a + +/^(){3,5}/ + abc + 0: + +/^(a+)*ax/ + aax + 0: aax + +/^((a|b)+)*ax/ + aax + 0: aax + +/^((a|bc)+)*ax/ + aax + 0: aax + +/(a|x)*ab/ + cab + 0: ab + +/(a)*ab/ + cab + 0: ab + +/(?:(?i)a)b/ + ab + 0: ab + +/((?i)a)b/ + ab + 0: ab + +/(?:(?i)a)b/ + Ab + 0: Ab + +/((?i)a)b/ + Ab + 0: Ab + +/(?:(?i)a)b/ + *** Failers +No match + cb +No match + aB +No match + +/((?i)a)b/ + +/(?i:a)b/ + ab + 0: ab + +/((?i:a))b/ + ab + 0: ab + +/(?i:a)b/ + Ab + 0: Ab + +/((?i:a))b/ + Ab + 0: Ab + +/(?i:a)b/ + *** Failers +No match + aB +No match + aB +No match + +/((?i:a))b/ + +/(?:(?-i)a)b/i + ab + 0: ab + +/((?-i)a)b/i + ab + 0: ab + +/(?:(?-i)a)b/i + aB + 0: aB + +/((?-i)a)b/i + aB + 0: aB + +/(?:(?-i)a)b/i + *** Failers +No match + aB + 0: aB + Ab +No match + +/((?-i)a)b/i + +/(?:(?-i)a)b/i + aB + 0: aB + +/((?-i)a)b/i + aB + 0: aB + +/(?:(?-i)a)b/i + *** Failers +No match + Ab +No match + AB +No match + +/((?-i)a)b/i + +/(?-i:a)b/i + ab + 0: ab + +/((?-i:a))b/i + ab + 0: ab + +/(?-i:a)b/i + aB + 0: aB + +/((?-i:a))b/i + aB + 0: aB + +/(?-i:a)b/i + *** Failers +No match + AB +No match + Ab +No match + +/((?-i:a))b/i + +/(?-i:a)b/i + aB + 0: aB + +/((?-i:a))b/i + aB + 0: aB + +/(?-i:a)b/i + *** Failers +No match + Ab +No match + AB +No match + +/((?-i:a))b/i + +/((?-i:a.))b/i + *** Failers +No match + AB +No match + a\nB +No match + +/((?s-i:a.))b/i + a\nB + 0: a\x0aB + +/(?:c|d)(?:)(?:a(?:)(?:b)(?:b(?:))(?:b(?:)(?:b)))/ + cabbbb + 0: cabbbb + +/(?:c|d)(?:)(?:aaaaaaaa(?:)(?:bbbbbbbb)(?:bbbbbbbb(?:))(?:bbbbbbbb(?:)(?:bbbbbbbb)))/ + caaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + 0: caaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + +/foo\w*\d{4}baz/ + foobar1234baz + 0: foobar1234baz + +/x(~~)*(?:(?:F)?)?/ + x~~ + 0: x~~ + 1: x + +/^a(?#xxx){3}c/ + aaac + 0: aaac + +/^a (?#xxx) (?#yyy) {3}c/x + aaac + 0: aaac + +/(?<![cd])b/ + *** Failers +No match + B\nB +No match + dbcb +No match + +/(?<![cd])[ab]/ + dbaacb + 0: a + +/(?<!(c|d))b/ + +/(?<!(c|d))[ab]/ + dbaacb + 0: a + +/(?<!cd)[ab]/ + cdaccb + 0: b + +/^(?:a?b?)*$/ + *** Failers +No match + dbcb +No match + a-- +No match + +/((?s)^a(.))((?m)^b$)/ + a\nb\nc\n + 0: a\x0ab + +/((?m)^b$)/ + a\nb\nc\n + 0: b + +/(?m)^b/ + a\nb\n + 0: b + +/(?m)^(b)/ + a\nb\n + 0: b + +/((?m)^b)/ + a\nb\n + 0: b + +/\n((?m)^b)/ + a\nb\n + 0: \x0ab + +/((?s).)c(?!.)/ + a\nb\nc\n + 0: \x0ac + a\nb\nc\n + 0: \x0ac + +/((?s)b.)c(?!.)/ + a\nb\nc\n + 0: b\x0ac + a\nb\nc\n + 0: b\x0ac + +/^b/ + +/()^b/ + *** Failers +No match + a\nb\nc\n +No match + a\nb\nc\n +No match + +/((?m)^b)/ + a\nb\nc\n + 0: b + +/(?(?!a)a|b)/ + +/(?(?!a)b|a)/ + a + 0: a + +/(?(?=a)b|a)/ + *** Failers +No match + a +No match + a +No match + +/(?(?=a)a|b)/ + a + 0: a + +/(\w+:)+/ + one: + 0: one: + +/$(?<=^(a))/ + a + 0: + +/([\w:]+::)?(\w+)$/ + abcd + 0: abcd + xy:z:::abcd + 0: xy:z:::abcd + +/^[^bcd]*(c+)/ + aexycd + 0: aexyc + +/(a*)b+/ + caab + 0: aab + +/([\w:]+::)?(\w+)$/ + abcd + 0: abcd + xy:z:::abcd + 0: xy:z:::abcd + *** Failers + 0: Failers + abcd: +No match + abcd: +No match + +/^[^bcd]*(c+)/ + aexycd + 0: aexyc + +/(>a+)ab/ + +/(?>a+)b/ + aaab + 0: aaab + +/([[:]+)/ + a:[b]: + 0: :[ + 1: : + +/([[=]+)/ + a=[b]= + 0: =[ + 1: = + +/([[.]+)/ + a.[b]. + 0: .[ + 1: . + +/((?>a+)b)/ + aaab + 0: aaab + +/(?>(a+))b/ + aaab + 0: aaab + +/((?>[^()]+)|\([^()]*\))+/ + ((abc(ade)ufh()()x + 0: abc(ade)ufh()()x + 1: abc(ade)ufh()() + 2: abc(ade)ufh() + 3: abc(ade)ufh + 4: abc(ade) + 5: abc + +/a\Z/ + *** Failers +No match + aaab +No match + a\nb\n +No match + +/b\Z/ + a\nb\n + 0: b + +/b\z/ + +/b\Z/ + a\nb + 0: b + +/b\z/ + a\nb + 0: b + *** Failers +No match + +/(?>.*)(?<=(abcd|wxyz))/ + alphabetabcd + 0: alphabetabcd + endingwxyz + 0: endingwxyz + *** Failers +No match + a rather long string that doesn't end with one of them +No match + +/word (?>(?:(?!otherword)[a-zA-Z0-9]+ ){0,30})otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark otherword + 0: word cat dog elephant mussel cow horse canary baboon snake shark otherword + word cat dog elephant mussel cow horse canary baboon snake shark +No match + +/word (?>[a-zA-Z0-9]+ ){0,30}otherword/ + word cat dog elephant mussel cow horse canary baboon snake shark the quick brown fox and the lazy dog and several other words getting close to thirty by now I hope +No match + +/(?<=\d{3}(?!999))foo/ + 999foo + 0: foo + 123999foo + 0: foo + *** Failers +No match + 123abcfoo +No match + +/(?<=(?!...999)\d{3})foo/ + 999foo + 0: foo + 123999foo + 0: foo + *** Failers +No match + 123abcfoo +No match + +/(?<=\d{3}(?!999)...)foo/ + 123abcfoo + 0: foo + 123456foo + 0: foo + *** Failers +No match + 123999foo +No match + +/(?<=\d{3}...)(?<!999)foo/ + 123abcfoo + 0: foo + 123456foo + 0: foo + *** Failers +No match + 123999foo +No match + +/((Z)+|A)*/ + ZABCDEFG + 0: ZA + 1: Z + 2: + +/(Z()|A)*/ + ZABCDEFG + 0: ZA + 1: Z + 2: + +/(Z(())|A)*/ + ZABCDEFG + 0: ZA + 1: Z + 2: + +/((?>Z)+|A)*/ + ZABCDEFG + 0: ZA + 1: Z + 2: + +/((?>)+|A)*/ + ZABCDEFG + 0: + +/a*/g + abbab + 0: a + 1: + 0: + 0: + 0: a + 1: + 0: + 0: + +/^[a-\d]/ + abcde + 0: a + -things + 0: - + 0digit + 0: 0 + *** Failers +No match + bcdef +No match + +/^[\d-a]/ + abcde + 0: a + -things + 0: - + 0digit + 0: 0 + *** Failers +No match + bcdef +No match + +/[[:space:]]+/ + > \x09\x0a\x0c\x0d\x0b< + 0: \x09\x0a\x0c\x0d\x0b + 1: \x09\x0a\x0c\x0d + 2: \x09\x0a\x0c + 3: \x09\x0a + 4: \x09 + 5: + +/[[:blank:]]+/ + > \x09\x0a\x0c\x0d\x0b< + 0: \x09 + 1: + +/[\s]+/ + > \x09\x0a\x0c\x0d\x0b< + 0: \x09\x0a\x0c\x0d + 1: \x09\x0a\x0c + 2: \x09\x0a + 3: \x09 + 4: + +/\s+/ + > \x09\x0a\x0c\x0d\x0b< + 0: \x09\x0a\x0c\x0d + 1: \x09\x0a\x0c + 2: \x09\x0a + 3: \x09 + 4: + +/a b/x + ab +No match + +/(?!\A)x/m + a\nxb\n + 0: x + +/(?!^)x/m + a\nxb\n +No match + +/abc\Qabc\Eabc/ + abcabcabc + 0: abcabcabc + +/abc\Q(*+|\Eabc/ + abc(*+|abc + 0: abc(*+|abc + +/ abc\Q abc\Eabc/x + abc abcabc + 0: abc abcabc + *** Failers +No match + abcabcabc +No match + +/abc#comment + \Q#not comment + literal\E/x + abc#not comment\n literal + 0: abc#not comment\x0a literal + +/abc#comment + \Q#not comment + literal/x + abc#not comment\n literal + 0: abc#not comment\x0a literal + +/abc#comment + \Q#not comment + literal\E #more comment + /x + abc#not comment\n literal + 0: abc#not comment\x0a literal + +/abc#comment + \Q#not comment + literal\E #more comment/x + abc#not comment\n literal + 0: abc#not comment\x0a literal + +/\Qabc\$xyz\E/ + abc\\\$xyz + 0: abc\$xyz + +/\Qabc\E\$\Qxyz\E/ + abc\$xyz + 0: abc$xyz + +/\Gabc/ + abc + 0: abc + *** Failers +No match + xyzabc +No match + +/\Gabc./g + abc1abc2xyzabc3 + 0: abc1 + 0: abc2 + +/abc./g + abc1abc2xyzabc3 + 0: abc1 + 0: abc2 + 0: abc3 + +/a(?x: b c )d/ + XabcdY + 0: abcd + *** Failers +No match + Xa b c d Y +No match + +/((?x)x y z | a b c)/ + XabcY + 0: abc + AxyzB + 0: xyz + +/(?i)AB(?-i)C/ + XabCY + 0: abC + *** Failers +No match + XabcY +No match + +/((?i)AB(?-i)C|D)E/ + abCE + 0: abCE + DE + 0: DE + *** Failers +No match + abcE +No match + abCe +No match + dE +No match + De +No match + +/[z\Qa-d]\E]/ + z + 0: z + a + 0: a + - + 0: - + d + 0: d + ] + 0: ] + *** Failers + 0: a + b +No match + +/[\z\C]/ + z + 0: z + C + 0: C + +/\M/ + M + 0: M + +/(a+)*b/ + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +No match + +/(?i)reg(?:ul(?:[aä]|ae)r|ex)/ + REGular + 0: REGular + regulaer + 0: regulaer + Regex + 0: Regex + regulär + 0: regul\xe4r + +/Åæåä[à-ÿÀ-ß]+/ + Åæåäà + 0: \xc5\xe6\xe5\xe4\xe0 + Åæåäÿ + 0: \xc5\xe6\xe5\xe4\xff + ÅæåäÀ + 0: \xc5\xe6\xe5\xe4\xc0 + Åæåäß + 0: \xc5\xe6\xe5\xe4\xdf + +/(?<=Z)X./ + \x84XAZXB + 0: XB + +/^(?(2)a|(1)(2))+$/ + 123a +Error -17 + +/(?<=a|bbbb)c/ + ac + 0: c + bbbbc + 0: c + +/abc/>testsavedregex +Compiled regex written to testsavedregex +<testsavedregex +Compiled regex loaded from testsavedregex +No study data + abc + 0: abc + *** Failers +No match + bca +No match + +/abc/F>testsavedregex +Compiled regex written to testsavedregex +<testsavedregex +Compiled regex (byte-inverted) loaded from testsavedregex +No study data + abc + 0: abc + *** Failers +No match + bca +No match + +/(a|b)/S>testsavedregex +Compiled regex written to testsavedregex +Study data written to testsavedregex +<testsavedregex +Compiled regex loaded from testsavedregex +Study data loaded from testsavedregex + abc + 0: a + *** Failers + 0: a + def +No match + +/(a|b)/SF>testsavedregex +Compiled regex written to testsavedregex +Study data written to testsavedregex +<testsavedregex +Compiled regex (byte-inverted) loaded from testsavedregex +Study data loaded from testsavedregex + abc + 0: a + *** Failers + 0: a + def +No match + +/line\nbreak/ + this is a line\nbreak + 0: line\x0abreak + line one\nthis is a line\nbreak in the second line + 0: line\x0abreak + +/line\nbreak/f + this is a line\nbreak + 0: line\x0abreak + ** Failers +No match + line one\nthis is a line\nbreak in the second line +No match + +/line\nbreak/mf + this is a line\nbreak + 0: line\x0abreak + ** Failers +No match + line one\nthis is a line\nbreak in the second line +No match + +/ End of testinput7 / diff --git a/testdata/testoutput8 b/testdata/testoutput8 new file mode 100644 index 0000000..25cde8d --- /dev/null +++ b/testdata/testoutput8 @@ -0,0 +1,1033 @@ +PCRE version 6.0 07-Jun-2005 + +/-- Do not use the \x{} construct except with patterns that have the --/ +/-- /8 option set, because PCRE doesn't recognize them as UTF-8 unless --/ +No match +/-- that option is set. However, the latest Perls recognize them always. --/ +No match + +/\x{100}ab/8 + \x{100}ab + 0: \x{100}ab + +/a\x{100}*b/8 + ab + 0: ab + a\x{100}b + 0: a\x{100}b + a\x{100}\x{100}b + 0: a\x{100}\x{100}b + +/a\x{100}+b/8 + a\x{100}b + 0: a\x{100}b + a\x{100}\x{100}b + 0: a\x{100}\x{100}b + *** Failers +No match + ab +No match + +/\bX/8 + Xoanon + 0: X + +Xoanon + 0: X + \x{300}Xoanon + 0: X + *** Failers +No match + YXoanon +No match + +/\BX/8 + YXoanon + 0: X + *** Failers +No match + Xoanon +No match + +Xoanon +No match + \x{300}Xoanon +No match + +/X\b/8 + X+oanon + 0: X + ZX\x{300}oanon + 0: X + FAX + 0: X + *** Failers +No match + Xoanon +No match + +/X\B/8 + Xoanon + 0: X + *** Failers +No match + X+oanon +No match + ZX\x{300}oanon +No match + FAX +No match + +/[^a]/8 + abcd + 0: b + a\x{100} + 0: \x{100} + +/^[abc\x{123}\x{400}-\x{402}]{2,3}\d/8 + ab99 + 0: ab9 + \x{123}\x{123}45 + 0: \x{123}\x{123}4 + \x{400}\x{401}\x{402}6 + 0: \x{400}\x{401}\x{402}6 + *** Failers +No match + d99 +No match + \x{123}\x{122}4 +No match + \x{400}\x{403}6 +No match + \x{400}\x{401}\x{402}\x{402}6 +No match + +/abc/8 + Ã] +Error -10 + à +Error -10 + ÃÃà +Error -10 + ÃÃÃ\? +No match + +/a.b/8 + acb + 0: acb + a\x7fb + 0: a\x{7f}b + a\x{100}b + 0: a\x{100}b + *** Failers +No match + a\nb +No match + +/a(.{3})b/8 + a\x{4000}xyb + 0: a\x{4000}xyb + a\x{4000}\x7fyb + 0: a\x{4000}\x{7f}yb + a\x{4000}\x{100}yb + 0: a\x{4000}\x{100}yb + *** Failers +No match + a\x{4000}b +No match + ac\ncb +No match + +/a(.*?)(.)/ + a\xc0\x88b + 0: a\xc0\x88b + 1: a\xc0\x88 + 2: a\xc0 + +/a(.*?)(.)/8 + a\x{100}b + 0: a\x{100}b + 1: a\x{100} + +/a(.*)(.)/ + a\xc0\x88b + 0: a\xc0\x88b + 1: a\xc0\x88 + 2: a\xc0 + +/a(.*)(.)/8 + a\x{100}b + 0: a\x{100}b + 1: a\x{100} + +/a(.)(.)/ + a\xc0\x92bcd + 0: a\xc0\x92 + +/a(.)(.)/8 + a\x{240}bcd + 0: a\x{240}b + +/a(.?)(.)/ + a\xc0\x92bcd + 0: a\xc0\x92 + 1: a\xc0 + +/a(.?)(.)/8 + a\x{240}bcd + 0: a\x{240}b + 1: a\x{240} + +/a(.??)(.)/ + a\xc0\x92bcd + 0: a\xc0\x92 + 1: a\xc0 + +/a(.??)(.)/8 + a\x{240}bcd + 0: a\x{240}b + 1: a\x{240} + +/a(.{3})b/8 + a\x{1234}xyb + 0: a\x{1234}xyb + a\x{1234}\x{4321}yb + 0: a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + 0: a\x{1234}\x{4321}\x{3412}b + *** Failers +No match + a\x{1234}b +No match + ac\ncb +No match + +/a(.{3,})b/8 + a\x{1234}xyb + 0: a\x{1234}xyb + a\x{1234}\x{4321}yb + 0: a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + 0: a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + 0: axxxxbcdefghijb + 1: axxxxb + a\x{1234}\x{4321}\x{3412}\x{3421}b + 0: a\x{1234}\x{4321}\x{3412}\x{3421}b + *** Failers +No match + a\x{1234}b +No match + +/a(.{3,}?)b/8 + a\x{1234}xyb + 0: a\x{1234}xyb + a\x{1234}\x{4321}yb + 0: a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + 0: a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + 0: axxxxbcdefghijb + 1: axxxxb + a\x{1234}\x{4321}\x{3412}\x{3421}b + 0: a\x{1234}\x{4321}\x{3412}\x{3421}b + *** Failers +No match + a\x{1234}b +No match + +/a(.{3,5})b/8 + a\x{1234}xyb + 0: a\x{1234}xyb + a\x{1234}\x{4321}yb + 0: a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + 0: a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + 0: axxxxb + a\x{1234}\x{4321}\x{3412}\x{3421}b + 0: a\x{1234}\x{4321}\x{3412}\x{3421}b + axbxxbcdefghijb + 0: axbxxb + axxxxxbcdefghijb + 0: axxxxxb + *** Failers +No match + a\x{1234}b +No match + axxxxxxbcdefghijb +No match + +/a(.{3,5}?)b/8 + a\x{1234}xyb + 0: a\x{1234}xyb + a\x{1234}\x{4321}yb + 0: a\x{1234}\x{4321}yb + a\x{1234}\x{4321}\x{3412}b + 0: a\x{1234}\x{4321}\x{3412}b + axxxxbcdefghijb + 0: axxxxb + a\x{1234}\x{4321}\x{3412}\x{3421}b + 0: a\x{1234}\x{4321}\x{3412}\x{3421}b + axbxxbcdefghijb + 0: axbxxb + axxxxxbcdefghijb + 0: axxxxxb + *** Failers +No match + a\x{1234}b +No match + axxxxxxbcdefghijb +No match + +/^[a\x{c0}]/8 + *** Failers +No match + \x{100} +No match + +/(?<=aXb)cd/8 + aXbcd + 0: cd + +/(?<=a\x{100}b)cd/8 + a\x{100}bcd + 0: cd + +/(?<=a\x{100000}b)cd/8 + a\x{100000}bcd + 0: cd + +/(?:\x{100}){3}b/8 + \x{100}\x{100}\x{100}b + 0: \x{100}\x{100}\x{100}b + *** Failers +No match + \x{100}\x{100}b +No match + +/\x{ab}/8 + \x{ab} + 0: \x{ab} + \xc2\xab + 0: \x{ab} + *** Failers +No match + \x00{ab} +No match + +/(?<=(.))X/8 + WXYZ + 0: X + \x{256}XYZ + 0: X + *** Failers +No match + XYZ +No match + +/[^a]+/8g + bcd + 0: bcd + 1: bc + 2: b + \x{100}aY\x{256}Z + 0: \x{100} + 0: Y\x{256}Z + 1: Y\x{256} + 2: Y + +/^[^a]{2}/8 + \x{100}bc + 0: \x{100}b + +/^[^a]{2,}/8 + \x{100}bcAa + 0: \x{100}bcA + 1: \x{100}bc + 2: \x{100}b + +/^[^a]{2,}?/8 + \x{100}bca + 0: \x{100}bc + 1: \x{100}b + +/[^a]+/8ig + bcd + 0: bcd + 1: bc + 2: b + \x{100}aY\x{256}Z + 0: \x{100} + 0: Y\x{256}Z + 1: Y\x{256} + 2: Y + +/^[^a]{2}/8i + \x{100}bc + 0: \x{100}b + +/^[^a]{2,}/8i + \x{100}bcAa + 0: \x{100}bc + 1: \x{100}b + +/^[^a]{2,}?/8i + \x{100}bca + 0: \x{100}bc + 1: \x{100}b + +/\x{100}{0,0}/8 + abcd + 0: + +/\x{100}?/8 + abcd + 0: + \x{100}\x{100} + 0: \x{100} + 1: + +/\x{100}{0,3}/8 + \x{100}\x{100} + 0: \x{100}\x{100} + 1: \x{100} + 2: + \x{100}\x{100}\x{100}\x{100} + 0: \x{100}\x{100}\x{100} + 1: \x{100}\x{100} + 2: \x{100} + 3: + +/\x{100}*/8 + abce + 0: + \x{100}\x{100}\x{100}\x{100} + 0: \x{100}\x{100}\x{100}\x{100} + 1: \x{100}\x{100}\x{100} + 2: \x{100}\x{100} + 3: \x{100} + 4: + +/\x{100}{1,1}/8 + abcd\x{100}\x{100}\x{100}\x{100} + 0: \x{100} + +/\x{100}{1,3}/8 + abcd\x{100}\x{100}\x{100}\x{100} + 0: \x{100}\x{100}\x{100} + 1: \x{100}\x{100} + 2: \x{100} + +/\x{100}+/8 + abcd\x{100}\x{100}\x{100}\x{100} + 0: \x{100}\x{100}\x{100}\x{100} + 1: \x{100}\x{100}\x{100} + 2: \x{100}\x{100} + 3: \x{100} + +/\x{100}{3}/8 + abcd\x{100}\x{100}\x{100}XX + 0: \x{100}\x{100}\x{100} + +/\x{100}{3,5}/8 + abcd\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}XX + 0: \x{100}\x{100}\x{100}\x{100}\x{100} + 1: \x{100}\x{100}\x{100}\x{100} + 2: \x{100}\x{100}\x{100} + +/\x{100}{3,}/8 + abcd\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}XX + 0: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 1: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 2: \x{100}\x{100}\x{100}\x{100}\x{100} + 3: \x{100}\x{100}\x{100}\x{100} + 4: \x{100}\x{100}\x{100} + +/(?<=a\x{100}{2}b)X/8 + Xyyya\x{100}\x{100}bXzzz + 0: X + +/\D*/8 + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +Matched, but too many subsidiary matches + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 4: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 5: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 6: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 7: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 8: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 9: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +11: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +12: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +13: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +14: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +15: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +16: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +17: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +18: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +19: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +20: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +21: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/\D*/8 + \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +Matched, but too many subsidiary matches + 0: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 1: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 2: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 3: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 4: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 5: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 6: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 7: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 8: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + 9: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +10: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +11: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +12: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +13: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +14: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +15: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +16: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +17: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +18: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +19: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +20: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} +21: \x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100}\x{100} + +/\D/8 + 1X2 + 0: X + 1\x{100}2 + 0: \x{100} + +/>\S/8 + > >X Y + 0: >X + > >\x{100} Y + 0: >\x{100} + +/\d/8 + \x{100}3 + 0: 3 + +/\s/8 + \x{100} X + 0: + +/\D+/8 + 12abcd34 + 0: abcd + 1: abc + 2: ab + 3: a + *** Failers + 0: *** Failers + 1: *** Failer + 2: *** Faile + 3: *** Fail + 4: *** Fai + 5: *** Fa + 6: *** F + 7: *** + 8: *** + 9: ** +10: * + 1234 +No match + +/\D{2,3}/8 + 12abcd34 + 0: abc + 1: ab + 12ab34 + 0: ab + *** Failers + 0: *** + 1: ** + 1234 +No match + 12a34 +No match + +/\D{2,3}?/8 + 12abcd34 + 0: abc + 1: ab + 12ab34 + 0: ab + *** Failers + 0: *** + 1: ** + 1234 +No match + 12a34 +No match + +/\d+/8 + 12abcd34 + 0: 12 + 1: 1 + *** Failers +No match + +/\d{2,3}/8 + 12abcd34 + 0: 12 + 1234abcd + 0: 123 + 1: 12 + *** Failers +No match + 1.4 +No match + +/\d{2,3}?/8 + 12abcd34 + 0: 12 + 1234abcd + 0: 123 + 1: 12 + *** Failers +No match + 1.4 +No match + +/\S+/8 + 12abcd34 + 0: 12abcd34 + 1: 12abcd3 + 2: 12abcd + 3: 12abc + 4: 12ab + 5: 12a + 6: 12 + 7: 1 + *** Failers + 0: *** + 1: ** + 2: * + \ \ +No match + +/\S{2,3}/8 + 12abcd34 + 0: 12a + 1: 12 + 1234abcd + 0: 123 + 1: 12 + *** Failers + 0: *** + 1: ** + \ \ +No match + +/\S{2,3}?/8 + 12abcd34 + 0: 12a + 1: 12 + 1234abcd + 0: 123 + 1: 12 + *** Failers + 0: *** + 1: ** + \ \ +No match + +/>\s+</8 + 12> <34 + 0: > < + *** Failers +No match + +/>\s{2,3}</8 + ab> <cd + 0: > < + ab> <ce + 0: > < + *** Failers +No match + ab> <cd +No match + +/>\s{2,3}?</8 + ab> <cd + 0: > < + ab> <ce + 0: > < + *** Failers +No match + ab> <cd +No match + +/\w+/8 + 12 34 + 0: 12 + 1: 1 + *** Failers + 0: Failers + 1: Failer + 2: Faile + 3: Fail + 4: Fai + 5: Fa + 6: F + +++=*! +No match + +/\w{2,3}/8 + ab cd + 0: ab + abcd ce + 0: abc + 1: ab + *** Failers + 0: Fai + 1: Fa + a.b.c +No match + +/\w{2,3}?/8 + ab cd + 0: ab + abcd ce + 0: abc + 1: ab + *** Failers + 0: Fai + 1: Fa + a.b.c +No match + +/\W+/8 + 12====34 + 0: ==== + 1: === + 2: == + 3: = + *** Failers + 0: *** + 1: *** + 2: ** + 3: * + abcd +No match + +/\W{2,3}/8 + ab====cd + 0: === + 1: == + ab==cd + 0: == + *** Failers + 0: *** + 1: ** + a.b.c +No match + +/\W{2,3}?/8 + ab====cd + 0: === + 1: == + ab==cd + 0: == + *** Failers + 0: *** + 1: ** + a.b.c +No match + +/[\x{100}]/8 + \x{100} + 0: \x{100} + Z\x{100} + 0: \x{100} + \x{100}Z + 0: \x{100} + *** Failers +No match + +/[Z\x{100}]/8 + Z\x{100} + 0: Z + \x{100} + 0: \x{100} + \x{100}Z + 0: \x{100} + *** Failers +No match + +/[\x{100}\x{200}]/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + *** Failers +No match + +/[\x{100}-\x{200}]/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{111}cd + 0: \x{111} + *** Failers +No match + +/[z-\x{200}]/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{111}cd + 0: \x{111} + abzcd + 0: z + ab|cd + 0: | + *** Failers +No match + +/[Q\x{100}\x{200}]/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + Q? + 0: Q + *** Failers +No match + +/[Q\x{100}-\x{200}]/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{111}cd + 0: \x{111} + Q? + 0: Q + *** Failers +No match + +/[Qz-\x{200}]/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{111}cd + 0: \x{111} + abzcd + 0: z + ab|cd + 0: | + Q? + 0: Q + *** Failers +No match + +/[\x{100}\x{200}]{1,3}/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{200}\x{100}\x{200}\x{100}cd + 0: \x{200}\x{100}\x{200} + 1: \x{200}\x{100} + 2: \x{200} + *** Failers +No match + +/[\x{100}\x{200}]{1,3}?/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{200}\x{100}\x{200}\x{100}cd + 0: \x{200}\x{100}\x{200} + 1: \x{200}\x{100} + 2: \x{200} + *** Failers +No match + +/[Q\x{100}\x{200}]{1,3}/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{200}\x{100}\x{200}\x{100}cd + 0: \x{200}\x{100}\x{200} + 1: \x{200}\x{100} + 2: \x{200} + *** Failers +No match + +/[Q\x{100}\x{200}]{1,3}?/8 + ab\x{100}cd + 0: \x{100} + ab\x{200}cd + 0: \x{200} + ab\x{200}\x{100}\x{200}\x{100}cd + 0: \x{200}\x{100}\x{200} + 1: \x{200}\x{100} + 2: \x{200} + *** Failers +No match + +/(?<=[\x{100}\x{200}])X/8 + abc\x{200}X + 0: X + abc\x{100}X + 0: X + *** Failers +No match + X +No match + +/(?<=[Q\x{100}\x{200}])X/8 + abc\x{200}X + 0: X + abc\x{100}X + 0: X + abQX + 0: X + *** Failers +No match + X +No match + +/(?<=[\x{100}\x{200}]{3})X/8 + abc\x{100}\x{200}\x{100}X + 0: X + *** Failers +No match + abc\x{200}X +No match + X +No match + +/[^\x{100}\x{200}]X/8 + AX + 0: AX + \x{150}X + 0: \x{150}X + \x{500}X + 0: \x{500}X + *** Failers +No match + \x{100}X +No match + \x{200}X +No match + +/[^Q\x{100}\x{200}]X/8 + AX + 0: AX + \x{150}X + 0: \x{150}X + \x{500}X + 0: \x{500}X + *** Failers +No match + \x{100}X +No match + \x{200}X +No match + QX +No match + +/[^\x{100}-\x{200}]X/8 + AX + 0: AX + \x{500}X + 0: \x{500}X + *** Failers +No match + \x{100}X +No match + \x{150}X +No match + \x{200}X +No match + +/[z-\x{100}]/8i + z + 0: z + Z + 0: Z + \x{100} + 0: \x{100} + *** Failers +No match + \x{102} +No match + y +No match + +/[\xFF]/ + >\xff< + 0: \xff + +/[\xff]/8 + >\x{ff}< + 0: \x{ff} + +/[^\xFF]/ + XYZ + 0: X + +/[^\xff]/8 + XYZ + 0: X + \x{123} + 0: \x{123} + +/^[ac]*b/8 + xb +No match + +/^[ac\x{100}]*b/8 + xb +No match + +/^[^x]*b/8i + xb +No match + +/^[^x]*b/8 + xb +No match + +/^\d*b/8 + xb +No match + +/(|a)/g8 + catac + 0: + 0: a + 1: + 0: + 0: a + 1: + 0: + 0: + a\x{256}a + 0: a + 1: + 0: + 0: a + 1: + 0: + +/^\x{85}$/8i + \x{85} + 0: \x{85} + +/ End of testinput 8 / diff --git a/testdata/testoutput9 b/testdata/testoutput9 new file mode 100644 index 0000000..1b77a98 --- /dev/null +++ b/testdata/testoutput9 @@ -0,0 +1,1234 @@ +PCRE version 6.0 07-Jun-2005 + +/\pL\P{Nd}/8 + AB + 0: AB + *** Failers + 0: Fa + A0 +No match + 00 +No match + +/\X./8 + AB + 0: AB + A\x{300}BC + 0: A\x{300}B + A\x{300}\x{301}\x{302}BC + 0: A\x{300}\x{301}\x{302}B + *** Failers + 0: ** + \x{300} +No match + +/\X\X/8 + ABC + 0: AB + A\x{300}B\x{300}\x{301}C + 0: A\x{300}B\x{300}\x{301} + A\x{300}\x{301}\x{302}BC + 0: A\x{300}\x{301}\x{302}B + *** Failers + 0: ** + \x{300} +No match + +/^\pL+/8 + abcd + 0: abcd + 1: abc + 2: ab + 3: a + a + 0: a + *** Failers +No match + +/^\PL+/8 + 1234 + 0: 1234 + 1: 123 + 2: 12 + 3: 1 + = + 0: = + *** Failers + 0: *** + 1: *** + 2: ** + 3: * + abcd +No match + +/^\X+/8 + abcdA\x{300}\x{301}\x{302} + 0: abcdA\x{300}\x{301}\x{302} + 1: abcd + 2: abc + 3: ab + 4: a + A\x{300}\x{301}\x{302} + 0: A\x{300}\x{301}\x{302} + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302} + 0: A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302} + 1: A\x{300}\x{301}\x{302} + a + 0: a + *** Failers + 0: *** Failers + 1: *** Failer + 2: *** Faile + 3: *** Fail + 4: *** Fai + 5: *** Fa + 6: *** F + 7: *** + 8: *** + 9: ** +10: * + \x{300}\x{301}\x{302} +No match + +/\X?abc/8 + abc + 0: abc + A\x{300}abc + 0: A\x{300}abc + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz + 0: A\x{300}abc + \x{300}abc + 0: abc + *** Failers +No match + +/^\X?abc/8 + abc + 0: abc + A\x{300}abc + 0: A\x{300}abc + *** Failers +No match + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz +No match + \x{300}abc +No match + +/\X*abc/8 + abc + 0: abc + A\x{300}abc + 0: A\x{300}abc + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz + 0: A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abc + \x{300}abc + 0: abc + *** Failers +No match + +/^\X*abc/8 + abc + 0: abc + A\x{300}abc + 0: A\x{300}abc + A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abcxyz + 0: A\x{300}\x{301}\x{302}A\x{300}A\x{300}A\x{300}abc + *** Failers +No match + \x{300}abc +No match + +/^\pL?=./8 + A=b + 0: A=b + =c + 0: =c + *** Failers +No match + 1=2 +No match + AAAA=b +No match + +/^\pL*=./8 + AAAA=b + 0: AAAA=b + =c + 0: =c + *** Failers +No match + 1=2 +No match + +/^\X{2,3}X/8 + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X + 0: A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X + 0: A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X + *** Failers +No match + X +No match + A\x{300}\x{301}\x{302}X +No match + A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}A\x{300}\x{301}\x{302}X +No match + +/^\pC\pL\pM\pN\pP\pS\pZ</8 + \x7f\x{c0}\x{30f}\x{660}\x{66c}\x{f01}\x{1680}< + 0: \x{7f}\x{c0}\x{30f}\x{660}\x{66c}\x{f01}\x{1680}< + \np\x{300}9!\$ < + 0: \x{0a}p\x{300}9!$ < + ** Failers +No match + ap\x{300}9!\$ < +No match + +/^\PC/8 + X + 0: X + ** Failers + 0: * + \x7f +No match + +/^\PL/8 + 9 + 0: 9 + ** Failers + 0: * + \x{c0} +No match + +/^\PM/8 + X + 0: X + ** Failers + 0: * + \x{30f} +No match + +/^\PN/8 + X + 0: X + ** Failers + 0: * + \x{660} +No match + +/^\PP/8 + X + 0: X + ** Failers +No match + \x{66c} +No match + +/^\PS/8 + X + 0: X + ** Failers + 0: * + \x{f01} +No match + +/^\PZ/8 + X + 0: X + ** Failers + 0: * + \x{1680} +No match + +/^\p{Cc}/8 + \x{017} + 0: \x{17} + \x{09f} + 0: \x{9f} + ** Failers +No match + \x{0600} +No match + +/^\p{Cf}/8 + \x{601} + 0: \x{601} + ** Failers +No match + \x{09f} +No match + +/^\p{Cn}/8 + ** Failers +No match + \x{09f} +No match + +/^\p{Co}/8 + \x{f8ff} + 0: \x{f8ff} + ** Failers +No match + \x{09f} +No match + +/^\p{Cs}/8 + \x{dfff} + 0: \x{dfff} + ** Failers +No match + \x{09f} +No match + +/^\p{Ll}/8 + a + 0: a + ** Failers +No match + Z +No match + \x{dfff} +No match + +/^\p{Lm}/8 + \x{2b0} + 0: \x{2b0} + ** Failers +No match + a +No match + +/^\p{Lo}/8 + \x{1bb} + 0: \x{1bb} + ** Failers +No match + a +No match + \x{2b0} +No match + +/^\p{Lt}/8 + \x{1c5} + 0: \x{1c5} + ** Failers +No match + a +No match + \x{2b0} +No match + +/^\p{Lu}/8 + A + 0: A + ** Failers +No match + \x{2b0} +No match + +/^\p{Mc}/8 + \x{903} + 0: \x{903} + ** Failers +No match + X +No match + \x{300} +No match + +/^\p{Me}/8 + \x{488} + 0: \x{488} + ** Failers +No match + X +No match + \x{903} +No match + \x{300} +No match + +/^\p{Mn}/8 + \x{300} + 0: \x{300} + ** Failers +No match + X +No match + \x{903} +No match + +/^\p{Nd}+/8 + 0123456789\x{660}\x{661}\x{662}\x{663}\x{664}\x{665}\x{666}\x{667}\x{668}\x{669}\x{66a} + 0: 0123456789\x{660}\x{661}\x{662}\x{663}\x{664}\x{665}\x{666}\x{667}\x{668}\x{669} + 1: 0123456789\x{660}\x{661}\x{662}\x{663}\x{664}\x{665}\x{666}\x{667}\x{668} + 2: 0123456789\x{660}\x{661}\x{662}\x{663}\x{664}\x{665}\x{666}\x{667} + 3: 0123456789\x{660}\x{661}\x{662}\x{663}\x{664}\x{665}\x{666} + 4: 0123456789\x{660}\x{661}\x{662}\x{663}\x{664}\x{665} + 5: 0123456789\x{660}\x{661}\x{662}\x{663}\x{664} + 6: 0123456789\x{660}\x{661}\x{662}\x{663} + 7: 0123456789\x{660}\x{661}\x{662} + 8: 0123456789\x{660}\x{661} + 9: 0123456789\x{660} +10: 0123456789 +11: 012345678 +12: 01234567 +13: 0123456 +14: 012345 +15: 01234 +16: 0123 +17: 012 +18: 01 +19: 0 + \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4}\x{6f5}\x{6f6}\x{6f7}\x{6f8}\x{6f9}\x{6fa} + 0: \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4}\x{6f5}\x{6f6}\x{6f7}\x{6f8}\x{6f9} + 1: \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4}\x{6f5}\x{6f6}\x{6f7}\x{6f8} + 2: \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4}\x{6f5}\x{6f6}\x{6f7} + 3: \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4}\x{6f5}\x{6f6} + 4: \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4}\x{6f5} + 5: \x{6f0}\x{6f1}\x{6f2}\x{6f3}\x{6f4} + 6: \x{6f0}\x{6f1}\x{6f2}\x{6f3} + 7: \x{6f0}\x{6f1}\x{6f2} + 8: \x{6f0}\x{6f1} + 9: \x{6f0} + \x{966}\x{967}\x{968}\x{969}\x{96a}\x{96b}\x{96c}\x{96d}\x{96e}\x{96f}\x{970} + 0: \x{966}\x{967}\x{968}\x{969}\x{96a}\x{96b}\x{96c}\x{96d}\x{96e}\x{96f} + 1: \x{966}\x{967}\x{968}\x{969}\x{96a}\x{96b}\x{96c}\x{96d}\x{96e} + 2: \x{966}\x{967}\x{968}\x{969}\x{96a}\x{96b}\x{96c}\x{96d} + 3: \x{966}\x{967}\x{968}\x{969}\x{96a}\x{96b}\x{96c} + 4: \x{966}\x{967}\x{968}\x{969}\x{96a}\x{96b} + 5: \x{966}\x{967}\x{968}\x{969}\x{96a} + 6: \x{966}\x{967}\x{968}\x{969} + 7: \x{966}\x{967}\x{968} + 8: \x{966}\x{967} + 9: \x{966} + ** Failers +No match + X +No match + +/^\p{Nl}/8 + \x{16ee} + 0: \x{16ee} + ** Failers +No match + X +No match + \x{966} +No match + +/^\p{No}/8 + \x{b2} + 0: \x{b2} + \x{b3} + 0: \x{b3} + ** Failers +No match + X +No match + \x{16ee} +No match + +/^\p{Pc}/8 + \x5f + 0: _ + \x{203f} + 0: \x{203f} + ** Failers +No match + X +No match + - +No match + \x{58a} +No match + +/^\p{Pd}/8 + - + 0: - + \x{58a} + 0: \x{58a} + ** Failers +No match + X +No match + \x{203f} +No match + +/^\p{Pe}/8 + ) + 0: ) + ] + 0: ] + } + 0: } + \x{f3b} + 0: \x{f3b} + ** Failers +No match + X +No match + \x{203f} +No match + ( +No match + [ +No match + { +No match + \x{f3c} +No match + +/^\p{Pf}/8 + \x{bb} + 0: \x{bb} + \x{2019} + 0: \x{2019} + ** Failers +No match + X +No match + \x{203f} +No match + +/^\p{Pi}/8 + \x{ab} + 0: \x{ab} + \x{2018} + 0: \x{2018} + ** Failers +No match + X +No match + \x{203f} +No match + +/^\p{Po}/8 + ! + 0: ! + \x{37e} + 0: \x{37e} + ** Failers + 0: * + X +No match + \x{203f} +No match + +/^\p{Ps}/8 + ( + 0: ( + [ + 0: [ + { + 0: { + \x{f3c} + 0: \x{f3c} + ** Failers +No match + X +No match + ) +No match + ] +No match + } +No match + \x{f3b} +No match + +/^\p{Sc}+/8 + $\x{a2}\x{a3}\x{a4}\x{a5}\x{a6} + 0: $\x{a2}\x{a3}\x{a4}\x{a5} + 1: $\x{a2}\x{a3}\x{a4} + 2: $\x{a2}\x{a3} + 3: $\x{a2} + 4: $ + \x{9f2} + 0: \x{9f2} + ** Failers +No match + X +No match + \x{2c2} +No match + +/^\p{Sk}/8 + \x{2c2} + 0: \x{2c2} + ** Failers +No match + X +No match + \x{9f2} +No match + +/^\p{Sm}+/8 + +<|~\x{ac}\x{2044} + 0: +<|~\x{ac}\x{2044} + 1: +<|~\x{ac} + 2: +<|~ + 3: +<| + 4: +< + 5: + + ** Failers +No match + X +No match + \x{9f2} +No match + +/^\p{So}/8 + \x{a6} + 0: \x{a6} + \x{482} + 0: \x{482} + ** Failers +No match + X +No match + \x{9f2} +No match + +/^\p{Zl}/8 + \x{2028} + 0: \x{2028} + ** Failers +No match + X +No match + \x{2029} +No match + +/^\p{Zp}/8 + \x{2029} + 0: \x{2029} + ** Failers +No match + X +No match + \x{2028} +No match + +/^\p{Zs}/8 + \ \ + 0: + \x{a0} + 0: \x{a0} + \x{1680} + 0: \x{1680} + \x{180e} + 0: \x{180e} + \x{2000} + 0: \x{2000} + \x{2001} + 0: \x{2001} + ** Failers +No match + \x{2028} +No match + \x{200d} +No match + +/\p{Nd}+(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + 2: \x{660}\x{661}\x{662} + +/\p{Nd}+?(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + 2: \x{660}\x{661}\x{662} + +/\p{Nd}{2,}(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + +/\p{Nd}{2,}?(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + +/\p{Nd}*(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + 2: \x{660}\x{661}\x{662} + 3: \x{660}\x{661} + +/\p{Nd}*?(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + 2: \x{660}\x{661}\x{662} + 3: \x{660}\x{661} + +/\p{Nd}{2}(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}A + +/\p{Nd}{2,3}(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + +/\p{Nd}{2,3}?(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + 1: \x{660}\x{661}\x{662}A + +/\p{Nd}?(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662} + 1: \x{660}\x{661} + +/\p{Nd}??(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662} + 1: \x{660}\x{661} + +/\p{Nd}*+(..)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}AB + +/\p{Nd}*+(...)/8 + \x{660}\x{661}\x{662}ABC + 0: \x{660}\x{661}\x{662}ABC + +/\p{Nd}*+(....)/8 + ** Failers + 0: ** F + \x{660}\x{661}\x{662}ABC +No match + +/\p{Lu}/8i + A + 0: A + a\x{10a0}B + 0: \x{10a0} + ** Failers + 0: F + a +No match + \x{1d00} +No match + +/\p{^Lu}/8i + 1234 + 0: 1 + ** Failers + 0: * + ABC +No match + +/\P{Lu}/8i + 1234 + 0: 1 + ** Failers + 0: * + ABC +No match + +/(?<=A\p{Nd})XYZ/8 + A2XYZ + 0: XYZ + 123A5XYZPQR + 0: XYZ + ABA\x{660}XYZpqr + 0: XYZ + ** Failers +No match + AXYZ +No match + XYZ +No match + +/(?<!\pL)XYZ/8 + 1XYZ + 0: XYZ + AB=XYZ.. + 0: XYZ + XYZ + 0: XYZ + ** Failers +No match + WXYZ +No match + +/[\p{Nd}]/8 + 1234 + 0: 1 + +/[\p{Nd}+-]+/8 + 1234 + 0: 1234 + 1: 123 + 2: 12 + 3: 1 + 12-34 + 0: 12-34 + 1: 12-3 + 2: 12- + 3: 12 + 4: 1 + 12+\x{661}-34 + 0: 12+\x{661}-34 + 1: 12+\x{661}-3 + 2: 12+\x{661}- + 3: 12+\x{661} + 4: 12+ + 5: 12 + 6: 1 + ** Failers +No match + abcd +No match + +/[\P{Nd}]+/8 + abcd + 0: abcd + 1: abc + 2: ab + 3: a + ** Failers + 0: ** Failers + 1: ** Failer + 2: ** Faile + 3: ** Fail + 4: ** Fai + 5: ** Fa + 6: ** F + 7: ** + 8: ** + 9: * + 1234 +No match + +/\D+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 +No match + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +Matched, but too many subsidiary matches + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 4: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 5: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 6: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 7: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 8: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 9: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +11: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +12: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +13: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +14: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +15: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +16: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +17: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +18: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +19: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +20: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +21: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/\P{Nd}+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 +No match + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +Matched, but too many subsidiary matches + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 4: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 5: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 6: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 7: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 8: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 9: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +11: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +12: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +13: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +14: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +15: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +16: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +17: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +18: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +19: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +20: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +21: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/[\D]+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 +No match + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +Matched, but too many subsidiary matches + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 4: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 5: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 6: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 7: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 8: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 9: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +11: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +12: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +13: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +14: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +15: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +16: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +17: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +18: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +19: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +20: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +21: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/[\P{Nd}]+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 +No match + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +Matched, but too many subsidiary matches + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 4: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 5: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 6: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 7: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 8: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 9: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +11: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +12: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +13: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +14: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +15: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +16: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +17: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +18: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +19: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +20: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +21: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/[\D\P{Nd}]+/8 + 11111111111111111111111111111111111111111111111111111111111111111111111 +No match + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +Matched, but too many subsidiary matches + 0: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 2: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 3: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 4: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 5: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 6: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 7: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 8: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 9: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +11: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +12: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +13: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +14: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +15: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +16: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +17: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +18: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +19: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +20: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +21: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + +/\pL/8 + a + 0: a + A + 0: A + +/\pL/8i + a + 0: a + A + 0: A + +/\p{Lu}/8 + A + 0: A + aZ + 0: Z + ** Failers + 0: F + abc +No match + +/\p{Lu}/8i + A + 0: A + aZ + 0: Z + ** Failers + 0: F + abc +No match + +/\p{Ll}/8 + a + 0: a + Az + 0: z + ** Failers + 0: a + ABC +No match + +/\p{Ll}/8i + a + 0: a + Az + 0: z + ** Failers + 0: a + ABC +No match + +/^\x{c0}$/8i + \x{c0} + 0: \x{c0} + \x{e0} + 0: \x{e0} + +/^\x{e0}$/8i + \x{c0} + 0: \x{c0} + \x{e0} + 0: \x{e0} + +/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8 + A\x{391}\x{10427}\x{ff3a}\x{1fb0} + 0: A\x{391}\x{10427}\x{ff3a}\x{1fb0} + ** Failers +No match + a\x{391}\x{10427}\x{ff3a}\x{1fb0} +No match + A\x{3b1}\x{10427}\x{ff3a}\x{1fb0} +No match + A\x{391}\x{1044F}\x{ff3a}\x{1fb0} +No match + A\x{391}\x{10427}\x{ff5a}\x{1fb0} +No match + A\x{391}\x{10427}\x{ff3a}\x{1fb8} +No match + +/A\x{391}\x{10427}\x{ff3a}\x{1fb0}/8i + A\x{391}\x{10427}\x{ff3a}\x{1fb0} + 0: A\x{391}\x{10427}\x{ff3a}\x{1fb0} + a\x{391}\x{10427}\x{ff3a}\x{1fb0} + 0: a\x{391}\x{10427}\x{ff3a}\x{1fb0} + A\x{3b1}\x{10427}\x{ff3a}\x{1fb0} + 0: A\x{3b1}\x{10427}\x{ff3a}\x{1fb0} + A\x{391}\x{1044F}\x{ff3a}\x{1fb0} + 0: A\x{391}\x{1044f}\x{ff3a}\x{1fb0} + A\x{391}\x{10427}\x{ff5a}\x{1fb0} + 0: A\x{391}\x{10427}\x{ff5a}\x{1fb0} + A\x{391}\x{10427}\x{ff3a}\x{1fb8} + 0: A\x{391}\x{10427}\x{ff3a}\x{1fb8} + +/\x{391}+/8i + \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} + 0: \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} + 1: \x{391}\x{3b1}\x{3b1}\x{3b1} + 2: \x{391}\x{3b1}\x{3b1} + 3: \x{391}\x{3b1} + 4: \x{391} + +/\x{391}{3,5}(.)/8i + \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391}X + 0: \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391}X + 1: \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} + 2: \x{391}\x{3b1}\x{3b1}\x{3b1} + +/\x{391}{3,5}?(.)/8i + \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391}X + 0: \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391}X + 1: \x{391}\x{3b1}\x{3b1}\x{3b1}\x{391} + 2: \x{391}\x{3b1}\x{3b1}\x{3b1} + +/[\x{391}\x{ff3a}]/8i + \x{391} + 0: \x{391} + \x{ff3a} + 0: \x{ff3a} + \x{3b1} + 0: \x{3b1} + \x{ff5a} + 0: \x{ff5a} + +/[\x{c0}\x{391}]/8i + \x{c0} + 0: \x{c0} + \x{e0} + 0: \x{e0} + +/[\x{105}-\x{109}]/8i + \x{104} + 0: \x{104} + \x{105} + 0: \x{105} + \x{109} + 0: \x{109} + ** Failers +No match + \x{100} +No match + \x{10a} +No match + +/[z-\x{100}]/8i + Z + 0: Z + z + 0: z + \x{39c} + 0: \x{39c} + \x{178} + 0: \x{178} + | + 0: | + \x{80} + 0: \x{80} + \x{ff} + 0: \x{ff} + \x{100} + 0: \x{100} + \x{101} + 0: \x{101} + ** Failers +No match + \x{102} +No match + Y +No match + y +No match + +/[z-\x{100}]/8i + +/^\X/8 + A + 0: A + A\x{300}BC + 0: A\x{300} + A\x{300}\x{301}\x{302}BC + 0: A\x{300}\x{301}\x{302} + *** Failers + 0: * + \x{300} +No match + +/^[\X]/8 + X123 + 0: X + *** Failers +No match + AXYZ +No match + +/^(\X*)C/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + 0: A\x{300}\x{301}\x{302}BC + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 0: A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 1: A\x{300}\x{301}\x{302}BC + +/^(\X*?)C/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + 0: A\x{300}\x{301}\x{302}BC + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 0: A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 1: A\x{300}\x{301}\x{302}BC + +/^(\X*)(.)/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + 0: A\x{300}\x{301}\x{302}BCA + 1: A\x{300}\x{301}\x{302}BC + 2: A\x{300}\x{301}\x{302}B + 3: A + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 0: A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 1: A\x{300}\x{301}\x{302}BCA + 2: A\x{300}\x{301}\x{302}BC + 3: A\x{300}\x{301}\x{302}B + 4: A + +/^(\X*?)(.)/8 + A\x{300}\x{301}\x{302}BCA\x{300}\x{301} + 0: A\x{300}\x{301}\x{302}BCA + 1: A\x{300}\x{301}\x{302}BC + 2: A\x{300}\x{301}\x{302}B + 3: A + A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 0: A\x{300}\x{301}\x{302}BCA\x{300}\x{301}C + 1: A\x{300}\x{301}\x{302}BCA + 2: A\x{300}\x{301}\x{302}BC + 3: A\x{300}\x{301}\x{302}B + 4: A + +/^\X(.)/8 + *** Failers + 0: ** + A\x{300}\x{301}\x{302} +No match + +/^\X{2,3}(.)/8 + A\x{300}\x{301}B\x{300}X + 0: A\x{300}\x{301}B\x{300}X + A\x{300}\x{301}B\x{300}C\x{300}\x{301} + 0: A\x{300}\x{301}B\x{300}C + A\x{300}\x{301}B\x{300}C\x{300}\x{301}X + 0: A\x{300}\x{301}B\x{300}C\x{300}\x{301}X + 1: A\x{300}\x{301}B\x{300}C + A\x{300}\x{301}B\x{300}C\x{300}\x{301}DA\x{300}X + 0: A\x{300}\x{301}B\x{300}C\x{300}\x{301}D + 1: A\x{300}\x{301}B\x{300}C + +/^\X{2,3}?(.)/8 + A\x{300}\x{301}B\x{300}X + 0: A\x{300}\x{301}B\x{300}X + A\x{300}\x{301}B\x{300}C\x{300}\x{301} + 0: A\x{300}\x{301}B\x{300}C + A\x{300}\x{301}B\x{300}C\x{300}\x{301}X + 0: A\x{300}\x{301}B\x{300}C\x{300}\x{301}X + 1: A\x{300}\x{301}B\x{300}C + A\x{300}\x{301}B\x{300}C\x{300}\x{301}DA\x{300}X + 0: A\x{300}\x{301}B\x{300}C\x{300}\x{301}D + 1: A\x{300}\x{301}B\x{300}C + +/^\pN{2,3}X/ + 12X + 0: 12X + 123X + 0: 123X + *** Failers +No match + X +No match + 1X +No match + 1234X +No match + +/\x{100}/i8 + \x{100} + 0: \x{100} + \x{101} + 0: \x{101} + +/ End / @@ -2,6 +2,10 @@ * libucp - Unicode Property Table handler * *************************************************/ + +#ifndef _UCP_H +#define _UCP_H + /* These are the character categories that are returned by ucp_findchar */ enum { @@ -49,10 +53,8 @@ enum { ucp_Zs /* Space separator */ }; -/* For use in PCRE we make this function static so that there is no conflict if -PCRE is linked with an application that makes use of an external version - -assuming an external version is ever released... */ +extern int ucp_findchar(const int, int *, int *); -static int ucp_findchar(const int, int *, int *); +#endif /* End of ucp.h */ @@ -2,12 +2,13 @@ * libucp - Unicode Property Table handler * *************************************************/ -/* This function provides a fast way of obtaining the basic Unicode properties -of a character, using a compact binary tree that occupies less than 100K bytes. +/* Copyright (c) University of Cambridge 2004 */ - Copyright (c) 2004 University of Cambridge +/* This little library provides a fast way of obtaining the basic Unicode +properties of a character, using a compact binary tree that occupies less than +100K bytes. -------------------------------------------------------------------------------- +----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: @@ -33,7 +34,7 @@ INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------------------------------------------------------------- +----------------------------------------------------------------------------- */ @@ -42,6 +43,14 @@ POSSIBILITY OF SUCH DAMAGE. #include "ucptable.c" /* The table itself */ +/* In some environments, external functions have to be preceded by some magic. +In my world (Unix), they do not. Use a macro to deal with this. */ + +#ifndef EXPORT +#define EXPORT +#endif + + /************************************************* * Search table and return data * @@ -59,7 +68,7 @@ Arguments: Returns: the character type category or -1 if not found */ -static int +EXPORT int ucp_findchar(const int c, int *type_ptr, int *case_ptr) { cnode *node = ucp_table; @@ -148,4 +157,4 @@ switch ((*type_ptr = ((node->f0 & f0_typemask) >> f0_typeshift))) } } -/* End of ucp.c */ +/* End of ucp_findchar.c */ |