diff options
Diffstat (limited to 'doc/pcretest.txt')
| -rw-r--r-- | doc/pcretest.txt | 213 |
1 files changed, 213 insertions, 0 deletions
diff --git a/doc/pcretest.txt b/doc/pcretest.txt new file mode 100644 index 0000000..29e2f5c --- /dev/null +++ b/doc/pcretest.txt @@ -0,0 +1,213 @@ +The pcretest program +-------------------- + +This program is intended for testing PCRE, but it can also be used for +experimenting with regular expressions. + +If it is given two filename arguments, it reads from the first and writes to +the second. If it is given only one filename argument, it reads from that file +and writes to stdout. Otherwise, it reads from stdin and writes to stdout, and +prompts for each line of input. + +The program handles any number of sets of input on a single input file. Each +set starts with a regular expression, and continues with any number of data +lines to be matched against the pattern. An empty line signals the end of the +set. The regular expressions are given enclosed in any non-alphameric +delimiters other than backslash, for example + + /(a|bc)x+yz/ + +White space before the initial delimiter is ignored. A regular expression may +be continued over several input lines, in which case the newline characters are +included within it. See the testinput files for many examples. It is possible +to include the delimiter within the pattern by escaping it, for example + + /abc\/def/ + +If you do so, the escape and the delimiter form part of the pattern, but since +delimiters are always non-alphameric, this does not affect its interpretation. +If the terminating delimiter is immediately followed by a backslash, for +example, + + /abc/\ + +then a backslash is added to the end of the pattern. This is done to provide a +way of testing the error condition that arises if a pattern finishes with a +backslash, because + + /abc\/ + +is interpreted as the first line of a pattern that starts with "abc/", causing +pcretest to read the next line as a continuation of the regular expression. + +The pattern may be followed by i, m, s, or x to set the PCRE_CASELESS, +PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively. For +example: + + /caseless/i + +These modifier letters have the same effect as they do in Perl. There are +others which set PCRE options that do not correspond to anything in Perl: /A, +/E, and /X set PCRE_ANCHORED, PCRE_DOLLAR_ENDONLY, and PCRE_EXTRA respectively. + +Searching for all possible matches within each subject string can be requested +by the /g or /G modifier. After finding a match, PCRE is called again to search +the remainder of the subject string. The difference between /g and /G is that +the former uses the startoffset argument to pcre_exec() to start searching at +a new point within the entire string (which is in effect what Perl does), +whereas the latter passes over a shortened substring. This makes a difference +to the matching process if the pattern begins with a lookbehind assertion +(including \b or \B). + +If any call to pcre_exec() in a /g or /G sequence matches an empty string, the +next call is done with the PCRE_NOTEMPTY flag set so that it cannot match an +empty string again at the same point. If however, this second match fails, the +start offset is advanced by one, and the match is retried. This imitates the +way Perl handles such cases when using the /g modifier or the split() function. + +There are a number of other modifiers for controlling the way pcretest +operates. + +The /+ modifier requests that as well as outputting the substring that matched +the entire pattern, pcretest should in addition output the remainder of the +subject string. This is useful for tests where the subject contains multiple +copies of the same substring. + +The /L modifier must be followed directly by the name of a locale, for example, + + /pattern/Lfr + +For this reason, it must be the last modifier letter. The given locale is set, +pcre_maketables() is called to build a set of character tables for the locale, +and this is then passed to pcre_compile() when compiling the regular +expression. Without an /L modifier, NULL is passed as the tables pointer; that +is, /L applies only to the expression on which it appears. + +The /I modifier requests that pcretest output information about the compiled +expression (whether it is anchored, has a fixed first character, and so on). It +does this by calling pcre_info() after compiling an expression, and outputting +the information it gets back. If the pattern is studied, the results of that +are also output. + +The /D modifier is a PCRE debugging feature, which also assumes /I. It causes +the internal form of compiled regular expressions to be output after +compilation. + +The /S modifier causes pcre_study() to be called after the expression has been +compiled, and the results used when the expression is matched. + +The /M modifier causes the size of memory block used to hold the compiled +pattern to be output. + +Finally, the /P modifier causes pcretest to call PCRE via the POSIX wrapper API +rather than its native API. When this is done, all other modifiers except /i, +/m, and /+ are ignored. REG_ICASE is set if /i is present, and REG_NEWLINE is +set if /m is present. The wrapper functions force PCRE_DOLLAR_ENDONLY always, +and PCRE_DOTALL unless REG_NEWLINE is set. + +Before each data line is passed to pcre_exec(), leading and trailing whitespace +is removed, and it is then scanned for \ escapes. The following are recognized: + + \a alarm (= BEL) + \b backspace + \e escape + \f formfeed + \n newline + \r carriage return + \t tab + \v vertical tab + \nnn octal character (up to 3 octal digits) + \xhh hexadecimal character (up to 2 hex digits) + + \A pass the PCRE_ANCHORED option to pcre_exec() + \B pass the PCRE_NOTBOL option to pcre_exec() + \Cdd call pcre_copy_substring() for substring dd after a successful match + (any decimal number less than 32) + \Gdd call pcre_get_substring() for substring dd after a successful match + (any decimal number less than 32) + \L call pcre_get_substringlist() after a successful match + \N pass the PCRE_NOTEMPTY option to pcre_exec() + \Odd set the size of the output vector passed to pcre_exec() to dd + (any number of decimal digits) + \Z pass the PCRE_NOTEOL option to pcre_exec() + +A backslash followed by anything else just escapes the anything else. If the +very last character is a backslash, it is ignored. This gives a way of passing +an empty line as data, since a real empty line terminates the data input. + +If /P was present on the regex, causing the POSIX wrapper API to be used, only +\B, and \Z have any effect, causing REG_NOTBOL and REG_NOTEOL to be passed to +regexec() respectively. + +When a match succeeds, pcretest outputs the list of captured substrings that +pcre_exec() returns, starting with number 0 for the string that matched the +whole pattern. Here is an example of an interactive pcretest run. + + $ pcretest + PCRE version 2.06 08-Jun-1999 + + re> /^abc(\d+)/ + data> abc123 + 0: abc123 + 1: 123 + data> xyz + No match + +If the strings contain any non-printing characters, they are output as \0x +escapes. If the pattern has the /+ modifier, then the output for substring 0 is +followed by the the rest of the subject string, identified by "0+" like this: + + re> /cat/+ + data> cataract + 0: cat + 0+ aract + +If the pattern has the /g or /G modifier, the results of successive matching +attempts are output in sequence, like this: + + re> /\Bi(\w\w)/g + data> Mississippi + 0: iss + 1: ss + 0: iss + 1: ss + 0: ipp + 1: pp + +"No match" is output only if the first match attempt fails. + +If any of \C, \G, or \L are present in a data line that is successfully +matched, the substrings extracted by the convenience functions are output with +C, G, or L after the string number instead of a colon. This is in addition to +the normal full list. The string length (that is, the return from the +extraction function) is given in parentheses after each string for \C and \G. + +Note that while patterns can be continued over several lines (a plain ">" +prompt is used for continuations), data lines may not. However newlines can be +included in data by means of the \n escape. + +If the -p option is given to pcretest, it is equivalent to adding /P to each +regular expression: the POSIX wrapper API is used to call PCRE. None of the +following flags has any effect in this case. + +If the option -d is given to pcretest, it is equivalent to adding /D to each +regular expression: the internal form is output after compilation. + +If the option -i is given to pcretest, it is equivalent to adding /I to each +regular expression: information about the compiled pattern is given after +compilation. + +If the option -m is given to pcretest, it outputs the size of each compiled +pattern after it has been compiled. It is equivalent to adding /M to each +regular expression. For compatibility with earlier versions of pcretest, -s is +a synonym for -m. + +If the -t option is given, each compile, study, and match is run 20000 times +while being timed, and the resulting time per compile or match is output in +milliseconds. Do not set -t with -s, because you will then get the size output +20000 times and the timing will be distorted. If you want to change the number +of repetitions used for timing, edit the definition of LOOPREPEAT at the top of +pcretest.c + +Philip Hazel <ph10@cam.ac.uk> +January 2000 |