diff options
| author | ph10 <ph10@6239d852-aaf2-0410-a92c-79f79f948069> | 2015-01-23 16:51:47 +0000 |
|---|---|---|
| committer | ph10 <ph10@6239d852-aaf2-0410-a92c-79f79f948069> | 2015-01-23 16:51:47 +0000 |
| commit | 61cb4c76713910670d84bc0e3bb9dad8c2661b37 (patch) | |
| tree | 914eae7e92f8016c26af1da38ef04db5521e7d5e /doc/html/pcre2serialize.html | |
| parent | cb63a5307cf484d7f274a808501be414e297f411 (diff) | |
| download | pcre2-61cb4c76713910670d84bc0e3bb9dad8c2661b37.tar.gz | |
Add serialization functions and tests with updated pcre2test. Fix
PCRE2_INFO_SIZE issues.
git-svn-id: svn://vcs.exim.org/pcre2/code/trunk@185 6239d852-aaf2-0410-a92c-79f79f948069
Diffstat (limited to 'doc/html/pcre2serialize.html')
| -rw-r--r-- | doc/html/pcre2serialize.html | 184 |
1 files changed, 184 insertions, 0 deletions
diff --git a/doc/html/pcre2serialize.html b/doc/html/pcre2serialize.html new file mode 100644 index 0000000..c32ebe0 --- /dev/null +++ b/doc/html/pcre2serialize.html @@ -0,0 +1,184 @@ +<html> +<head> +<title>pcre2serialize specification</title> +</head> +<body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> +<h1>pcre2serialize man page</h1> +<p> +Return to the <a href="index.html">PCRE2 index page</a>. +</p> +<p> +This page is part of the PCRE2 HTML documentation. It was generated +automatically from the original man page. If there is any nonsense in it, +please consult the man page, in case the conversion went wrong. +<br> +<ul> +<li><a name="TOC1" href="#SEC1">SAVING AND RE-USING PRECOMPILED PCRE2 PATTERNS</a> +<li><a name="TOC2" href="#SEC2">SAVING COMPILED PATTERNS</a> +<li><a name="TOC3" href="#SEC3">RE-USING PRECOMPILED PATTERNS</a> +<li><a name="TOC4" href="#SEC4">AUTHOR</a> +<li><a name="TOC5" href="#SEC5">REVISION</a> +</ul> +<br><a name="SEC1" href="#TOC1">SAVING AND RE-USING PRECOMPILED PCRE2 PATTERNS</a><br> +<P> +<b>int32_t pcre2_serialize_decode(pcre2_code **<i>codes</i>,</b> +<b> int32_t <i>number_of_codes</i>, const uint32_t *<i>bytes</i>,</b> +<b> pcre2_general_context *<i>gcontext</i>);</b> +<br> +<br> +<b>int32_t pcre2_serialize_encode(pcre2_code **<i>codes</i>,</b> +<b> int32_t <i>number_of_codes</i>, uint32_t **<i>serialized_bytes</i>,</b> +<b> PCRE2_SIZE *<i>serialized_size</i>, pcre2_general_context *<i>gcontext</i>);</b> +<br> +<br> +<b>void pcre2_serialize_free(uint8_t *<i>bytes</i>);</b> +<br> +<br> +<b>int32_t pcre2_serialize_get_number_of_codes(const uint8_t *<i>bytes</i>);</b> +<br> +<br> +If you are running an application that uses a large number of regular +expression patterns, it may be useful to store them in a precompiled form +instead of having to compile them every time the application is run. However, +if you are using the just-in-time optimization feature, it is not possible to +save and reload the JIT data, because it is position-dependent. In addition, +the host on which the patterns are reloaded must be running the same version of +PCRE2, with the same code unit width, and must also have the same endianness, +pointer width and PCRE2_SIZE type. For example, patterns compiled on a 32-bit +system using PCRE2's 16-bit library cannot be reloaded on a 64-bit system, nor +can they be reloaded using the 8-bit library. +</P> +<br><a name="SEC2" href="#TOC1">SAVING COMPILED PATTERNS</a><br> +<P> +Before compiled patterns can be saved they must be serialized, that is, +converted to a stream of bytes. A single byte stream may contain any number of +compiled patterns, but they must all use the same character tables. A single +copy of the tables is included in the byte stream (its size is 1088 bytes). For +more details of character tables, see the +<a href="pcre2api.html#localesupport">section on locale support</a> +in the +<a href="pcre2api.html"><b>pcre2api</b></a> +documentation. +</P> +<P> +The function <b>pcre2_serialize_encode()</b> creates a serialized byte stream +from a list of compiled patterns. Its first two arguments specify the list, +being a pointer to a vector of pointers to compiled patterns, and the length of +the vector. The third and fourth arguments point to variables which are set to +point to the created byte stream and its length, respectively. The final +argument is a pointer to a general context, which can be used to specify custom +memory mangagement functions. If this argument is NULL, <b>malloc()</b> is used +to obtain memory for the byte stream. The yield of the function is the number +of serialized patterns, or one of the following negative error codes: +<pre> + PCRE2_ERROR_BADDATA the number of patterns is zero or less + PCRE2_ERROR_BADMAGIC mismatch of id bytes in one of the patterns + PCRE2_ERROR_MEMORY memory allocation failed + PCRE2_ERROR_MIXEDTABLES the patterns do not all use the same tables + PCRE2_ERROR_NULL the 1st, 3rd, or 4th argument is NULL +</pre> +PCRE2_ERROR_BADMAGIC means either that a pattern's code has been corrupted, or +that a slot in the vector does not point to a compiled pattern. +</P> +<P> +Once a set of patterns has been serialized you can save the data in any +appropriate manner. Here is sample code that compiles two patterns and writes +them to a file. It assumes that the variable <i>fd</i> refers to a file that is +open for output. The error checking that should be present in a real +application has been omitted for simplicity. +<pre> + int errorcode; + uint8_t *bytes; + PCRE2_SIZE erroroffset; + PCRE2_SIZE bytescount; + pcre2_code *list_of_codes[2]; + list_of_codes[0] = pcre2_compile("first pattern", + PCRE2_ZERO_TERMINATED, 0, &errorcode, &erroroffset, NULL); + list_of_codes[1] = pcre2_compile("second pattern", + PCRE2_ZERO_TERMINATED, 0, &errorcode, &erroroffset, NULL); + errorcode = pcre2_serialize_encode(list_of_codes, 2, &bytes, + &bytescount, NULL); + errorcode = fwrite(bytes, 1, bytescount, fd); +</pre> +Note that the serialized data is binary data that may contain any of the 256 +possible byte values. On systems that make a distinction between binary and +non-binary data, be sure that the file is opened for binary output. +</P> +<P> +Serializing a set of patterns leaves the original data untouched, so they can +still be used for matching. Their memory must eventually be freed in the usual +way by calling <b>pcre2_code_free()</b>. When you have finished with the byte +stream, it too must be freed by calling <b>pcre2_serialize_free()</b>. +</P> +<br><a name="SEC3" href="#TOC1">RE-USING PRECOMPILED PATTERNS</a><br> +<P> +In order to re-use a set of saved patterns you must first make the serialized +byte stream available in main memory (for example, by reading from a file). The +management of this memory block is up to the application. You can use the +<b>pcre2_serialize_get_number_of_codes()</b> function to find out how many +compiled patterns are in the serialized data without actually decoding the +patterns: +<pre> + uint8_t *bytes = <serialized data>; + int32_t number_of_codes = pcre2_serialize_get_number_of_codes(bytes); +</pre> +The <b>pcre2_serialize_decode()</b> function reads a byte stream and recreates +the compiled patterns in new memory blocks, setting pointers to them in a +vector. The first two arguments are a pointer to a suitable vector and its +length, and the third argument points to a byte stream. The final argument is a +pointer to a general context, which can be used to specify custom memory +mangagement functions for the decoded patterns. If this argument is NULL, +<b>malloc()</b> and <b>free()</b> are used. After deserialization, the byte +stream is no longer needed and can be discarded. +<pre> + int32_t number_of_codes; + pcre2_code *list_of_codes[2]; + uint8_t *bytes = <serialized data>; + int32_t number_of_codes = + pcre2_serialize_decode(list_of_codes, 2, bytes, NULL); +</pre> +If the vector is not large enough for all the patterns in the byte stream, it +is filled with those that fit, and the remainder are ignored. The yield of the +function is the number of decoded patterns, or one of the following negative +error codes: +<pre> + PCRE2_ERROR_BADDATA second argument is zero or less + PCRE2_ERROR_BADMAGIC mismatch of id bytes in the data + PCRE2_ERROR_BADMODE mismatch of variable unit size or PCRE2 version + PCRE2_ERROR_MEMORY memory allocation failed + PCRE2_ERROR_NULL first or third argument is NULL +</pre> +PCRE2_ERROR_BADMAGIC may mean that the data is corrupt, or that it was compiled +on a system with different endianness. +</P> +<P> +Decoded patterns can be used for matching in the usual way, and must be freed +by calling <b>pcre2_code_free()</b> as normal. A single copy of the character +tables is used by all the decoded patterns. A reference count is used to +arrange for its memory to be automatically freed when the last pattern is +freed. +</P> +<P> +If a pattern was processed by <b>pcre2_jit_compile()</b> before being +serialized, the JIT data is discarded and so is no longer available after a +save/restore cycle. You can, however, process a restored pattern with +<b>pcre2_jit_compile()</b> if you wish. +</P> +<br><a name="SEC4" href="#TOC1">AUTHOR</a><br> +<P> +Philip Hazel +<br> +University Computing Service +<br> +Cambridge, England. +<br> +</P> +<br><a name="SEC5" href="#TOC1">REVISION</a><br> +<P> +Last updated: 20 January 2015 +<br> +Copyright © 1997-2015 University of Cambridge. +<br> +<p> +Return to the <a href="index.html">PCRE2 index page</a>. +</p> |