summaryrefslogtreecommitdiff
path: root/src/third_party/wiredtiger/src/docs/compression.dox
blob: 56715e207527f6a1e900af4f47e3e4c8da01166c (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
/*! @m_page{{c,java},compression,Compressors}

This section explains how to configure WiredTiger's builtin support for
the bzip2, lz4, snappy and zlib compression engines.

@section compression_bzip2 Using bzip2 compression

To use the builtin support for
<a href="http://www.bzip.org/">Julian Seward's bzip2</a>
compression, first check that bzip2 is installed in include and library
directories searched by the compiler.  Once bzip2 is installed, you can
enable bzip2 using the \c --enable-bzip2 option to configure.

If bzip2 is installed in a location not normally searched by the
compiler toolchain, you'll need to modify the \c CPPFLAGS and \c LDFLAGS
to indicate these locations.  For example, with the bzip2 includes and
libraries installed in \c /usr/local/include and \c /usr/local/lib, you
would run configure with the following additional arguments:

@code
--enable-bzip2 CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/include"
@endcode

When opening the WiredTiger database, load the bzip2 shared library as
an extension.  For example, with the WiredTiger library installed in
\c /usr/local/lib, you would use the following extension:

@snippet ex_all.c Configure bzip2 extension

Finally, when creating the WiredTiger object, set \c block_compressor
to \c bzip2:

@snippet ex_all.c Create a bzip2 compressed table

@section compression_lz4 Using LZ4 compression

To use the builtin support for Yann Collet's
<a href="https://github.com/Cyan4973/lz4">LZ4</a>
compression, first check that LZ4 is installed in include and library
directories searched by the compiler.  Once LZ4 is installed, you can
enable LZ4 using the \c --enable-lz4 option to configure.

If LZ4 is installed in a location not normally searched by the
compiler toolchain, you'll need to modify the \c CPPFLAGS and \c LDFLAGS
to indicate these locations.  For example, with the LZ4 includes and
libraries installed in \c /usr/local/include and \c /usr/local/lib, you
would run configure with the following additional arguments:

@code
--enable-lz4 CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/include"
@endcode

When opening the WiredTiger database, load the LZ4 shared library as
an extension.  For example, with the WiredTiger library installed in
\c /usr/local/lib, you would use the following extension:

@snippet ex_all.c Configure lz4 extension

Finally, when creating the WiredTiger object, set \c block_compressor
to \c lz4:

@snippet ex_all.c Create a lz4 compressed table

@section compression_snappy Using snappy compression

To use the builtin support for
<a href="http://code.google.com/p/snappy/">Google's snappy</a>
compression, first check that snappy is installed in include and library
directories searched by the compiler.  Once snappy is installed, you can
enable snappy using the \c --enable-snappy option to configure.

If snappy is installed in a location not normally searched by the
compiler toolchain, you'll need to modify the \c CPPFLAGS and \c LDFLAGS
to indicate these locations.  For example, with the snappy includes and
libraries installed in \c /usr/local/include and \c /usr/local/lib, you
would run configure with the following additional arguments:

@code
--enable-snappy CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/include"
@endcode

When opening the WiredTiger database, load the snappy shared library as
an extension.  For example, with the WiredTiger library installed in
\c /usr/local/lib, you would use the following extension:

@snippet ex_all.c Configure snappy extension

Finally, when creating the WiredTiger object, set \c block_compressor
to \c snappy:

@snippet ex_all.c Create a snappy compressed table

@section compression_zlib Using zlib compression

To use the builtin support for Greg Roelofs' and Mark Adler's
<a href="http://www.zlib.net/">zlib</a>
compression, first check that zlib is installed in include and library
directories searched by the compiler.  Once zlib is installed, you can
enable zlib using the \c --enable-zlib option to configure.

If zlib is installed in a location not normally searched by the compiler
toolchain, you'll need to modify the \c CPPFLAGS and \c LDFLAGS to
indicate these locations.  For example, with the zlib includes and
libraries installed in \c /usr/local/include and \c /usr/local/lib, you
would run configure with the following additional arguments:

@code
--enable-zlib CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/include"
@endcode

When opening the WiredTiger database, load the zlib shared library as
an extension.  For example, with the WiredTiger library installed in
\c /usr/local/lib, you would use the following extension:

@snippet ex_all.c Configure zlib extension

Finally, when creating the WiredTiger object, set \c block_compressor
to \c zlib:

@snippet ex_all.c Create a zlib compressed table

@section compression_upgrading Upgrading compression engines

WiredTiger does not store information with file blocks to identify the
compression engine used to compressed the block.  Applications wanting
to upgrade to some future compression engine (without requiring a file
dump and re-load), should ensure each compressed block includes enough
information to identify the compression engine used, so its compression
code can correctly decompress old and new blocks.

@section compression_custom Custom compression engines

WiredTiger may be extended by adding custom compression engines; see
WT_COMPRESSOR for more information.

 */