1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>BLOB support</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" />
<link rel="up" href="am_misc.html" title="Chapter 4. Access Method Wrapup" />
<link rel="prev" href="am_misc_diskspace.html" title="Disk space requirements" />
<link rel="next" href="am_misc_db_sql.html" title="Specifying a Berkeley DB schema using SQL DDL" />
</head>
<body>
<div xmlns="" class="navheader">
<div class="libver">
<p>Library Version 12.1.6.1</p>
</div>
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">BLOB support</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="am_misc_diskspace.html">Prev</a> </td>
<th width="60%" align="center">Chapter 4. Access Method Wrapup </th>
<td width="20%" align="right"> <a accesskey="n" href="am_misc_db_sql.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="blobs"></a>BLOB support</h2>
</div>
</div>
</div>
<div class="toc">
<dl>
<dt>
<span class="sect2">
<a href="blobs.html#blob_threshold">The BLOB threshold</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="blobs.html#blob_create">Creating BLOBs</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="blobs.html#blob_access">BLOB access</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="blobs.html#blob_storage">BLOB storage</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="blobs.html#blob_replication">BLOBs and Replication</a>
</span>
</dt>
</dl>
</div>
<p>
Binary Large Objects (BLOB) support is designed for
efficient storage of large objects. An object is considered to
be large if it is more than a third of the size of a page.
Without BLOB support, large objects must be broken up into
smaller pieces, and then reassembled and/or disassembled every
time the record is read or updated. Berkeley DB BLOB support
avoids this assembly/disassembly process by storing the large
object in a special directory set aside for the purpose. The
data itself is not kept in the database, nor is it placed into
the in-memory cache.
</p>
<p>
BLOBs can only be stored using the data portion of a
key/data pair. They are supported only for Btree, Hash, and
Heap databases, and only so long as the database is not
configured for checksums, encryption, duplicate records, or
duplicate sorted records. In addition, the DBT that you use to
access the BLOB data cannot be configured as a partial DBT if
you want to access the data using the BLOB's streaming
interface (introduced below).
</p>
<p>
Note that if the environment is transactionally-protected,
then all access to the BLOB is also transactionally protected.
</p>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="blob_threshold"></a>The BLOB threshold</h3>
</div>
</div>
</div>
<p>
The BLOB threshold is a positive integer, in bytes,
which indicates how large an object must be before it is
considered a BLOB. By default, the BLOB threshold for any
given database is <code class="literal">0</code>, which means that
no object will ever be considered a BLOB. This means that
the BLOB feature is not used by default for Berkeley DB
databases.
</p>
<p>
In order to use the BLOB feature, you must set the BLOB
threshold to a non-zero, positive integer value. You do
this for a given database using the <a href="../api_reference/C/set_blob_threshold.html" class="olink">DB->set_blob_threshold()</a>
method. Note that this value must be set before you create
the database. At any point after database creation time,
this method is ignored.
</p>
<p>
In addition, if you are using an environment, you can
change the default threshold for databases created in that
environment to something other than <code class="literal">0</code>
by using the <a href="../api_reference/C/envset_blob_threshold.html" class="olink">DB_ENV->set_blob_threshold()</a> method.
</p>
<p> You can retrieve the BLOB threshold set for a database
using the <a href="../api_reference/C/get_blob_threshold.html" class="olink">DB->get_blob_threshold()</a>. You can retrieve the
default BLOB threshold set for your environment using the
<a href="../api_reference/C/envget_blob_threshold.html" class="olink">DB_ENV->get_blob_threshold()</a>.
</p>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="blob_create"></a>Creating BLOBs</h3>
</div>
</div>
</div>
<p>
There are two ways to create a BLOB. Before you can use
either mechanism, you must set the BLOB threshold to a
non-zero positive integer value (see the previous section
for details). Once the BLOB threshold has been set, you
create a BLOB using one of the two following mechanisms:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
<p>
Configure the DBT used to access the BLOB data
(that is, the DBT used for the data portion of the
record) with the <a href="../api_reference/C/dbt.html#dbt_DB_DBT_BLOB" class="olink">DB_DBT_BLOB</a> flag. This causes
the data to be stored as a BLOB regardless of its
size, so long as the database otherwise supports
BLOBs.
</p>
</li>
<li>
<p>
Alternatively, creating a data item with a size
greater than the BLOB threshold will cause that
data item to be automatically stored as a BLOB.
</p>
</li>
</ul>
</div>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="blob_access"></a>BLOB access</h3>
</div>
</div>
</div>
<p>
BLOBs may be accessed in the same way as other DBT
data, so long as the data itself will fit into memory.
More likely, you will find it necessary to use the BLOB
streaming API to read and write BLOB data. You open a BLOB
stream using the <a href="../api_reference/C/dbstream.html" class="olink">DBC->db_stream()</a> method, close it with the
<a href="../api_reference/C/dbstream_close.html" class="olink">DB_STREAM->close()</a> method, write to it using the the
<a href="../api_reference/C/dbstream_write.html" class="olink">DB_STREAM->write()</a> method, and read it using the
<a href="../api_reference/C/dbstream_read.html" class="olink">DB_STREAM->read()</a> method.
</p>
<p>
The following example code fragment can be found in
your DB distribution at
<code class="filename">.../db/examples/c/ex_blob.c</code>.
</p>
<pre class="programlisting">...
/* Some necessary variable declarations */
DBC *dbc; /* Cursor handle */
DB_ENV *dbenv; /* Environment handle */
DB *dbp; /* Database handle */
DB_STREAM *dbs; /* Stream handle */
DB_TXN *txn; /* Transaction handle */
DBT data, key; /* DBT handles */
int ret;
db_off_t size;
...
/* Environment creation skipped for brevity's sake */
...
/* Enable blob files and set the size threshold. */
if ((ret = dbenv->set_blob_threshold(dbenv, 1000, 0)) != 0) {
dbenv->err(dbenv, ret, "set_blob_threshold");
goto err;
}
...
/* Database and DBT creation skipped for brevity's sake */
...
/*
Access the BLOB using the DB_STREAM API.
*/
if ((ret = dbenv->txn_begin(dbenv, NULL, &txn, 0)) != 0){
dbenv->err(dbenv, ret, "txn");
goto err;
}
if ((ret = dbp->cursor(dbp, txn, &dbc, 0)) != 0) {
dbenv->err(dbenv, ret, "cursor");
goto err;
}
/*
* Set the cursor to a blob. Use DB_DBT_PARTIAL with
* dlen == 0 to avoid getting any blob data.
*/
data.flags = DB_DBT_USERMEM | DB_DBT_PARTIAL;
data.dlen = 0;
if ((ret = dbc->get(dbc, &key, &data, DB_FIRST)) != 0) {
dbenv->err(dbenv, ret, "Not a blob");
goto err;
}
data.flags = DB_DBT_USERMEM;
/* Create a stream on the blob the cursor points to. */
if ((ret = dbc->db_stream(dbc, &dbs, DB_STREAM_WRITE)) != 0) {
dbenv->err(dbenv, 0, "Creating stream.");
goto err;
}
/* Get the size of the blob. */
if ((ret = dbs->size(dbs, &size, 0)) != 0) {
dbenv->err(dbenv, 0, "Stream size.");
goto err;
}
/* Read from the blob. */
if ((ret = dbs->read(dbs, &data, 0, (u_int32_t)size, 0)) != 0) {
dbenv->err(dbenv, 0, "Stream read.");
goto err;
}
/* Write data to the blob, increasing its size. */
if ((ret = dbs->write(dbs, &data, size/2, 0)) != 0) {
dbenv->err(dbenv, 0, "Stream write.");
goto err;
}
/* Close the stream. */
if ((ret = dbs->close(dbs, 0)) != 0) {
dbenv->err(dbenv, 0, "Stream close.");
goto err;
}
dbs = NULL;
dbc->close(dbc);
dbc = NULL;
txn->commit(txn, 0);
txn = NULL;
free(data.data);
data.data = NULL;
...
/* Handle clean up skipped. */ </pre>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="blob_storage"></a>BLOB storage</h3>
</div>
</div>
</div>
<p>
BLOBs are not stored in the normal database files on
disk in the same way as is other data managed by DB.
Instead, they are stored as binary files in a special
directory set aside for the purpose.
</p>
<p>
If you are not using environments, this special BLOB
directory is created relative to the current working
directory from which your application is running. You can
modify this default location using the <a href="../api_reference/C/set_blob_dir.html" class="olink">DB->set_blob_dir()</a>
method, and retrieve the current BLOB directory using
<a href="../api_reference/C/get_blob_dir.html" class="olink">DB->get_blob_dir()</a>.
</p>
<p>
If you are using an environment, then by default the
BLOB directory is created within the environment's home
directory. You can change this default location using
<a href="../api_reference/C/envset_blob_dir.html" class="olink">DB_ENV->set_blob_dir()</a> and retrieve the current default
location using <a href="../api_reference/C/envget_blob_dir.html" class="olink">DB_ENV->get_blob_dir()</a>. (Note that
<a href="../api_reference/C/envget_blob_dir.html" class="olink">DB_ENV->get_blob_dir()</a> can successfully retrieve the BLOB
directory only if <a href="../api_reference/C/envset_blob_dir.html" class="olink">DB_ENV->set_blob_dir()</a> was previously
called.)
</p>
<p>
Note that because BLOBs are stored outside of the
Berkeley DB database files, they are not confined by the
four gigabyte limit used for Berkeley DB key and data
items. The BLOB size limit is system dependent. It can be
the maximum value in bytes of a signed 32 bit integer (if
the Berkeley DB-defined type <code class="literal">db_off_t</code>
is four bytes in size), or a signed 64 bit integer (if
<code class="literal">db_off_t</code> is eight bytes in size).
</p>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="blob_replication"></a>BLOBs and Replication</h3>
</div>
</div>
</div>
<p>
Replication supports BLOBs without any special requirements.
However, enabling BLOBs in a replicated environment can
result in long synchronization times between the client
and master sites. To avoid this, execute a transaction
checkpoint after updating or deleting one or more BLOB
records.
</p>
</div>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="am_misc_diskspace.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="am_misc.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="am_misc_db_sql.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Disk space
requirements </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> Specifying a Berkeley DB schema
using SQL DDL</td>
</tr>
</table>
</div>
</body>
</html>
|
