1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
|
/*! @page tune_cache Cache and eviction tuning
@section tuning_cache_size Cache size
The size of the cache is the single most important tuning knob for a
WiredTiger application. Ideally the cache should be configured to be
large enough to hold an application's working set.
The cache size for the database is normally configured by setting the
\c cache_size configuration string when calling the ::wiredtiger_open
function. The cache size can be adjusted after the open call with
WT_CONNECTION::reconfigure.
An example of setting a cache size to 5GB:
@snippet ex_all.c Open a connection
The effectiveness of the chosen cache size can be measured by reviewing
the page eviction statistics for the database.
@section tuning_cache_resident Cache resident objects
Objects can be created as cache resident - that is their contents will
remain in cache and never be considered for the purposes of cache
eviction. Cache residence can be configured with the WT_SESSION::create
"cache_resident" configuration string. LSM tree objects do not support
the "cache_resident" setting.
Configuring a cache resident object has several effects:
- Once the object's pages have been created or instantiated in memory
no further I/O cost is ever paid for object access, minimizing
potential latency.
- Cache resident objects can be accessed faster than objects tracked for
potential eviction.
- If cache resident objects require a significant proportion of the
configured cache size then non cache-resident objects can incur
significantly higher I/O churn.
- If cache resident objects require more space than the configured cache
size, then further operations will either return error or stall until
space is made available by closing objects.
An example of configuring a cache-resident object:
@snippet ex_all.c Create a cache-resident object
@section cache_eviction Eviction tuning
When an application approaches the maximum cache size, WiredTiger begins
eviction to stop memory use from growing too large, approximating a
least-recently-used algorithm.
WiredTiger provides several configuration options for tuning how pages
are evicted from the cache. Different settings will improve performance
depending on an application's particular workload. Customizing the
eviction configuration settings can reduce latency spikes in
application threads and can improve throughput in some applications.
WiredTiger eviction tuning options can be configured when first opening
a database via ::wiredtiger_open, or changed after open with
WT_CONNECTION::reconfigure.
The \c eviction_target configuration value (default 80%) is the level at
which WiredTiger attempts to keep the overall cache usage. Eviction worker
threads are active when the cache contains at least this much content,
expressed as a percentage of the total cache size.
The \c eviction_trigger configuration value (default 95%) is the level at
which application threads start to perform eviction. This will throttle
application operations, increasing operation latency, usually resulting in
the cache usage staying at this level when there is more cache pressure
than eviction worker threads can handle in the background.
Operations will stall when the cache reaches 100% of the cache size.
Application may want to change these settings from their defaults to either
increase the range in which worker threads operate before application
threads are throttled, or to use a larger proportion of RAM, if eviction
worker threads have no difficulty handling the cache pressure generated by
the application.
The \c eviction_dirty_target (default 5%) and \c eviction_dirty_trigger
(default 20%) operate in a similar way to the overall targets, but only
apply to dirty data in cache. In particular, application threads will be
throttled if the percentage of dirty data reaches the
\c eviction_dirty_trigger. Any page that has been modified since it was
read from disk is considered dirty.
The dirty eviction settings control how much work checkpoints have to do
in the worst case, and also limit how much memory fragmentation is likely
for memory allocations related to the cache. Most memory fragmentation is
caused by workloads that generate a mix of updates (small allocations) with
cache misses (large allocations). Limiting the percentage of cache that
can be dirty limits the worst case fragmentation to the approximately the
same level.
@snippet ex_all.c Eviction configuration
By default, WiredTiger cache eviction is handled by a single, separate
thread. In a large, busy cache, a single thread will be insufficient
(especially when the eviction thread must wait for I/O). The
\c eviction=(threads_min) and \c eviction=(threads_max) configuration
values can be used to configure the minimum and maximum number of
additional threads WiredTiger will create to keep up with the
application eviction load. Finally, if the Wiredtiger eviction threads
are unable to keep up with application demand for cache space,
application threads will be tasked with eviction as well, potentially
resulting in latency spikes.
@snippet ex_all.c Eviction worker configuration
*/
|
