blob: 21af5d1f62feeabe79b410ee09dd496de706b3b3 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
|
/*! @page arch-index WiredTiger Architecture Guide
WiredTiger is built as a library that is linked into the application. Applications
use the API interface to direct WiredTiger's operations (see @ref basic_api).
Here is an overview of the software components in WiredTiger and how they are organized.
An arrow indicates the "from" component uses "to" component.
<div class="arch_diagram">
@plantuml_start{wt_diagram.png}
@startuml{wt_diagram.png}
' We add spacing to the diagram in places to influence the layout.
' To do this, we create some invisible components with hidden arrows
' pointing to them. Since we don't otherwise use the "file" component,
' we set all its parts to be transparent, and any use of "file" results
' in an invisible spacer whose width is directed by the length of its label.
' When modifying this diagram, it's sometimes useful to comment out the
' following lines, and any [hidden] directives used below, to see how
' the spacers influence the layout. Note that this may be fragile;
' the spacers give hints to the layout, such hints will not always be honored.
skinparam fileBorderColor Transparent
skinparam fileBackgroundColor Transparent
skinparam fileFontColor Transparent
skinparam fileShadowing false
' Our diagram is simple. First, we define lots of labeled rectangles
' with most nesting within the "engine" rectangle.
together {
rectangle "[[arch-python.html Python API]]" as python_api
' "storage" displays as an oval.
storage " C/C++ \n applications " as application
rectangle "[[command_line.html wt Utility]]" as utility
}
' Trailing spaces for this label puts the text to the left.
rectangle "**WiredTiger Engine** " as wt_engine {
' Leading and trailing spaces make a wide rectangle.
together {
' Putting two invisible file boxes on either side centers the middle box.
file "____" as SPACE_api
rectangle " [[modules.html C API]] " as c_api
file "____" as SPACE_api2
' Influence the ordering of the invisible boxes using (hidden) arrows.
SPACE_api -[hidden]right-> c_api
c_api -[hidden]right-> SPACE_api2
}
rectangle "[[arch-schema.html Schema]]" as schema
rectangle "[[arch-cursor.html Cursor]]" as cursor
rectangle "[[arch-transaction.html Transactions]]" as txn
rectangle "[[arch-metadata.html Metadata]]" as meta
rectangle "[[arch-dhandle.html dhandle/\nBtree]]" as btree
rectangle "[[arch-row.html Row\nStorage]]" as row
rectangle "[[arch-column.html Column\nStorage]]" as column
rectangle "[[arch-hs.html History\nStore]]" as history
rectangle "[[arch-snapshot.html Snapshots]]" as snapshot
rectangle "[[arch-cache.html Cache]]" as cache
rectangle "[[arch-eviction.html Eviction]]" as evict
together {
rectangle "[[arch-block.html Block\nManager]]" as block
file "__________" as SPACE_log
rectangle "[[arch-logging.html Logging]]" as log
file "___" as SPACE_log2
}
rectangle " [[arch-fs-os.html File System & OS interface]] " as os
}
together {
database "[[arch-data-file.html Database\nFiles]]" as wt_file
database " [[arch-log-file.html Log\nFiles]]" as log_file
}
' Influence the ordering at the top using (hidden) arrows.
python_api -[hidden]right-> application
application -[hidden]right-> utility
python_api -down-> c_api
application -down-> c_api
utility -down-> c_api
c_api -down-> schema
c_api -down-> cursor
c_api -down-> txn
schema -down-> meta
schema -down-> btree
cursor -down-> btree
btree -down-> row
btree -down-> column
meta -up-> cursor
' The hidden arrow helps our boxes to line up in a better way.
meta -[hidden]right-> btree
cursor -[hidden]right-> txn
txn -down-> snapshot
row -down-> cache
column -down-> cache
cache -down-> history
evict -down-> history
history -up-> cursor
snapshot -down-> evict
cache -right-> evict
cache -down-> block
evict -down-> block
txn -down-> log
block -[hidden]right-> SPACE_log
cache -[hidden]down-> SPACE_log
evict -[hidden]down-> SPACE_log
SPACE_log -[hidden]right-> log
log -[hidden]right-> SPACE_log2
block -down-> os
log -down-> os
os -down-> wt_file
os -down-> log_file
wt_file -[hidden]right-> log_file
@enduml
@plantuml_end
</div>
We go into some detail for some of the internal components.
@subpage arch-block
The Block Manager manages the reading and writing of disk blocks.
@subpage arch-cache
Cache is represented by the various shared data structures that
make up in-memory Btrees and subordinate data structures.
@subpage arch-column
Column Stores are Btrees that have as their key a record id.
@subpage arch-cursor
Cursors are used to get and modify data.
@subpage arch-data-file
The format of the data file is given by structures in \c block.h .
@subpage arch-dhandle
An internal structure called a Data Handle (dhandle) is used to represent and
access Btrees and other data sources in WiredTiger.
@subpage arch-eviction
Eviction represents the process or removing old data from the cache,
writing it to disk if it is dirty.
@subpage arch-fs-os
A layer of abstraction is above all operating system calls and
a set of functions can be registered to be called for each file system
operation.
@subpage arch-hs
The History Store tracks old versions of records.
@subpage arch-logging
WiredTiger writes all changes into a write-ahead log when configured.
@subpage arch-log-file
The format of a log file is defined in \c log.h .
@subpage arch-metadata
Metadata is stored as <code>uri, config</code> K/V pairs in a designated table.
@subpage arch-python
WiredTiger has a Python API that is useful for scripting and experimentation.
@subpage arch-row
Row Stores are Btrees that have a variable size key and data.
@subpage arch-schema
A schema defines the format of the application data in WiredTiger.
@subpage arch-snapshot
Snapshots are implemented by storing transaction ids committed before
the transaction started.
@subpage arch-transaction
Transactions provide a powerful abstraction for multiple threads to operate on data concurrently.
@subpage arch-glossary
WiredTiger assigns specific meanings to certain words. Here we decode them.
*/
|
