1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
|
Small cloud with trusted tenants
================================
Story
-----
As an operator I would like to build a small cloud with both virtual and bare
metal instances or add bare metal provisioning to my existing small or medium
scale single-site OpenStack cloud. The expected number of bare metal machines
is less than 100, and the rate of provisioning and unprovisioning is expected
to be low. All users of my cloud are trusted by me to not conduct malicious
actions towards each other or the cloud infrastructure itself.
As a user I would like to occasionally provision bare metal instances through
the Compute API by selecting an appropriate Compute flavor. I would like
to be able to boot them from images provided by the Image service or from
volumes provided by the Volume service.
Components
----------
This architecture assumes `an OpenStack installation`_ with the following
components participating in the bare metal provisioning:
* The `Compute service`_ manages bare metal instances.
* The `Networking service`_ provides DHCP for bare metal instances.
* The `Image service`_ provides images for bare metal instances.
The following services can be optionally used by the Bare Metal service:
* The `Volume service`_ provides volumes to boot bare metal instances from.
* The `Bare Metal Introspection service`_ simplifies enrolling new bare metal
machines by conducting in-band introspection.
Node roles
----------
An OpenStack installation in this guide has at least these three types of
nodes:
* A *controller* node hosts the control plane services.
* A *compute* node runs the virtual machines and hosts a subset of Compute
and Networking components.
* A *block storage* node provides persistent storage space for both virtual
and bare metal nodes.
The *compute* and *block storage* nodes are configured as described in the
installation guides of the `Compute service`_ and the `Volume service`_
respectively. The *controller* nodes host the Bare Metal service components.
Networking
----------
The networking architecture will highly depend on the exact operating
requirements. This guide expects the following existing networks:
*control plane*, *storage* and *public*. Additionally, two more networks
will be needed specifically for bare metal provisioning: *bare metal* and
*management*.
.. TODO(dtantsur): describe the storage network?
.. TODO(dtantsur): a nice picture to illustrate the layout
Control plane network
~~~~~~~~~~~~~~~~~~~~~
The *control plane network* is the network where OpenStack control plane
services provide their public API.
The Bare Metal API will be served to the operators and to the Compute service
through this network.
Public network
~~~~~~~~~~~~~~
The *public network* is used in a typical OpenStack deployment to create
floating IPs for outside access to instances. Its role is the same for a bare
metal deployment.
.. note::
Since, as explained below, bare metal nodes will be put on a flat provider
network, it is also possible to organize direct access to them, without
using floating IPs and bypassing the Networking service completely.
Bare metal network
~~~~~~~~~~~~~~~~~~
The *Bare metal network* is a dedicated network for bare metal nodes managed by
the Bare Metal service.
This architecture uses :ref:`flat bare metal networking <network-interfaces>`,
in which both tenant traffic and technical traffic related to the Bare Metal
service operation flow through this one network. Specifically, this network
will serve as the *provisioning*, *cleaning* and *rescuing* network. It will
also be used for introspection via the Bare Metal Introspection service.
See :ref:`common networking considerations <refarch-common-networking>` for
an in-depth explanation of the networks used by the Bare Metal service.
DHCP and boot parameters will be provided on this network by the Networking
service's DHCP agents.
For booting from volumes this network has to have a route to
the *storage network*.
Management network
~~~~~~~~~~~~~~~~~~
*Management network* is an independent network on which BMCs of the bare
metal nodes are located.
The ``ironic-conductor`` process needs access to this network. The tenants
of the bare metal nodes must not have access to it.
.. note::
The :ref:`direct deploy interface <direct-deploy>` and certain
:doc:`/admin/drivers` require the *management network* to have access
to the Object storage service backend.
Controllers
-----------
A *controller* hosts the OpenStack control plane services as described in the
`control plane design guide`_. While this architecture allows using
*controllers* in a non-HA configuration, it is recommended to have at least
three of them for HA. See :ref:`refarch-common-ha` for more details.
Bare Metal services
~~~~~~~~~~~~~~~~~~~
The following components of the Bare Metal service are installed on a
*controller* (see :ref:`components of the Bare Metal service
<refarch-common-components>`):
* The Bare Metal API service either as a WSGI application or the ``ironic-api``
process. Typically, a load balancer, such as HAProxy, spreads the load
between the API instances on the *controllers*.
The API has to be served on the *control plane network*. Additionally,
it has to be exposed to the *bare metal network* for the ramdisk callback
API.
* The ``ironic-conductor`` process. These processes work in active/active HA
mode as explained in :ref:`refarch-common-ha`, thus they can be installed on
all *controllers*. Each will handle a subset of bare metal nodes.
The ``ironic-conductor`` processes have to have access to the following
networks:
* *control plane* for interacting with other services
* *management* for contacting node's BMCs
* *bare metal* for contacting deployment, cleaning or rescue ramdisks
* TFTP and HTTP service for booting the nodes. Each ``ironic-conductor``
process has to have a matching TFTP and HTTP service. They should be exposed
only to the *bare metal network* and must not be behind a load balancer.
* The ``nova-compute`` process (from the Compute service). These processes work
in active/active HA mode when dealing with bare metal nodes, thus they can be
installed on all *controllers*. Each will handle a subset of bare metal
nodes.
.. note::
There is no 1-1 mapping between ``ironic-conductor`` and ``nova-compute``
processes, as they communicate only through the Bare Metal API service.
* The networking-baremetal_ ML2 plugin should be loaded into the Networking
service to assist with binding bare metal ports.
The ironic-neutron-agent_ service should be started as well.
* If the Bare Metal introspection is used, its ``ironic-inspector`` process
has to be installed on all *controllers*. Each such process works as both
Bare Metal Introspection API and conductor service. A load balancer should
be used to spread the API load between *controllers*.
The API has to be served on the *control plane network*. Additionally,
it has to be exposed to the *bare metal network* for the ramdisk callback
API.
.. TODO(dtantsur): a nice picture to illustrate the above
Shared services
~~~~~~~~~~~~~~~
A *controller* also hosts two services required for the normal operation
of OpenStack:
* Database service (MySQL/MariaDB is typically used, but other
enterprise-grade database solutions can be used as well).
All Bare Metal service components need access to the database service.
* Message queue service (RabbitMQ is typically used, but other
enterprise-grade message queue brokers can be used as well).
Both Bare Metal API (WSGI application or ``ironic-api`` process) and
the ``ironic-conductor`` processes need access to the message queue service.
The Bare Metal Introspection service does not need it.
.. note::
These services are required for all OpenStack services. If you're adding
the Bare Metal service to your cloud, you may reuse the existing
database and messaging queue services.
Bare metal nodes
----------------
Each bare metal node must be capable of booting from network, virtual media
or other boot technology supported by the Bare Metal service as explained
in :ref:`refarch-common-boot`. Each node must have one NIC on the *bare metal
network*, and this NIC (and **only** it) must be configured to be able to boot
from network. This is usually done in the *BIOS setup* or a similar firmware
configuration utility. There is no need to alter the boot order, as it is
managed by the Bare Metal service. Other NICs, if present, will not be managed
by OpenStack.
The NIC on the *bare metal network* should have untagged connectivity to it,
since PXE firmware usually does not support VLANs - see
:ref:`refarch-common-networking` for details.
Storage
-------
If your hardware **and** its bare metal :doc:`driver </admin/drivers>` support
booting from remote volumes, please check the driver documentation for
information on how to enable it. It may include routing *management* and/or
*bare metal* networks to the *storage network*.
In case of the standard :ref:`pxe-boot`, booting from remote volumes is done
via iPXE. In that case, the Volume storage backend must support iSCSI_
protocol, and the *bare metal network* has to have a route to the *storage
network*. See :doc:`/admin/boot-from-volume` for more details.
.. _an OpenStack installation: https://docs.openstack.org/arch-design/use-cases/use-case-general-compute.html
.. _Compute service: https://docs.openstack.org/nova/train/
.. _Networking service: https://docs.openstack.org/neutron/train/
.. _Image service: https://docs.openstack.org/glance/train/
.. _Volume service: https://docs.openstack.org/cinder/train/
.. _Bare Metal Introspection service: https://docs.openstack.org/ironic-inspector/train/
.. _control plane design guide: https://docs.openstack.org/arch-design/design-control-plane.html
.. _networking-baremetal: https://docs.openstack.org/networking-baremetal/train/
.. _ironic-neutron-agent: https://docs.openstack.org/networking-baremetal/train/install/index.html#configure-ironic-neutron-agent
.. _iSCSI: https://en.wikipedia.org/wiki/ISCSI
|
