summaryrefslogtreecommitdiff
path: root/doc/source/install/configure-pxe.rst
blob: 59346ce140e2b43e9f081e29a8a78c57d4f1a784 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
Configuring PXE and iPXE
========================

DHCP server setup
-----------------

A DHCP server is required by PXE/iPXE client. You need to follow steps below.

#. Set the ``[dhcp]/dhcp_provider`` to ``neutron`` in the Bare Metal Service's
   configuration file (``/etc/ironic/ironic.conf``):

   .. note::
    Refer :doc:`/install/configure-tenant-networks` for details. The
    ``dhcp_provider`` configuration is already set by the configuration
    defaults, and when you create subnet, DHCP is also enabled if you do not add
    any dhcp options at "openstack subnet create" command.

#. Enable DHCP in the subnet of PXE network.

#. Set the ip address range in the subnet for DHCP.

   .. note::
    Refer :doc:`/install/configure-networking` for details about the two
    precedent steps.

#. Connect the openstack DHCP agent to the external network through the OVS
   bridges and the interface ``eth2``.

   .. note::
    Refer :doc:`/install/configure-networking` for details. You do not require
    this part if br-int, br-eth2 and eth2 are already connected.


#. Configure the host ip at ``br-eth2``. If it locates at ``eth2``, do below::

    ip addr del 192.168.2.10/24 dev eth2
    ip addr add 192.168.2.10/24 dev br-eth2

   .. note::
    Replace eth2 with the interface on the network node which you are using to
    connect to the Bare Metal service.

TFTP server setup
-----------------

In order to deploy instances via PXE, a TFTP server needs to be
set up on the Bare Metal service nodes which run the ``ironic-conductor``.

#. Make sure the tftp root directory exist and can be written to by the
   user the ``ironic-conductor`` is running as. For example::

    sudo mkdir -p /tftpboot
    sudo chown -R ironic /tftpboot

#. Install tftp server:

   Ubuntu::

       sudo apt-get install xinetd tftpd-hpa

   RHEL8/CentOS8/Fedora::

       sudo dnf install tftp-server xinetd

   SUSE::

       sudo zypper install tftp xinetd

#. Using xinetd to provide a tftp server setup to serve ``/tftpboot``.
   Create or edit ``/etc/xinetd.d/tftp`` as below::

    service tftp
    {
      protocol        = udp
      port            = 69
      socket_type     = dgram
      wait            = yes
      user            = root
      server          = /usr/sbin/in.tftpd
      server_args     = -v -v -v -v -v --map-file /tftpboot/map-file /tftpboot
      disable         = no
      # This is a workaround for Fedora, where TFTP will listen only on
      # IPv6 endpoint, if IPv4 flag is not used.
      flags           = IPv4
    }

   and restart the ``xinetd`` service:

   Ubuntu::

       sudo service xinetd restart

   Fedora/RHEL8/CentOS8/SUSE::

       sudo systemctl restart xinetd

   .. note::

    In certain environments the network's MTU may cause TFTP UDP packets to get
    fragmented. Certain PXE firmwares struggle to reconstruct the fragmented
    packets which can cause significant slow down or even prevent the server
    from PXE booting. In order to avoid this, TFTPd provides an option to limit
    the packet size so that it they do not get fragmented. To set this
    additional option in the server_args above::

      --blocksize <MAX MTU minus 32>

#. Create a map file in the tftp boot directory (``/tftpboot``)::

    echo 're ^(/tftpboot/) /tftpboot/\2' > /tftpboot/map-file
    echo 're ^/tftpboot/ /tftpboot/' >> /tftpboot/map-file
    echo 're ^(^/) /tftpboot/\1' >> /tftpboot/map-file
    echo 're ^([^/]) /tftpboot/\1' >> /tftpboot/map-file

.. _uefi-pxe-grub:

UEFI PXE - Grub setup
---------------------

In order to deploy instances with PXE on bare metal nodes which support
UEFI, perform these additional steps on the ironic conductor node to configure
the PXE UEFI environment.

#. Install Grub2 and shim packages:

   Ubuntu (18.04LTS and later)::

       sudo apt-get install grub-efi-amd64-signed shim-signed

   RHEL8/CentOS8/Fedora::

       sudo dnf install grub2-efi shim

   SUSE::

       sudo zypper install grub2-x86_64-efi shim

#. Copy grub and shim boot loader images to ``/tftpboot`` directory:

   Ubuntu (18.04LTS and later)::

       sudo cp /usr/lib/shim/shimx64.efi.signed /tftpboot/bootx64.efi
       sudo cp /usr/lib/grub/x86_64-efi-signed/grubnetx64.efi.signed /tftpboot/grubx64.efi

   Fedora::

       sudo cp /boot/efi/EFI/fedora/shim.efi /tftpboot/bootx64.efi
       sudo cp /boot/efi/EFI/fedora/grubx64.efi /tftpboot/grubx64.efi

   RHEL8/CentOS8::

       sudo cp /boot/efi/EFI/centos/shim.efi /tftpboot/bootx64.efi
       sudo cp /boot/efi/EFI/centos/grubx64.efi /tftpboot/grubx64.efi

   SUSE::

       sudo cp /usr/lib64/efi/shim.efi /tftpboot/bootx64.efi
       sudo cp /usr/lib/grub2/x86_64-efi/grub.efi /tftpboot/grubx64.efi

#. Update the bare metal node with ``boot_mode:uefi`` capability in
   node's properties field. See :ref:`boot_mode_support` for details.

#. Make sure that bare metal node is configured to boot in UEFI boot mode and
   boot device is set to network/pxe.

   .. note::
    Some drivers, e.g. ``ilo``, ``irmc`` and ``redfish``, support automatic
    setting of the boot mode during deployment. This step is not required
    for them. Please check :doc:`../admin/drivers` for information on whether
    your driver requires manual UEFI configuration.


Legacy BIOS - Syslinux setup
----------------------------

In order to deploy instances with PXE on bare metal using Legacy BIOS boot
mode, perform these additional steps on the ironic conductor node.

#. Install the syslinux package with the PXE boot images:

   Ubuntu (16.04LTS and later)::

       sudo apt-get install syslinux-common pxelinux

   RHEL8/CentOS8/Fedora::

       sudo dnf install syslinux-tftpboot

   SUSE::

       sudo zypper install syslinux

#. Copy the PXE image to ``/tftpboot``. The PXE image might be found at [1]_:

   Ubuntu (16.04LTS and later)::

       sudo cp /usr/lib/PXELINUX/pxelinux.0 /tftpboot

   RHEL8/CentOS8/SUSE::

       sudo cp /usr/share/syslinux/pxelinux.0 /tftpboot

#. If whole disk images need to be deployed via PXE-netboot, copy the
   chain.c32 image to ``/tftpboot`` to support it:

   Ubuntu (16.04LTS and later)::

       sudo cp /usr/lib/syslinux/modules/bios/chain.c32 /tftpboot

   Fedora::

       sudo cp /boot/extlinux/chain.c32 /tftpboot

   RHEL8/CentOS8/SUSE::

       sudo cp /usr/share/syslinux/chain.c32 /tftpboot/

#. If the version of syslinux is **greater than** 4 we also need to make sure
   that we copy the library modules into the ``/tftpboot`` directory [2]_
   [1]_. For example, for Ubuntu run::

       sudo cp /usr/lib/syslinux/modules/*/ldlinux.* /tftpboot

#. Update the bare metal node with ``boot_mode:bios`` capability in
   node's properties field. See :ref:`boot_mode_support` for details.

#. Make sure that bare metal node is configured to boot in Legacy BIOS boot mode
   and boot device is set to network/pxe.

.. [1] On **Fedora/RHEL** the ``syslinux-tftpboot`` package already installs
       the library modules and PXE image at ``/tftpboot``. If the TFTP server
       is configured to listen to a different directory you should copy the
       contents of ``/tftpboot`` to the configured directory
.. [2] http://www.syslinux.org/wiki/index.php/Library_modules


iPXE setup
----------

If you will be using iPXE to boot instead of PXE, iPXE needs to be set up
on the Bare Metal service node(s) where ``ironic-conductor`` is running.

#. Make sure these directories exist and can be written to by the user
   the ``ironic-conductor`` is running as. For example::

    sudo mkdir -p /tftpboot
    sudo mkdir -p /httpboot
    sudo chown -R ironic /tftpboot
    sudo chown -R ironic /httpboot

#. Create a map file in the tftp boot directory (``/tftpboot``)::

    echo 'r ^([^/]) /tftpboot/\1' > /tftpboot/map-file
    echo 'r ^(/tftpboot/) /tftpboot/\2' >> /tftpboot/map-file

   .. _HTTP server:

#. Set up TFTP and HTTP servers.

   These servers should be running and configured to use the local
   /tftpboot and /httpboot directories respectively, as their root
   directories. (Setting up these servers is outside the scope of this
   install guide.)

   These root directories need to be mounted locally to the
   ``ironic-conductor`` services, so that the services can access them.

   The Bare Metal service's configuration file (/etc/ironic/ironic.conf)
   should be edited accordingly to specify the TFTP and HTTP root
   directories and server addresses. For example:

   .. code-block:: ini

      [pxe]

      # Ironic compute node's tftp root path. (string value)
      tftp_root=/tftpboot

      # IP address of Ironic compute node's tftp server. (string
      # value)
      tftp_server=192.168.0.2

      [deploy]
      # Ironic compute node's http root path. (string value)
      http_root=/httpboot

      # Ironic compute node's HTTP server URL. Example:
      # http://192.1.2.3:8080 (string value)
      http_url=http://192.168.0.2:8080

   See also: :ref:`l3-external-ip`.

#. Install the iPXE package with the boot images:

   Ubuntu::

       apt-get install ipxe

   RHEL8/CentOS8/Fedora::

       dnf install ipxe-bootimgs

   .. note::
      SUSE does not provide a package containing iPXE boot images. If you are
      using SUSE or if the packaged version of the iPXE boot image doesn't
      work, you can download a prebuilt one from http://boot.ipxe.org or build
      one image from source, see http://ipxe.org/download for more information.

#. Copy the iPXE boot image (``undionly.kpxe`` for **BIOS** and
   ``ipxe.efi`` for **UEFI**) to ``/tftpboot``. The binary might
   be found at:

   Ubuntu::

       cp /usr/lib/ipxe/{undionly.kpxe,ipxe.efi,snponly.efi} /tftpboot

   Fedora/RHEL8/CentOS8::

       cp /usr/share/ipxe/{undionly.kpxe,ipxe-x86_64.efi,ipxe-snponly-x86_64.efi} /tftpboot

   .. note:: ``snponly`` variants may not be available for all distributions.

#. Enable/Configure iPXE overrides in the Bare Metal Service's configuration
   file **if required** (/etc/ironic/ironic.conf):

   .. code-block:: ini

      [pxe]

      # Neutron bootfile DHCP parameter. (string value)
      ipxe_bootfile_name=undionly.kpxe

      # Bootfile DHCP parameter for UEFI boot mode. (string value)
      uefi_ipxe_bootfile_name=ipxe.efi

      # Template file for PXE configuration. (string value)
      ipxe_config_template=$pybasedir/drivers/modules/ipxe_config.template

   .. note::
      Most UEFI systems have integrated networking which means the
      ``[pxe]uefi_ipxe_bootfile_name`` setting should be set to
      ``snponly.efi`` or ``ipxe-snponly-x86_64.efi`` if it's available for
      your distribution.

   .. note::
      Setting the iPXE parameters noted in the code block above to no value,
      in other words setting a line to something like ``ipxe_bootfile_name=``
      will result in ironic falling back to the default values of the non-iPXE
      PXE settings. This is for backwards compatability.

#. Ensure iPXE is the default PXE, if applicable.

   In earlier versions of ironic, a ``[pxe]ipxe_enabled`` setting allowing
   operators to declare the behavior of the conductor to exclusively operate
   as if only iPXE was to be used. As time moved on, iPXE functionality was
   moved to it's own ``ipxe`` boot interface.

   If you want to emulate that same hehavior, set the following in the
   configuration file (/etc/ironic/ironic.conf):

   .. code-block:: ini

      [DEFAULT]
      default_boot_interface=ipxe
      enabled_boot_interfaces=ipxe,pxe

   .. note::
      The ``[DEFAULT]enabled_boot_interfaces`` setting may be exclusively set
      to ``ipxe``, however ironic has multiple interfaces available depending
      on the hardware types available for use.

#. It is possible to configure the Bare Metal service in such a way
   that nodes will boot into the deploy image directly from Object Storage.
   Doing this avoids having to cache the images on the ironic-conductor
   host and serving them via the ironic-conductor's `HTTP server`_.
   This can be done if:

   #. the Image Service is used for image storage;
   #. the images in the Image Service are internally stored in
      Object Storage;
   #. the Object Storage supports generating temporary URLs
      for accessing objects stored in it.
      Both the OpenStack Swift and RADOS Gateway provide support for this.

      * See :doc:`/admin/radosgw` on how to configure
        the Bare Metal Service with RADOS Gateway as the Object Storage.

   Configure this by setting the ``[pxe]/ipxe_use_swift`` configuration
   option to ``True`` as follows:

   .. code-block:: ini

      [pxe]

      # Download deploy images directly from swift using temporary
      # URLs. If set to false (default), images are downloaded to
      # the ironic-conductor node and served over its local HTTP
      # server. Applicable only when 'ipxe_enabled' option is set to
      # true. (boolean value)
      ipxe_use_swift=True

   Although the `HTTP server`_ still has to be deployed and configured
   (as it will serve iPXE boot script and boot configuration files for nodes),
   such configuration will shift some load from ironic-conductor hosts
   to the Object Storage service which can be scaled horizontally.

   Note that when SSL is enabled on the Object Storage service
   you have to ensure that iPXE firmware on the nodes can indeed
   boot from generated temporary URLs that use HTTPS protocol.

#. Restart the ``ironic-conductor`` process:

   Fedora/RHEL8/CentOS8/SUSE::

     sudo systemctl restart openstack-ironic-conductor

   Ubuntu::

     sudo service ironic-conductor restart

PXE multi-architecture setup
----------------------------

It is possible to deploy servers of different architecture by one conductor.
To use this feature, architecture-specific boot and template files must
be configured using the configuration options
``[pxe]pxe_bootfile_name_by_arch`` and ``[pxe]pxe_config_template_by_arch``
respectively, in the Bare Metal service's configuration file
(/etc/ironic/ironic.conf).

These two options are dictionary values; the key is the architecture and the
value is the boot (or config template) file. A node's ``cpu_arch`` property is
used as the key to get the appropriate boot file and template file. If the
node's ``cpu_arch`` is not in the dictionary, the configuration options (in
[pxe] group) ``pxe_bootfile_name``, ``pxe_config_template``,
``uefi_pxe_bootfile_name`` and ``uefi_pxe_config_template`` will be used
instead.

In the following example, since 'x86' and 'x86_64' keys are not in the
``pxe_bootfile_name_by_arch`` or ``pxe_config_template_by_arch`` options, x86
and x86_64 nodes will be deployed by 'pxelinux.0' or 'bootx64.efi', depending
on the node's ``boot_mode`` capability ('bios' or 'uefi'). However, aarch64
nodes will be deployed by 'grubaa64.efi', and ppc64 nodes by 'bootppc64'::

    [pxe]

    # Bootfile DHCP parameter. (string value)
    pxe_bootfile_name=pxelinux.0

    # On ironic-conductor node, template file for PXE
    # configuration. (string value)
    pxe_config_template = $pybasedir/drivers/modules/pxe_config.template

    # Bootfile DHCP parameter for UEFI boot mode. (string value)
    uefi_pxe_bootfile_name=bootx64.efi

    # On ironic-conductor node, template file for PXE
    # configuration for UEFI boot loader. (string value)
    uefi_pxe_config_template=$pybasedir/drivers/modules/pxe_grub_config.template

    # Bootfile DHCP parameter per node architecture. (dict value)
    pxe_bootfile_name_by_arch=aarch64:grubaa64.efi,ppc64:bootppc64

    # On ironic-conductor node, template file for PXE
    # configuration per node architecture. For example:
    # aarch64:/opt/share/grubaa64_pxe_config.template (dict value)
    pxe_config_template_by_arch=aarch64:pxe_grubaa64_config.template,ppc64:pxe_ppc64_config.template

.. note::
   The grub implementation may vary on different architecture, you may need to
   tweak the pxe config template for a specific arch. For example, grubaa64.efi
   shipped with CentoOS7 does not support ``linuxefi`` and ``initrdefi``
   commands, you'll need to switch to use ``linux`` and ``initrd`` command
   instead.

.. note::
   A ``[pxe]ipxe_bootfile_name_by_arch`` setting is available for multi-arch
   iPXE based deployment, and defaults to the same behavior as the comperable
   ``[pxe]pxe_bootfile_by_arch`` setting for standard PXE.

PXE timeouts tuning
-------------------

Because of its reliance on UDP-based protocols (DHCP and TFTP), PXE is
particularly vulnerable to random failures during the booting stage. If the
deployment ramdisk never calls back to the bare metal conductor, the build will
be aborted, and the node will be moved to the ``deploy failed`` state, after
the deploy callback timeout. This timeout can be changed via the
:oslo.config:option:`conductor.deploy_callback_timeout` configuration option.

Starting with the Train release, the Bare Metal service can retry PXE boot if
it takes too long. The timeout is defined via
:oslo.config:option:`pxe.boot_retry_timeout` and must be smaller than the
``deploy_callback_timeout``, otherwise it will have no effect.

For example, the following configuration sets the overall timeout to 60
minutes, allowing two retries after 20 minutes:

.. code-block:: ini

    [conductor]
    deploy_callback_timeout = 3600

    [pxe]
    boot_retry_timeout = 1200

PXE artifacts
-------------

Ironic features the capability to load PXE artifacts into the conductor
startup, minimizing the need for external installation and configuration
management tooling from having to do additional work to facilitate.

While this is an advanced feature, and destination file names must match
existing bootloader configured filenames.

For example, if using iPXE and GRUB across interfaces, you may desire
a configuration similar to this example.

.. code-block:: ini

   [pxe]
   loader_file_paths = ipxe.efi:/usr/share/ipxe/ipxe-snponly-x86_64.efi,undionly.kpxe:/usr/share/ipxe/undionly.kpxe,bootx64.efi,/boot/efi/EFI/boot/grubx64.efi,bootx64.efi:/boot/efi/EFI/boot/BOOTX64.EFI

If you choose to use relative paths as part of your destination,
those paths will be created using configuration parameter
``[pxe]dir_permission`` where as actual files copied are set with
the configuration parameter ``[pxe]file_permission``. Absolute destination
paths are not supported and will result in ironic failing to start up as
it is a misconfiguration of the deployment.