1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
|
==============
Manage volumes
==============
Depending on the setup of your cloud provider, they may give you an endpoint to
use to manage volumes. You can use the ``openstack`` CLI to manage volumes.
For the purposes of the compute service, attaching, detaching and
:doc:`creating a server from a volume </user/launch-instance-from-volume>` are
of primary interest.
Refer to the :python-openstackclient-doc:`CLI documentation
<cli/command-objects/volume.html>` for more information.
Volume multi-attach
-------------------
Nova `added support for multiattach volumes`_ in the 17.0.0 Queens release.
This document covers the nova-specific aspects of this feature. Refer
to the :cinder-doc:`block storage admin guide
<admin/blockstorage-volume-multiattach.html>` for more details about creating
multiattach-capable volumes.
:term:`Boot from volume <Boot From Volume>` and attaching a volume to a server
that is not SHELVED_OFFLOADED is supported. Ultimately the ability to perform
these actions depends on the compute host and hypervisor driver that
is being used.
There is also a `recorded overview and demo`_ for volume multi-attach.
Requirements
~~~~~~~~~~~~
* The minimum required compute API microversion for attaching a
multiattach-capable volume to more than one server is :ref:`2.60
<api-microversion-queens>`.
* Cinder 12.0.0 (Queens) or newer is required.
* The ``nova-compute`` service must be running at least Queens release level
code (17.0.0) and the hypervisor driver must support attaching block storage
devices to more than one guest. Refer to :doc:`/user/support-matrix` for
details on which compute drivers support volume multiattach.
* When using the libvirt compute driver, the following native package versions
determine multiattach support:
* libvirt must be greater than or equal to 3.10, or
* qemu must be less than 2.10
* Swapping an *in-use* multiattach volume is not supported (this is actually
controlled via the block storage volume retype API).
Known issues
~~~~~~~~~~~~
* Creating multiple servers in a single request with a multiattach-capable
volume as the root disk is not yet supported: https://bugs.launchpad.net/nova/+bug/1747985
* Subsequent attachments to the same volume are all attached in *read/write*
mode by default in the block storage service. A future change either in nova
or cinder may address this so that subsequent attachments are made in
*read-only* mode, or such that the mode can be specified by the user when
attaching the volume to the server.
Testing
~~~~~~~
Continuous integration testing of the volume multiattach feature is done
via the ``tempest-full`` and ``tempest-slow`` jobs, which, along with the
tests themselves, are defined in the `tempest repository`_.
.. _added support for multiattach volumes: https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/multi-attach-volume.html
.. _recorded overview and demo: https://www.youtube.com/watch?v=hZg6wqxdEHk
.. _tempest repository: http://codesearch.openstack.org/?q=CONF.compute_feature_enabled.volume_multiattach&i=nope&files=&repos=tempest
Managing volume attachments
---------------------------
During the lifecycle of an instance admins may need to check various aspects of
how a given volume is mapped both to an instance and the underlying compute
hosting the instance. This could even include refreshing different elements of
the attachment to ensure the latest configuration changes within the
environment have been applied.
Checking an existing attachment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Existing volume attachments can be checked using the following
:python-openstackclient-doc:`OpenStack Client commands <cli/command-objects>`:
List all volume attachments for a given instance:
.. code-block:: shell
$ openstack server volume list 216f9481-4c9d-4530-b865-51cedfa4b8e7
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| ID | Device | Server ID | Volume ID |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| 8b9b3491-f083-4485-8374-258372f3db35 | /dev/vdb | 216f9481-4c9d-4530-b865-51cedfa4b8e7 | 8b9b3491-f083-4485-8374-258372f3db35 |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
List all volume attachments for a given instance with the Cinder volume
attachment and Block Device Mapping UUIDs also listed with microversion >=2.89:
.. code-block:: shell
$ openstack --os-compute-api-version 2.89 server volume list 216f9481-4c9d-4530-b865-51cedfa4b8e7
+----------+--------------------------------------+--------------------------------------+------+------------------------+--------------------------------------+--------------------------------------+
| Device | Server ID | Volume ID | Tag | Delete On Termination? | Attachment ID | BlockDeviceMapping UUID |
+----------+--------------------------------------+--------------------------------------+------+------------------------+--------------------------------------+--------------------------------------+
| /dev/vdb | 216f9481-4c9d-4530-b865-51cedfa4b8e7 | 8b9b3491-f083-4485-8374-258372f3db35 | None | False | d338fb38-cfd5-461f-8753-145dcbdb6c78 | 4e957e6d-52f2-44da-8cf8-3f1ab755e26d |
+----------+--------------------------------------+--------------------------------------+------+------------------------+--------------------------------------+--------------------------------------+
List all Cinder volume attachments for a given volume from microversion >=
3.27:
.. code-block:: shell
$ openstack --os-volume-api-version 3.27 volume attachment list --volume-id 8b9b3491-f083-4485-8374-258372f3db35
+--------------------------------------+--------------------------------------+--------------------------------------+----------+
| ID | Volume ID | Server ID | Status |
+--------------------------------------+--------------------------------------+--------------------------------------+----------+
| d338fb38-cfd5-461f-8753-145dcbdb6c78 | 8b9b3491-f083-4485-8374-258372f3db35 | 216f9481-4c9d-4530-b865-51cedfa4b8e7 | attached |
+--------------------------------------+--------------------------------------+--------------------------------------+----------+
Show the details of a Cinder volume attachment from microversion >= 3.27:
.. code-block:: shell
$ openstack --os-volume-api-version 3.27 volume attachment show d338fb38-cfd5-461f-8753-145dcbdb6c78
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID | d338fb38-cfd5-461f-8753-145dcbdb6c78 |
| Volume ID | 8b9b3491-f083-4485-8374-258372f3db35 |
| Instance ID | 216f9481-4c9d-4530-b865-51cedfa4b8e7 |
| Status | attached |
| Attach Mode | rw |
| Attached At | 2021-09-14T13:03:38.000000 |
| Detached At | |
| Properties | access_mode='rw', attachment_id='d338fb38-cfd5-461f-8753-145dcbdb6c78', auth_method='CHAP', auth_password='4XyNNFV2TLPhKXoP', auth_username='jsBMQhWZJXupA4eWHLQG', cacheable='False', driver_volume_type='iscsi', encrypted='False', qos_specs=, target_discovered='False', target_iqn='iqn.2010-10.org.openstack:volume-8b9b3491-f083-4485-8374-258372f3db35', target_lun='0', target_portal='192.168.122.99:3260', volume_id='8b9b3491-f083-4485-8374-258372f3db35' |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Refresh a volume attachment with nova-manage
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. versionadded:: 24.0.0 (Xena)
Admins may also refresh an existing volume attachment using the following
:program:`nova-manage` commands.
.. note::
Users can also refresh volume attachments by shelving and later unshelving
their instances. The following is an alternative to that workflow and
useful for admins when having to mass refresh attachments across an
environment.
.. note::
Future work will look into introducing an os-refresh admin API that will
include orchestrating the shutdown of an instance and refreshing volume
attachments among other things.
To begin the admin can use the `volume_attachment show` subcommand to dump
existing details of the attachment directly from the Nova database. This
includes the stashed `connection_info` not shared by the API.
.. code-block:: shell
$ nova-manage volume_attachment show 216f9481-4c9d-4530-b865-51cedfa4b8e7 8b9b3491-f083-4485-8374-258372f3db35 --json | jq .attachment_id
"d338fb38-cfd5-461f-8753-145dcbdb6c78"
If the stored `connection_info` or `attachment_id` are incorrect then the
admin may want to refresh the attachment to the compute host entirely by
recreating the Cinder volume attachment record(s) and pulling down fresh
`connection_info`. To do this we first need to ensure the instance is stopped:
.. code-block:: shell
$ openstack server stop 216f9481-4c9d-4530-b865-51cedfa4b8e7
Once stopped the host connector of the compute hosting the instance has to be
fetched using the `volume_attachment get_connector` subcommand:
.. code-block:: shell
root@compute $ nova-manage volume_attachment get_connector --json > connector.json
.. note::
Future work will remove this requirement and incorporate the gathering of
the host connector into the main refresh command. Unfortunately until then
it must remain a separate manual step.
We can then provide this connector to the `volume_attachment refresh`
subcommand. This command will connect to the compute, disconnect any host
volume connections, delete the existing Cinder volume attachment,
recreate the volume attachment and finally update Nova's database.
.. code-block:: shell
$ nova-manage volume_attachment refresh 216f9481-4c9d-4530-b865-51cedfa4b8e7 8b9b3491-f083-4485-8374-258372f3db35 connector.json
The Cinder volume attachment and connection_info stored in the Nova database
should now be updated:
.. code-block:: shell
$ nova-manage volume_attachment show 216f9481-4c9d-4530-b865-51cedfa4b8e7 8b9b3491-f083-4485-8374-258372f3db35 --json | jq .attachment_id
"9ce46f49-5cfc-4c6c-b2f0-0287540d3246"
The instance can then be restarted and the event list checked
.. code-block:: shell
$ openstack server start $instance
|
