summaryrefslogtreecommitdiff
path: root/nova/api/metadata
Commit message (Collapse)AuthorAgeFilesLines
* Log failed metadata retrievalsMohammed Naser2021-10-281-0/+1
| | | | | | | | | | | | | | | | | | | | | In our cloud environments, we're observing a weird issue where this function is returning `None` for systems that are already running and exist. Due to the fact that it returns `None` right away, we don't actually get a reason as to why the request failed and it is returning a 404. This should not clobber up the logs much more, due to the fact that if you are sending a request to the metadata service, you're supposed to already exist, so not being able to find the metadata for an instance seems like a broken case. This patch should help operators at least uncover why exactly their instances are failing to get the metadata to further troubleshoot. Change-Id: Iadca69fee24ec89408f9fca68d1da46c918577bc
* Remove (almost) all references to 'instance_type'Stephen Finucane2021-03-291-2/+2
| | | | | | | | | | | This continues on from I81fec10535034f3a81d46713a6eda813f90561cf and removes all other references to 'instance_type' where it's possible to do so. The only things left are DB columns, o.vo fields, some unversioned objects, and RPC API methods. If we want to remove these, we can but it's a lot more work. Change-Id: I264d6df1809d7283415e69a66a9153829b8df537 Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* Remove six.text_type (1/2)Takashi Natsume2020-12-132-8/+4
| | | | | | | | | Replace six.text_type with str. A subsequent patch will replace other six.text_type. Change-Id: I23bb9e539d08f5c6202909054c2dd49b6c7a7a0e Implements: blueprint six-removal Signed-off-by: Takashi Natsume <takanattie@gmail.com>
* Remove six.binary_type/integer_types/string_typesTakashi Natsume2020-12-131-2/+2
| | | | | | | | | | | | | | Replace the following items with Python 3 style code. - six.binary_type - six.integer_types - six.string_types Subsequent patches will replace other six usages. Change-Id: Ide65686cf02463045f5c32771ca949802b19636f Implements: blueprint six-removal Signed-off-by: Takashi Natsume <takanattie@gmail.com>
* Remove six.movesTakashi Natsume2020-11-071-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | Replace the following items with Python 3 style code. - six.moves.configparser - six.moves.StringIO - six.moves.cStringIO - six.moves.urllib - six.moves.builtins - six.moves.range - six.moves.xmlrpc_client - six.moves.http_client - six.moves.http_cookies - six.moves.queue - six.moves.zip - six.moves.reload_module - six.StringIO - six.BytesIO Subsequent patches will replace other six usages. Change-Id: Ib2c406327fef2fb4868d8050fc476a7d17706e23 Implements: blueprint six-removal Signed-off-by: Takashi Natsume <takanattie@gmail.com>
* Clean up the DynamicVendorData constructorBalazs Gibizer2020-10-122-13/+4
| | | | | | | | | | | | | | There are a bunch of unused parameters in DynamicVendorData constructor and a comment that they cannot be removed due to JsonFileVendorData. But JsonFileVendorData does not depends on those paramters and both the base class and JsonFileVendorData uses *args **kwargs. So it is safe to remove the unused params. The context field of DynamicVendorData is also removed as it is unused. This makes the request_context parameter of the InstanceMeta constructor also unused so that is removed. Change-Id: Ie27fd6a5513e53903b9acd5d63038b3b484acbde
* Use cell targeted context to query BDMs for metadataBalazs Gibizer2020-10-121-5/+7
| | | | | | | | | | | | | | | | The metadata service supports a multicell deployment in a configuration where the nova-api service implements the metadata API. In this case the metadata query needs to be cell targeted. This was partly implemented already. The instance itself is queried from the cell DB properly. However the BDM data used the non targeted context resulting in an empty BDM returned by the metadata service. Functional reproduction test is not added as I did not find a way to have a cell setup in the functional test that reproduce the problem. I reproduced the bug and tested the fix in a devstack. Change-Id: I48f57082edaef3ec4722bd31ce29a90b94d32523 Closes-Bug: #1881944
* Remove six.reraiseTakashi Natsume2020-08-151-2/+1
| | | | | | | | | Replace six.reraise with Python 3 style code. Subsequent patches will replace other six usages. Change-Id: Ib129cb399d1521ad6d18fcf0b8ac9fd793888c81 Implements: blueprint six-removal Signed-off-by: Takashi Natsume <takanattie@gmail.com>
* Remove six.PY2 and six.PY3Takashi Natsume2020-08-151-2/+1
| | | | | | | | | Remove six.PY2 and six.PY3. Subsequent patches will replace other six usages. Change-Id: Iccce0ab50eee515e533ab36c8e7adc10cb3f7019 Implements: blueprint six-removal Signed-off-by: Takashi Natsume <takanattie@gmail.com>
* metadata: export the vCPU IDs that are pinning on the host CPUsWang Huaqiang2020-07-211-1/+17
| | | | | | | | | | | | Add the 'dedicated_cpus' section in metadata API service to tell the instance CPUs that are set as 'dedicated' CPUs if instance has any dedicated CPUs. If instance is using a 'shared' CPU allocation policy, it reports 'None' for no dedicated CPU. Part of blueprint use-pcpu-and-vcpu-in-one-instance Change-Id: I3d19195ddefb856c10fa6756dd98850119a4dfcb Signed-off-by: Wang Huaqiang <huaqiang.wang@intel.com>
* Fix H702 pep8 error with latest hackingGhanshyam Mann2020-02-271-2/+2
| | | | | | | | | | | | | | nova test job on hacking master is failing with 2020-02-24 01:14:53.698965 | ubuntu-bionic | ./nova/api/metadata/handler.py:251:55: H702: Formatting operation should be outside of localization method call - https://review.opendev.org/#/c/705514/ This is H702 error for localization formatting. Change-Id: I9eaa90c273327a3ca0ca1722a45017e59e9f0f0c
* Support large network queries towards neutronKobi Samoray2020-02-161-5/+14
| | | | | | | | | | | While querying metadata from an LB source, the subnet query may result with a large number of networks. That might exceed the maximum request length on the following port query towards Neutron. The patch addresses such issue. Closes-Bug: #1861087 Change-Id: I9d72c80574d08d8409ed0dcc0476f52a0d173a1e
* Avoid fetching metadata when no subnets foundKobi Samoray2020-01-281-2/+18
| | | | | | | | | | | Metadata service uses the provider id to identify the requesting instance. However, when provider query doesn't find any networks, the request should fail. The same goes to the case where multiple ports are found. Closes-Bug: #1841933 Change-Id: I8ce3703ca86a3a0769edd42a790d82796d1071d7
* nova-net: Make the security group API a moduleStephen Finucane2019-11-291-3/+2
| | | | | | | | | | | | | We're wrestling with multiple imports for this thing and have introduced a cache to avoid having to load the thing repeatedly. However, Python already has a way to ensure this doesn't happen: the use of a module. Given that we don't have any state, we can straight up drop the class and just call functions directly. Along the way, we drop the 'ensure_default' function, which is a no-op for neutron and switch all the mocks over, where necessary. Change-Id: Ia8dbe8ba61ec6d1b8498918a53a103a6eff4d488 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* nova-net: Remove layer of indirection in 'nova.network'Stephen Finucane2020-01-152-3/+3
| | | | | | | | | | | | | | | | At some point in the past, there was only nova-network and its code could be found in 'nova.network'. Neutron was added and eventually found itself (mostly!) in the 'nova.network.neutronv2' submodule. With nova-network now gone, we can remove one layer of indirection and move the code from 'nova.network.neutronv2' back up to 'nova.network', mirroring what we did with the old nova-volume code way back in 2012 [1]. To ensure people don't get nova-network and 'nova.network' confused, 'neutron' is retained in filenames. [1] https://review.opendev.org/#/c/14731/ Change-Id: I329f0fd589a4b2e0426485f09f6782f94275cc07 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* ec2: Move ec2utils functions to their callersStephen Finucane2019-09-271-3/+1
| | | | | | | | | All of these functions only have a single caller now, so move them to their calling site. Two functions are renamed to better fit their purpose but there is otherwise no change in functionality. Change-Id: I48237f306fb901dec9ec805704c891fd3cbb12e3 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* hacking: Resolve W503 (line break occurred before a binary operator)Stephen Finucane2019-06-241-2/+2
| | | | | Change-Id: I6381365ff882cf23808e8dabfce41143c5e35192 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* docs: Rework all things metadata'yStephen Finucane2019-06-201-2/+2
| | | | | | | | | | | | | | | | | | | | | Turns out we've a *lot* of disparate metadata systems. Attempt to both link them somewhat through extensive cross-referencing and extract out deployment-specific stuff from user-facing docs. Lots of changes here, but in summary: - Split out admin-focused content from the metadata API, config drive, user data and vendordata docs. - Merge the config drive, metadata service, vendordata and user-data user docs, which are mostly talking about the same thing and are fairly barren without the deployment components - Make use of various oslo.config and Sphinx roles Side note: I miss when we have tech writers to do this stuff for us :( Change-Id: I4fb2b628bd93358a752e2397ae353221758e2984 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* Ensure that metadata proxy raises correct exceptionGary Kotton2019-05-271-3/+2
| | | | | | | | | When nova has a configuration using metadata_proxy_shared_secret and neutron NSXv does not have the metadata_shared_secret we need to make sure that a valid exception is raised. Change-Id: I5d94a6f4d4f78ad821567164f6810d19ac5e3e67 Closes-bug: #1830535
* Improve metadata performanceGary Kotton2019-05-031-4/+13
| | | | | | | | | | | | | Save many requests to the neutron service if the provider has recently been queried. Commit 622a845b757aa35932f703963beffc764d8ae244 has the details about the provider support. The caching helps improve this support as the neutron service will not be queried many times. Closes-Bug: #1810642 Change-Id: I19054fae14f95dc3ea73efde45df7e1a59086b44
* Remove 'get_keypair_at_top'Stephen Finucane2019-04-291-19/+6
| | | | | | | | | | This was being called within the 'InstanceMetadata' object. Remove it in its entirety, given that nothing should be calling it now. Part of blueprint remove-cells-v1 Change-Id: I92bd4c8f465b81a941885f9db4f708a34731de8b Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* conf: Undeprecate and move the 'dhcp_domain' optionStephen Finucane2019-04-151-2/+4
| | | | | | | | | | | | The metadata service makes use of the deprecated '[DEFAULT] dhcp_domain' option when providing a hostname to the instance. This is used by cloud-init to configure the hostname in the instance. This use was not captured when the option was initially deprecated. This option is now undeprecated and moved to the '[api]' group to ensure it won't be removed alongside the other nova-network options. Change-Id: I3940ebd1888d8019716e7d4eb6d4a413a37b9b78 Closes-Bug: #1698010
* Handle unsetting '[DEFAULT] dhcp_domain'Stephen Finucane2019-04-151-3/+4
| | | | | | | | Fix a long-standing issue whereby setting 'dhcp_domain' to 'None' would result in a hostname of '${hostname}None' instead of '${hostname}'. Change-Id: Ic9aa74f5344ba469b61a87de1ebd27e6f49c3318 Closes-Bug: #1824813
* trivial: Remove dead codeStephen Finucane2019-04-031-8/+0
| | | | | | | Remove two always False blocks which, at this point, will never be implemented, along with an unnecessary return after raise. Change-Id: I7034f38ef211e6432e656c8bbd54c647ba8856bf
* Merge "Allow run metadata api per cell"Zuul2019-01-152-6/+21
|\
| * Allow run metadata api per cellKevin_Zheng2019-01-142-6/+21
| | | | | | | | | | | | | | | | | | | | | | Adds configuration option ``[api]/local_metadata_per_cell`` to allow user run Nova metadata API service per cell. Doing this can avoid query API DB for instance information each time an instance query for its metadata. Implements blueprint run-meta-api-per-cell Change-Id: I2e6ebb551e782e8aa0ac90169f4d4b8895311b3c
* | Properly log request headers in metadata APIMatt Riedemann2018-12-171-1/+7
|/ | | | | | | | | | | | | | | | | | | Currently we log the request headers in the metadata API in a useless form: DEBUG nova.api.metadata.handler [None \ req-4b5eab00-e132-4551-b7f8-c80a238727a2 None None] \ Metadata request headers: <webob.headers.EnvironHeaders object \ at 0x7f2d6fdb5e90> {{(pid=9311) __call__ \ /opt/stack/nova/nova/api/metadata/handler.py:99}} This change builds a dict from the headers and then also masks any sensitive information before logging it, but only does this if debug logging is enabled. Closes-Bug: #1808879 Change-Id: I609d96293f7e77f59df3f33240f5fc4bb72470d0
* metadata: add vf_trusted field to device metadataSahid Orentino Ferdjaoui2018-05-311-11/+15
| | | | | | | | | | | | | The trusted vf attribute will be exposed to the instance through the metadata API and on the config drive. Note the logic when dealing with NetworkInterfaceMetadata devices was refactored slightly in order to handle the existing cases where these types of devices are skipped. Implements blueprint sriov-trusted-vfs Change-Id: Icbac4f11b2383b3d8295ec3362db0fc60b9c35a9 Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>
* make metadata doc up to datejichen2018-04-101-0/+2
| | | | | | | | | | | | | | | | currently we have following output: $ curl http://169.254.169.254/openstack 2012-08-10 2013-04-04 2013-10-17 2015-10-15 2016-06-30 2016-10-06 2017-02-22 latest Change-Id: I6b4aed63d5c981abc9374baf929f05b26760e645
* Refactor WSGI apps and utils to limit importsChris Dent2018-03-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The file nova/api/openstack/__init__.py had imported a lot of modules, notably nova.utils. This means that any code which runs within that package, notably the placement service, imports all those modules, even if it is not going to use them. This results in scripts/binaries that are heavier than they need to be and in some cases including modules, like eventlet, that it would feel safe to not have in the stack. Unfortunately we cannot sinply rename nova/api/openstack/__init__.py to another name because it contains FaultWrapper and FaultWrapper is referred to, by package path, from the paste.ini file and that file is out there in config land, and something we prefer not to change. Therefore alternate methods of cleaning up were explored and this has led to some useful changes: Fault wrapper is the only consumer of walk_class_hierarchy so there is no reason for it it to be in nova.utils. nova.wsgi contains a mismash of WSGI middleware and applications, which need only a small number of imports, and Server classes which are more complex and not required by the WSGI wares. Therefore nova.wsgi was split into nova.wsgi and nova.api.wsgi. The name choices may not be ideal, but they were chosen to limit the cascades of changes that are needed across code and tests. Where utils.utf8 was used it has been replaced with the similar (but not exactly equivalient) method from oslo_utils.encodeutils. Change-Id: I297f30aa6eb01fe3b53fd8c9b7853949be31156d Partial-Bug: #1743120
* Merge "conf: Remove 'vendordata_driver' opt"Zuul2018-01-191-26/+1
|\
| * conf: Remove 'vendordata_driver' optStephen Finucane2017-11-021-26/+1
| | | | | | | | | | | | This has been deprecated for a long time. It's time to go. Change-Id: Ia3d88dd795b7049938265acc44465261df3fec36
* | Merge "Add debug information to metadata requests"Zuul2017-12-051-1/+3
|\ \
| * | Add debug information to metadata requestsGary Kotton2017-10-201-1/+3
| |/ | | | | | | | | | | | | | | A some debug information to help debug metadata issues. TrivialFix Change-Id: I8717848b20a7f64a28974e83ff76ea0438e974a6
* | Add error message on metadata APIKen'ichi Ohmichi2017-11-141-1/+2
| | | | | | | | | | | | | | The changed BadRequest message didn't explain why that is wrong to API consumers. This patch adds it. Change-Id: Idbd183f13045c85485dfa428974511feb6e407f7
* | Handle InstanceNotFound when setting password via metadataMatt Riedemann2017-11-061-1/+5
|/ | | | | | | | | | | | When setting an instance password via the metadata service, if the instance is not found it results in a 500 response to the caller. This change handles the InstanceNotFound error and returns it as a 400. Note it's a 400 since the instance uuid is part of the POST request body, not on the URL path so it's not a 404 response. Change-Id: I4aa99b563e1a5a87aa3e3dfb28800f107676df92 Partial-Bug: #1696848
* Merge "Handle keypair not found from metadata server using cells"Jenkins2017-09-121-3/+8
|\
| * Handle keypair not found from metadata server using cellsLuis Pigueiras2017-07-031-3/+8
| | | | | | | | | | | | | | | | | | | | Fixes https://bugs.launchpad.net/nova/+bug/1592167 for the cells case. The fix done in https://bugs.launchpad.net/nova/+bug/1592167 only solves the problem when cells are not used at all Closes-bug: #1592167 Change-Id: Id663b426261150a1cce310cb4a61d9572f78c016
* | Pre-load instance.device_metadata in InstanceMetadataMatt Riedemann2017-07-061-0/+1
|/ | | | | | | | | | | | | | | | | | | Change Ie7d97ce5c62c8fb9da5822590a64210521f8ae7a orphans the Instance object so that we can pickle it. However, this means we can't lazy-load any attributes that we need when building up metadata, like when building the config drive during server create. We need to pre-load the device_metadata field so it's available when building the config drive. This isn't a problem for the libvirt driver since it loads device_metadata before building the config drive, but the hyperv driver doesn't do that, so the change above breaks hyperv when there are device tags without this fix. Change-Id: I08b905d2734ff9d484b373369f36d48c4d056fd8 Closes-Bug: #1702150
* Sanitize instance in InstanceMetadata to avoid un-pickleable contextDan Smith2017-06-291-0/+9
| | | | | | | | | | | | | This is a more strategic fix for the issue of us trying to pickle an instance with a context that has complex data structures inside (i.e. SQLAlchemy connections) into the oslo cache. The right solution is for us to stop storing random python objects (InstanceMetadata) in the cache with pickle. However, that's a larger change and more complex for deployers to roll over. This attempts to sanitize the instance before we pickle it to get things working again. Change-Id: Ie7d97ce5c62c8fb9da5822590a64210521f8ae7a Closes-Bug: #1694666
* XenAPI: device taggingJohn Hua2017-06-141-0/+2
| | | | | | | | XenAPI implementation of device tagging Partially-Implements: blueprint virt-device-role-tagging-xenapi Change-Id: I565617e05acf33e6254ea091b88d975270ffde05 Depends-On: I9fe520bc9b68d0bc7f879617f2cd27dd1029e4de
* Fix lookup of instance mapping in metadata set-passwordDan Smith2017-06-081-1/+3
| | | | | | | | The set-password function in metadata may try to lookup the instance by uuid without proper cell targeting. This fixes that. Change-Id: I8f27e4d89f0ec69b79e03a114570bedc37563cbe Closes-Bug: #1696843
* Move to proper target_cell calling conventionDan Smith2017-05-231-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | When target_cell was originally written, the intent was to yield the context that should be used. It currently mutates the input context, which means you don't have to actually use the yielded one, because we have a lot of stuff that would break otherwise. This fixes all the current uses of it to be proper, and adjusts tests accordingly. This is separate from changing target_cell's behavior to not mutate the input context specifically to isolate the mechanical changes from ones that actually need different behavior. In addition to code that was already using target_cell() but not depending on the yielded context, the _create_block_device_mapping() method in conductor/manager.py was still depending on the shared context switching to target the BDM objects on create. Since these were prepared with the context prior to having determined where the instanace was going to end up, we need to explicitly target the object context on create (like the rest of the boot workflow does for other objects in schedule_and_build_instances()). Related to blueprint cells-aware-api Change-Id: I35206e665f2c81531a2269dd66f8c5c0df834245
* Add pbr-installed wsgi application for metadata apiChris Dent2017-05-021-0/+20
| | | | | | | | Modify the wsgi application for the compute api so that it can be used by different services and use it for the metadata service, resulting in a wsgi script named nova-metadata-wsgi. Change-Id: Icb35fe2b94ab02c0ba8ba8129ae18aae0f794756
* Get instance availability_zone without hitting the api dbDan Smith2017-03-231-3/+1
| | | | | | | | | | | This would be an upcall from the cell to the api db, which we don't want. We store the availability_zone on the instance now, and even use it for pre-scheduled instances, so just use that instead of the extra lookup. Related to blueprint cells-aware-api Change-Id: I73c3b10e52ab4cfda9dacc0c0ba92d1fcb60bcc9
* remove log translation from nova.api.metadataSean Dague2017-03-214-44/+39
| | | | | | | This removes log translation markers from nova.api.metadata, now that these are no longer being used. Change-Id: Ie36fce60822cbff9abe19c7a72be5280330fc370
* Merge "Revert "Add some metadata logging to root cause ssh failure""Jenkins2017-03-072-7/+0
|\
| * Revert "Add some metadata logging to root cause ssh failure"Kevin Benton2017-03-052-7/+0
| | | | | | | | | | | | | | | | | | | | This reverts commit 99989cb6526fb05bd47941dcaa1e6d95dfedbb20. The bug this was added for (bug 1668958) has been resolved so logging metadata contents is no longer necessary. Change-Id: Ifd05aaacafaafc3ff01cd6c7323f57c0028e38ff Related-Bug: #1668958
* | Use flake8-import-orderJohn L. Villalovos2017-03-061-1/+1
| | | | | | | | | | | | | | Use the flake8 plugin flake8-import-order to check import ordering. It can do it automatically and don't need reviewers to check it. Change-Id: Ia3d81bbbb44b40804b3268c0e648276a36cb4805
* | Merge "Only create vendordata_dynamic ksa session if needed"Jenkins2017-03-031-1/+4
|\ \
View Lorry Controller Status