summaryrefslogtreecommitdiff
path: root/ironic_python_agent
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Add support for CentOS SUM files"HEADmasterZuul2023-05-092-6/+169
|\
| * Add support for CentOS SUM filesHarald Jensås2023-05-032-6/+169
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The CentOS Stream SUM files uses format: # FILENAME: <size> bytes ALGORITHM (FILENAME) = CHECKSUM Compared to the more common format: CHECKSUM *FILE_A CHECKSUM FILE_B Use regular expressions to check for filename both in the middle with parentheses and at the end. Similarly look for valid checksums at beginning or end of line. Also look for know checsum patterns in case file only contain the checksum iteself. Change-Id: I9e49c1a6c66e51a7b884485f0bcaf7f1802bda33
* | Merge "Revert disabling MD5 checksums"Zuul2023-05-052-2/+4
|\ \
| * | Revert disabling MD5 checksumsDmitry Tantsur2023-05-042-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was a significant breaking change that was landed despite explicit disagreement by some community members (myself included). It has already resulted in an accidental Ironic CI breakage, has broken Bifrost and has a potential of breaking Metal3. In case of Metal3, MD5 support is a part of its public API. While MD5 is a potential security hazard, I don't see the need to hurry this change without giving the community time to prepare. This change reverts the new option md5_enabled to True. Change-Id: I32b291ea162e8eb22429712c15cb5b225a6daafd
* | | Merge "Add network interface speed to the inventory"Zuul2023-05-044-542/+607
|\ \ \ | |/ / |/| |
| * | Add network interface speed to the inventoryDmitry Tantsur2023-05-034-542/+607
| |/ | | | | | | | | | | | | This is another fact that Metal3's baremetal-operator is currently consuming from extra-hardware. Change-Id: I2ec9d5e9369f5508e7583a4e13c2083f5c8b28ba
* | Fix checksum validation logicJulia Kreger2023-05-022-2/+22
|/ | | | | | | | | | | The checksum validation logic, which was updated early on in the whole process of deprecating md5, didn't account for a URL *or* a longer checksum (i.e. sha256/sha512) which was decided while the overall approach was being decided. Fixes the logic, and adds additional tests. Change-Id: Ic4053776e131fc02ace295a1e69e9f9faab47f42
* Merge "Disable MD5 image checksums"Zuul2023-05-024-77/+404
|\
| * Disable MD5 image checksumsJulia Kreger2023-04-244-77/+404
| | | | | | | | | | | | | | | | | | | | | | | | MD5 image checksums have long been supersceeded by the use of a ``os_hash_algo`` and ``os_hash_value`` field as part of the properties of an image. In the process of doing this, we determined that checksum via URL usage was non-trivial and determined that an appropriate path was to allow the checksum type to be determined as needed. Change-Id: I26ba8f8c37d663096f558e83028ff463d31bd4e6
* | Merge "Deprecate LLDP in inventory in favour of a new collector"Zuul2023-04-274-28/+45
|\ \
| * | Deprecate LLDP in inventory in favour of a new collectorDmitry Tantsur2023-04-264-28/+45
| |/ | | | | | | | | | | | | | | | | | | | | | | | | Binary LLDP data is bloating inventory causing us to disable its collection by default. For other similar low-level information, such as PCI devices or DMI data, we already use inspection collectors instead. Now that the inventory format is shared with out-of-band inspection, having LLDP there makes even less sense. This change adds a new collector ``lldp`` to replace the now-deprecated inventory field. Change-Id: I56be06a7d1db28407e1128c198c12bea0809d3a3
* | Fix UTF-16 result handling for efibootmgrJulia Kreger2023-04-173-63/+82
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | The tl;dr is that UEFI NVRAM is in encoded in UTF-16, and when we run the efibootmgr command, we can get unicode characters back. Except we previously were forcing everything to be treated as UTF-8 due to the way oslo.concurrency's processutils module works. This could be observed with UTF character 0x00FF which raises up a nice exception when we try to decode it. Anyhow! while fixing handling of this, we discovered we could get basically the cruft out of the NVRAM, by getting what was most likey a truncated string out of our own test VMs. As such, we need to also permit decoding to be tollerant of failures. This could be binary data or as simple as flipped bits which get interpretted invalid characters. As such, we have introduced such data into one of our tests involving UEFI record de-duplication. Closes-Bug: 2015602 Change-Id: I006535bf124379ed65443c7b283bc99ecc95568b
* Report system firmware information in the inventoryDmitry Tantsur2023-03-314-19/+82
| | | | Change-Id: I5b6ceb9cdcf4baa97a6f0482d1030d14f3f2ecff
* [Trivial] Fix typo in efi_utilsArne Wiebalck2023-03-151-1/+1
| | | | Change-Id: I692e045e6bc8683038a2e85a6a132687d2b30f18
* Merge "update NVIDIA NIC firmware images and settings by ironic-python-agent"9.4.0Zuul2023-01-315-0/+2152
|\
| * update NVIDIA NIC firmware images and settings by ironic-python-agentwaleed mousa2023-01-115-0/+2152
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add "update_nvidia_nic_firmware_image" and "update_nvidia_nic_firmware_settings" clean steps to MellanoxDeviceHardwareManager. By adding those two steps, we can update the firmware image and firmware settings of NVIDIA NICs by ironic-python-agent using manual cleaning command The clean steps require mstflint package installed on the image. The "update_nvidia_nic_firmware_image" clean step requires to pass "images" parameter to the clean command The "images" parameter is a json blob contains a list of images, where each image contains a map of: * url: to firmware image (file://, http://) * checksum: checksum of the provided image * checksumType: md5/sha512/sha256 * componentFlavor: PSID of the nic * version: version of the FW The "update_nvidia_nic_firmware_settings" clean step requires to pass "settings" parameter to the clean command The "settings" parameter is a json blob contains a list of settings, where each settings contains a map of: * deviceID: device ID * globalConfig: global config * function0Config: function 0 config * function1Config: function 1 config Change-Id: Icfaffd7c58c3c73c3fa28cfc2a6c954d2c93c16e Story: 2010228 Task: 46016
* | Make logs collection a hardware manager callDmitry Tantsur2023-01-254-156/+156
|/ | | | | | This allows hardware managers to collect additional logs. Change-Id: If082b921d4bf71c4cc41a5a72db6995b08637374
* Fix create configuration unit testsRiccardo Pittau2022-12-151-0/+2
| | | | | | | | | | | The unit tests for create_configuration give different result if ran on a bios or uefi booted machine because they get the partition table type value based on the utils function get_node_boot_mode. Let's mock the boot_mode as we do in other tests to get an independent result. Change-Id: Ic0e7daea7ec4ce0806cd126c27166f84690c5d9e
* Merge "Fix failure of bind mount in _install_grub2"9.2.0bugfix/9.2Zuul2022-10-172-3/+46
|\
| * Fix failure of bind mount in _install_grub2Vanou Ishii2022-09-222-3/+46
| | | | | | | | | | | | | | | | | | | | | | | | When IPA runs _install_grub2, IPA tries to bind mount /dev, /proc and /run to <temporal directory path root partition mounted>/{dev,proc,run}. However that bind mount fails because there aren't such mount point path under temporal directory. To fix this failure, this patch add mkdir command before bind mount. Story: 2010292 Task: 46273 Change-Id: I434ce1bf1863ee0f11c4d09918d6d2d8dc065c02
* | prioritize lsblk as a source of device serialsRozzii2022-10-105-77/+182
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current way of prioritizing ID/DM_SERIAL_SHORT or ID/DM_SERIAL works in most cases but the udev values seem to be unreliable. Based on experience it looks like lsblk might be a better source of truth than udev in regerards to serial number information. This commit makes lsblk the default provider of block device serial number information. Story: 2010263 Task: 46161 Change-Id: I16039b46676f1a61b32ee7ca7e6d526e65829113
* | SoftwareRAID: Enable skipping RAIDSJakub Jelinek2022-09-056-59/+735
|/ | | | | | | | | | | | | | Extend the ability to skip disks to RAID devices This allows users to specify the volume name of a logical device in the skip list which is then not cleaned or created again during the create/apply configuration phase The volume name can be specified in target raid config provided the change https://review.opendev.org/c/openstack/ironic-python-agent/+/853182/ passes Story: 2010233 Change-Id: Ib9290a97519bc48e585e1bafb0b60cc14e621e0f
* Merge "Create RAIDs with volume name"Zuul2022-09-024-44/+112
|\
| * Create RAIDs with volume nameJakub Jelinek2022-09-024-44/+112
| | | | | | | | | | | | | | | | Use 'volume_name' field from 'target_raid_config' to create logical disks if it is present Do not allow two logical disks to have the same volume name Change-Id: If3e4e9f8698ec3e0cb49717f8ed2087d2ba03f2c
* | Fix software raid output poisoningJulia Kreger2022-08-243-1/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In the event a device name is set to contain a raid device path, it is possible for the Name and Events field values of mdadm's detailed output to contain text which inadvertently gets captured and mapped as component data for the "holder" devices of the RAID set. This would cause invalid values to get passed to UEFI methods which would cause a deployment to fail under these circumstances. We now ignore the Name and Events fields in mdadm output. Change-Id: If721dfe1caa5915326482969e55fbf4697538231
* | Merge "Improve function list_block_devices_check_skip_list"Zuul2022-08-171-22/+14
|\ \
| * | Improve function list_block_devices_check_skip_listJakub Jelinek2022-08-161-22/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fix minor issues suggested by dtantsur Add an example of skip list specification to the documentation A follow-up patch to I3bdad3cca8acb3e0a69ebb218216e8c8419e9d65 Change-Id: Ic94a33b7bc0572a1cc8f92b330474ec63a173e81
* | | Merge "Enable skipping disks for cleaning"9.0.0Zuul2022-08-162-9/+251
|\ \ \ | |/ / | | / | |/ |/|
| * Enable skipping disks for cleaningJakub Jelinek2022-08-112-9/+251
| | | | | | | | | | | | | | | | | | | | | | | | Introduce a field skip_block_devices in properties - this is a list of dictionaries Create a helper function list_block_devices_check_skip_list Update tests of erase_devices_express to use node when calling _list_erasable_devices Add tests covering various options of the skip list definition Use the helper function in get_os_install_device when node is cached Story: 2009914 Change-Id: I3bdad3cca8acb3e0a69ebb218216e8c8419e9d65
* | Merge "Use lsblk json output for safety_check_block_device"Zuul2022-08-032-32/+31
|\ \ | |/ |/|
| * Use lsblk json output for safety_check_block_deviceRiccardo Pittau2022-07-202-32/+31
| | | | | | | | Change-Id: Ibfc2e203287d92e66567c33dc48f59392852b88e
* | Remove unused lines of codeJakub Jelinek2022-07-201-5/+0
| | | | | | | | | | | | | | The 5 lines of code were extracted from erase_devices_metadata to _list_erasable_devices, but now are duplicated in both functions. The variable block_devices is not used in erase_devices_metadata. Change-Id: I89f56c69d90fb0eb61907d6667266fbd57d333af
* | Merge "Guard shared device/cluster filesystems"Zuul2022-07-204-6/+347
|\ \ | |/
| * Guard shared device/cluster filesystemsJulia Kreger2022-07-194-6/+347
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Certain filesystems are sometimes used in specialty computing environments where a shared storage infrastructure or fabric exists. These filesystems allow for multi-host shared concurrent read/write access to the underlying block device by *not* locking the entire device for exclusive use. Generally ranges of the disk are reserved for each interacting node to write to, and locking schemes are used to prevent collissions. These filesystems are common for use cases where high availability is required or ability for individual computers to collaborate on a given workload is critical, such as a group of hypervisors supporting virtual machines because it can allow for nearly seamless transfer of workload from one machine to another. Similar technologies are also used for cluster quorum and cluster durable state sharing, however that is not specifically considered in scope. Where things get difficult is becuase the entire device is not exclusively locked with the storage fabrics, and in some cases locking is handled by a Distributed Lock Manager on the network, or via special sector interactions amongst the cluster members which understand and support the filesystem. As a reult of this IO/Interaction model, an Ironic-Python-Agent performing cleaning can effectively destroy the cluster just by attempting to clean storage which it percieves as attached locally. This is not IPA's fault, often this case occurs when a Storage Administrator forgot to update LUN masking or volume settings on a SAN as it relates to an individual host in the overall computing environment. The net result of one node cleaning the shared volume may include restoration from snapshot, backup storage, or may ultimately cause permenant data loss, depending on the environment and the usage of that environment. Included in this patch: - IBM GPFS - Can be used on a shared block device... apparently according to IBM's documentation. The standard use of GPFS is more Ceph like in design... however GPFS is also a specially licensed commercial offering, so it is a red flag if this is encountered, and should be investigated by the environment's systems operator. - Red Hat GFS2 - Is used with shared common block devices in clusters. - VMware VMFS - Is used with shared SAN block devices, as well as local block devices. With shared block devices, ranges of the disk are locked instead of the whole disk, and the ranges are mapped to virtual machine disk interfaces. It is unknown, due to lack of information, if this will detect and prevent erasure of VMFS logical extent volumes. Co-Authored-by: Jay Faulkner <jay@jvf.cc> Change-Id: Ic8cade008577516e696893fdbdabf70999c06a5b Story: 2009978 Task: 44985
* | Drop support for instance netbootDmitry Tantsur2022-07-074-46/+16
|/ | | | Change-Id: I2b4c543537dac8904028fdcdb590c1c214238e10
* Merge "Fix passing kwargs in clean steps"Zuul2022-07-042-1/+31
|\
| * Fix passing kwargs in clean stepswaleedm2022-07-012-1/+31
| | | | | | | | | | | | | | | | Pass kwargs to dispatch_to_managers method in execute_clean_step Change-Id: Ida4ed4646659b2ee3f8f92b0a4d73c0266dd5a99 Story: 2010123 Task: 45705
* | Merge "Gather details about bond interfaces if present"Zuul2022-07-022-11/+61
|\ \
| * | Gather details about bond interfaces if presentDerek Higgins2022-06-212-11/+61
| |/ | | | | | | | | | | | | | | | | If present gather information about bonded interfaces. Story: #2010093 Task: #45637 Change-Id: I394187640b4788ebec21c3391d33ed728fb72ffa
* | Merge "Remove oslo.serialization dependency"Zuul2022-07-026-26/+33
|\ \
| * | Remove oslo.serialization dependencyRiccardo Pittau2022-06-176-26/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | Use pure json instead of jsonutils. Borrow encode function from oslo.serialization to be used in the utils module. Change-Id: Ied9a2259a4329a86b4f0853bd1fb187563c0a036
* | | Merge "Collect udev properties in the ramdisk logs"Zuul2022-07-022-4/+84
|\ \ \
| * | | Collect udev properties in the ramdisk logsDmitry Tantsur2022-06-172-4/+84
| | |/ | |/| | | | | | | Change-Id: Ifcf3dfff00b604dec1e2f430369ab8053f50f137
* | | Merge "Use json for lsblk output"Zuul2022-06-304-201/+226
|\ \ \ | | |/ | |/|
| * | Use json for lsblk outputRiccardo Pittau2022-06-144-201/+226
| |/ | | | | | | | | | | | | | | | | The lsblk output is available in json format since version 2.27 of util-linux [1] https: //mirrors.edge.kernel.org/pub/linux/utils/util-linux/v2.27/v2.27-ReleaseNotes Change-Id: I0c5812736b7a320cc4ecc333f80db70eb78cc76d
* | Merge "Warn when smartctl not found"Zuul2022-06-271-1/+2
|\ \
| * | Warn when smartctl not foundMark Goddard2022-06-241-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, if smartctl is not found by IPA, it will silently skip ATA secure erase and proceed to shred (if enabled). This is supposedly for backwards compatibility, but is quite hard to diagnose. This change adds a warning message to make it more obvious what is happening. TrivialFix Change-Id: I03a381e99de79f201ec7e9a388777c3d48457e93
* | | Merge "Remove importlib-metadata from requirements"Zuul2022-06-241-6/+1
|\ \ \
| * | | Remove importlib-metadata from requirementsRiccardo Pittau2022-06-211-6/+1
| | |/ | |/| | | | | | | | | | | | | | | | | | | We don't need it anymore as we don't support python < 3.8 Also it was removed from global requirements so it breaks the requirements check. Change-Id: Ia12cbef3515f823fdd627a36020cf7801bf6d734
* | | Fix discovering WWN/serial for devicemapper devicesDmitry Tantsur2022-06-142-16/+27
|/ / | | | | | | | | | | | | UDev prefix is DM_ not ID_ for them. On top of that, they don't have short serials (or at least don't always have). Change-Id: I5b6075fbff72201a2fd620f789978acceafc417b
View Lorry Controller Status