| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
| |
Signed-off-by: Jacob Salmela <jacob.salmela@hpe.com>
|
|
|
|
|
|
|
|
|
| |
this FreeBSD specific resizer resizes the root partition and grows the
Filesystem all in one.
All we have to do is call ``service growfs onestart``
Document behaviour: especially that growfs will insert a swap partition
if none is present, unless instructed otherwise.
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
| |
Add additional test in test_azure.py to vet the expected behavior of
suppressing error messages from mount_cb.
This is addressing PR #2134
|
|
|
|
|
| |
This fixes KeyError on specific network configuration when running
cloud-init on "network" stage. The same problem was mentioned in
#746 and #1041.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix cloud-init schema --system being unable to find merged
userdata stored at /var/lib/cloud/instance/cloud_config.txt.
Init.paths.get_ipath only has visibility to merged cloud config in
/var/lib/cloud/<instance_id>/cloud-config.txt after fetching the
existing cached datasource which provides instance-id from metadata
in order to determine the unique instance-id which represents the
path to the cloud-config.txt.
To support reuse of read_cfg_paths helper function, add an optional
parameter fetch_existing_datasource which indicates whether reading
the existing datasource is necessary for this helper function.
cloud-init schema --system calls read_cfg_paths providing
fetch_existing_datasource="trust" prior to calls to
paths.get_ipath().
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Always report failure to host, but report failure to fabric only
outside of _check_if_nic_is_primary() which is expected to fail if
nic is not primary.
Add two types of reportable errors for IMDS metadata:
- add ReportableErrorImdsUrlError() for url errors.
- add ReportableErrorImdsMetadataParsingException() for parsing errors.
Tweak ReportableError repr to be a bit friendlier.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Add host_only flag to _report_failure() to allow caller to only
report the failure to host. This is for cases where we don't want
_report_failure() to attempt DHCP or we expect that we may recover
from the reported error (there is no issue reporting multiple times
to host, whereas fabric reports will immediately fail the VM
provisioning).
- Add ReportableErrorDhcpLease() to report lease failures.
- Add ReportableErrorDhcpInterfaceNotFound() to report errors where the
DHCP interface hasn't been found yet.
- Add TestReportFailure class with new test coverage. Will migrate other
_report_failure() tests in the future as they currently depend on
TestAzureDataSource/CiTestCase.
Future work will add the interface name to supporting data, but as that
information is not available with iface=None, another PR will explicitly
add a call to net.find_fallback_nic() to specify it.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
| |
It was only used by Hyper-V which now has a filtering
mechanism that does not require the use of a denylist.
This exposed some issues with tests misspelling "hv_netvsc"
and using unmatched mac addresses. This fixes those to work
with the current filter that does not rely on the driver name.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
| |
|
|
|
|
|
|
|
| |
Systems running systemd-resolved or dnsmasq can utlize more than three
namervers. Older systems will just use the first three and ignore the
rest.
Signed-off-by: Major Hayden <major@redhat.com>
|
|
|
|
|
|
| |
Provide an option to suppress error logging from mount_cb as some
errors can be expected error and handled appropriately by
DataSources. For example: failure to mount NTFS volumes on VMs that
do not have NTFS drivers.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
(#2142)
DSA and ED25519 key types are not supported when FIPS is enabled in crypto.
Check if FIPS has been enabled on the system and if so, do not generate those
key types. Presently the check is only available on Linux systems.
LP: 2017761
RHBZ: 2187164
Signed-off-by: Ani Sinha <anisinha@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add success reporting to the host via KVP.
- Move _report_failure_to_host() into kvp module.
- Tweak error description to use result=error instead of
PROVISIONING_ERROR: ...
- Use result=success for the successful ("ready") reports.
- report_x_via_kvp => report_x_to_host for consistency with fabric.
ReportableError.as_description() => as_encoded_report()
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
| |
The `network_data.json` allows the definition of the DNS through the
`services` list at the network level.
See:
- https://opendev.org/openstack/nova/src/commit/700db274c613d6f8f30e5cdc3462beaeb0fda456/nova/tests/unit/network/test_network_info.py#L979-L980
- https://opendev.org/openstack/metalsmith/src/commit/f98dfa61c1d7475b81c20dabbf2c74198c38c793/metalsmith/test/test_network_metadata.py#L52-L90
- https://opendev.org/openstack/nova/commit/4b333b989dfc778a8b61db4a1b8552e988a10471
|
|
|
|
|
|
|
|
|
|
|
| |
The CLI cloud-init schema now asserts that the leading header comment in user-data files is a valid user-data type. Raise an informative error otherwise about valid user-data types.
For user-data files declared with '## template: jinja', render those files first sourcing jinja variables from /run/cloud-init/instance-data.json or a new --instance-data parameter.
Once the jinja template is rendered, validate schema of the resulting #cloud-config user-data.
This branch also ensures any errors and deprecation warnings are unique.
LP: #1881925
|
|
|
|
|
|
|
| |
Google wants to allow users to make changes on nics while the instance
is stopped. Activate network discovery on every boot.
Additionally, skip the call to `netplan generate` if the rendered
config is the same on subsequent boots.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Because user data and vendor data may contain sensitive information,
this commit ensures that any user data or vendor data written to
instance-data.json gets redacted and is only available to root user.
Also, modify the permissions of cloud-init.log to be 640, so that
sensitive data leaked to the log isn't world readable.
Additionally, remove the logging of user data and vendor data to
cloud-init.log from the Vultr datasource.
LP: #2013967
CVE: CVE-2023-1786
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Azure can report provisioning failures via the Wireserver health
endpoint. However, in the event of networking failures or Wireserver
issues, this report cannot be made and the VM will result in an OS
provisioning timeout and a generic error is presented to the user.
Report the failure via KVP using the "PROVISIONING_REPORT" key so
that the host can relay the provisioning error report to the user
when the VM fails to provision.
The format used is subject to change and/or removal.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of a fixed number of retries, allow up to 5 minutes to fetch
metadata from IMDS. The current approach allows for up to 11 attempts
depending on the path. Given the timeout setting, this can vary from
~11 seconds up to ~32 seconds depending on whether or not read/connection
timeouts are encountered.
Delaying boot on the rare occasion that IMDS is delayed is better than
ignoring the metadata as it ensures the VM is configured as expected.
This is a very conservative timeout and may be reduced in the future.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move isc-dhclient code to dhcp.py
In support of the upcoming deprecation of
isc-dhcp-client, this code refactors current
dhcp code into classes in dhcp.py. The
primary user-visible change should be the
addition of the following log:
dhcp.py[DEBUG]: DHCP client selected: dhclient
This code lays groundwork to enable
alternate implementations to live side by
side in the codebase to be selected with
distro-defined priority fallback. Note that
maybe_perform_dhcp_discovery() now selects
which dhcp client to call, and then runs the
corresponding client's dhcp_discovery()
method. Currently only class IscDhclient is
implemented, however a yet-to-be-implemented
class Dhcpcd exists to test fallback behavior
and this will be implemented in part two of
this series.
Part of this refactor includes shifting
dhclient service management from hardcoded
calls to the distro-defined manage_service()
method in the *BSDs. Future work is required
in this area to support multiple clients via
select_dhcp_client().
|
|
|
|
|
|
| |
Save a few characters by decoding it as utf-8 string rather than using
the bytes representation.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When provisioning failures occur an Azure, a generic description is
used in the report and ultimately returned to the user. To improve
the user experience, report details of the failure in a manner that is
parsable, readable and succinct. The current approach is to use csv
with a custom delimiter ("|") and quote character ("'"). This format
may change in the future.
Gracefully handle reportable errors thrown while crawling metadata and
treat other exceptions as ReportableErrorUnhandledException. Future
work will introduce more reportable errors to handle the expected
failure cases.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
| |
When defining json schema in a63f45f7, we were a bit too strict by
setting `minItems: 1` on the `users:` list.
This schema definition regressed the ability to prevent default_user
creation with user-data. Remove that schema constraint because the
code already supports this case.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Add query_system_uuid() for getting system uuid from dmi in
normalized (lower-cased) form.
- Add byte_swap_system_uuid() to convert a system uuid for gen1
instances to the compute.vmId as presented by IMDS.
- Add convert_system_uuid_to_vm() to convert system uuid to vm
id depending on whether it is gen1 or gen2.
- Add is_vm_gen1() to determine if VM is Azure's gen1 by checking
for available of EFI (used in gen2).
- Add query_vm_id() helper to get VM id without system uuid.
- Move ChassisAssetTag from Azure helpers into identity.
- Update DataSourceAzure._iid() to use this module.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
| |
Historically ds=nocloud-net was a required argument
for the user to pass in to tell cloud-init which mode
to use. This argument, however, is redundant when a
seedfrom argument is passed. Allow the mode to be
automatically determined, so that the user need not
pass a mode configuration to achieve desired behavior.
|
|
|
|
|
|
|
|
| |
Truncate any trailing semi-colon delimited kernel
commandline parameters when trying to match the
designated datasource from /proc/cmdline.
This was broken in 612b4de892d on systemd systems.
Add an integration test for this codepath.
|
|
|
| |
Fixes test broken by e02c460 on Focal.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add support for IPv4 and/or IPv6 provided metadata. Upcoming
features will allow for Instances to be started with only IPv6
connectivity. The datasource must be able to fetch its metadata from
the following endpoints :
* IPv4 on http://169.264.42.42/conf
* IPv6 on http://[fd00:42::42]/conf
This URL may eventually be overridden in the future by a DNS
resolvable URL defined in /etc/cloud/config.d/scaleway.conf
Add support for the configuration of one or many IP addresses when
provided by the `public_ips` metadata key. In such configurations,
instances no longer have a `private_ip`. The previous configuration
method is kept for backward compatibility.
This commit also fixes a bug in the definition of ssh keys from tags.
|
|
|
|
|
| |
If an exception occurs during EphemeralIPv4Network setup, any routes
that were setup need to be torn down. This wasn't happening, and this
commit adds the teardown.
|
|
|
|
|
|
|
|
| |
Cloud-init's host key generation mimics that of sshd-keygen.
It used to generate 640 permissions, but going forward it
should be 600. Check sshd version to set the permissions
appropriately.
LP: #2011291
|
|
|
|
|
|
| |
- deprecate ci.ds= and ci.datasource= in favor of ds=
- enable semi-colon-delimited datasource everywhere
- add support for case-insensitive datasource match
- add integration tests
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Daily PPA builds were crashing due to
refactor: stop passing log instances to cc_* handlers (#2016).
When dropping the logger argument, the tuple became a list which
then gets expanded in the call to do_resize() and crashes.
```
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/config/modules.py", line 257, in _run_modules
run_name, mod.handle, func_args, freq=freq
File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 67, in run
return self._runners.run(name, functor, args, freq, clear_on_fail)
File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 172, in run
results = functor(**args)
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_resizefs.py", line 309, in handle
args=(resize_cmd),
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2722, in log_time
ret = func(*args, **kwargs)
TypeError: do_resize() takes 1 positional argument but 2 were given
```
Restore args as a tuple.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
| |
Test was missing a mock of subp.which. This was missed because dhclient
is installed on machines it was tested on.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some variants of dhclient will exit with non-zero codes on lease
failure. For example, on RHEL 8.7:
```
[cpatterson@test-rhel87 ~]$ sudo /usr/sbin/dhclient -1 -v -lf /tmp/my.lease -pf /tmp/my.pid bridge2nowhere -sf /bin/true
Internet Systems Consortium DHCP Client 4.3.6
Copyright 2004-2017 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on LPF/bridge2nowhere/42:ef:d5:38:1d:19
Sending on LPF/bridge2nowhere/42:ef:d5:38:1d:19
Sending on Socket/fallback
Created duid "\000\004E<\225X\232\304J\337\243\026T\324\243O\270\177".
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 4 (xid=0x777bc142)
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 7 (xid=0x777bc142)
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 13 (xid=0x777bc142)
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 6 (xid=0x777bc142)
No DHCPOFFERS received.
Unable to obtain a lease on first try. Exiting.
[cpatterson@test-rhel87 ~]$ echo $?
2
```
This results in an unhandled subp.ProcessExecutionError exception.
Catch these failures and re-raise as NoDHCPLeaseError.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Pull out remaining PPS handling bits from _poll_imds() and add two
explicit methods for the overloaded path:
- _wait_for_pps_running_reuse() for running PPS logic.
- _wait_for_pps_unknown_reuse() for unknown and recovery PPS logic.
For consistency:
- Rename _wait_for_all_nics_ready() -> _wait_for_pps_savable_reuse().
- Move reporting ready logic into _wait_for_pps_os_disk_shutdown().
Drop several impacted tests as coverage already exists in
TestProvisioning, and update the rest to handle the +/- 1 DHCP attempt
due to varying assumptions around PPS state and DHCP.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Only set additionalProperties = False on draft4 schema
definition in jsonschema 3.0 or greater
because cloud-init globally registers its draft4 extensions
as the primary validator for any draft4-based schemas in
the same python process.
This affects solutions such as subiquity and
ubuntu-desktop-installer which invoke jsonschema.validate
in the same process at runtime just after calling
cloudinit.schema.get_jsonschema_validator.
The resulting Tracebacks are seen as something like:
jsonschema.exceptions.SchemaError:
{'$ref': '#/definitions/ref_id'} is not valid under any of the
given schema
Background:
cloud-init needs to extend draft4 schema to better
validate and warn 'deprecated' properties in draft4-based
cloud-init schema definitions. Our unittests also attempt
to strictly validate any meta schema definitions for the
cc_* config modules.
To accomplish strict meta schema validation cloud-init makes
a copy of the draft4 meta schema and adds an
'additionalProperties' = True to that schema to raise specific
errors and catch typos in cc_ module schema definitions.
Given that cloud-init at runtime extends and registers
a draft4 schema validator, any external consumers
of jsonschema.validate with draft4-base schemas are
exposed to cloud-init's validator so let's limit our risk
exposure.
For python 2.6.0, we cannot specify make draft4 schema
strict because any "$ref" keys are not yet resolved
to their actual #/defintions/<id> values so the traceback above
will always be generated in 'strict' mode for complex schemas.
This does not affect jsonschema 3.0+ which appears to resolve
schema $refs values before schema validation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When fetching metadata in _check_if_nic_is_primary() the retry count
is 300, but fails out after 10 connection errors. In some cases,
fetching from IMDS may fail with read timeout for more than 10 attempts,
far sooner than the desired 300.
Keeping the existing max_connection_errors = 10 is fine so long
as it is truly a connection error. These generally shouldn't occur
when using the primary NIC.
Always retry on timeout errors (up until desired limit) and count
only connections errors against max_connection_errors.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
| |
|
|
|
|
|
|
| |
Using individual release and platform marks to specify our test support
matrix was leading to too many marks specifying different combinations
of things. Rather, we can rely on the "skipif" mark to perform any
needed release or platform checks.
|
|
|
|
| |
Use the module level Log instances instead of passing log instances
to the cc_* handlers
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Patch is_vmware_platform to return False to avoid failure:
```
def test_no_data_access_method(self):
ds = get_ds(self.tmp)
ds.vmware_rpctool = None
> ret = ds._get_data()
tests/unittests/sources/test_vmware.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cloudinit/sources/DataSourceVMware.py:193: in _get_data
if require_vmware_platform and not is_vmware_platform():
cloudinit/sources/DataSourceVMware.py:401: in is_vmware_platform
system_type = dmi.read_dmi_data("system-product-name")
cloudinit/dmi.py:180: in read_dmi_data
return _call_dmidecode(key, dmidecode_path)
cloudinit/dmi.py:130: in _call_dmidecode
(result, _err) = subp.subp(cmd)
E RuntimeError: called subp. set self.allowed_subp=True to allow
E subp(['/usr/sbin/dmidecode', '--string', 'system-product-name'])
tests/unittests/helpers.py:176: RuntimeError
```
Bypassing is_vmware_platform() avoids the dmi reads.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
| |
Set permissions if file doesn't exist. Leave them if it does.
LP: #2011783
Co-authored-by: Chad Smith <chad.smith@canonical.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Recent changes to override_ds_detect() triggers a call to get_cmdline(),
which invokes subp.subp() for various container checks. This causes
tests to fail when running a specific test module instead of the full
set. This is because test_smartos.py on module load will trigger these
calls and the lru_cache() will retain the results. So if the module
does not load, the tests will fail.
Patch util.get_cmdline() for all data source tests to avoid this
behavior.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Another physical modem which has duplicate MAC addresses.
Cloud-init needs to ignore the subordinate devices which are
associated with the qmi_wwan driver.
Fixes network rendering for the following modems:
Quectel EG25
Quectel RM510Q-GLHA
Sierra Wireless MC7455
LP: #2008888
|
|
|
|
|
|
|
|
|
| |
Sending dhclient command failed for InfiniBand ports because
dhcp-client-identifier is not specified.
So, providing this patch to allow send dhcp-client-identifier hardware
with the dhclient command for InfiniBand ports.
Signed-off-by: waleedm <waleedm@nvidia.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Do not prompt for CloudName when instance-data.json exists and is valid
YAML.
When instance-data.json exists, general-hooks/cloud_init.py will add the
following fields to bug reports:
CloudName, CloudID, CloudPlatform and CloudSubplatform.
Downstream ubuntu packaging braches deliver:
debian/apport-general-hook.py to
/usr/share/apport/general-hooks/cloud-init.py
Only prompt in during apport bug when the general-hook can't
process instance-data.json.
|