| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
| |
Signed-off-by: Jacob Salmela <jacob.salmela@hpe.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Linux, we only disable login via Password authentication.
On FreeBSD, we were right out locking the account.
This was reported back when #93 was merged, but back then I didn't
understand that.
Newly created FreeBSD users can now login with with SSH keys.
Sponsored by: The FreeBSD Foundation
LP: #1854594
Fixes: #3507
|
|
|
|
|
|
|
|
|
| |
this FreeBSD specific resizer resizes the root partition and grows the
Filesystem all in one.
All we have to do is call ``service growfs onestart``
Document behaviour: especially that growfs will insert a swap partition
if none is present, unless instructed otherwise.
Sponsored by: The FreeBSD Foundation
|
| |
|
|
|
|
|
| |
This fixes KeyError on specific network configuration when running
cloud-init on "network" stage. The same problem was mentioned in
#746 and #1041.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix cloud-init schema --system being unable to find merged
userdata stored at /var/lib/cloud/instance/cloud_config.txt.
Init.paths.get_ipath only has visibility to merged cloud config in
/var/lib/cloud/<instance_id>/cloud-config.txt after fetching the
existing cached datasource which provides instance-id from metadata
in order to determine the unique instance-id which represents the
path to the cloud-config.txt.
To support reuse of read_cfg_paths helper function, add an optional
parameter fetch_existing_datasource which indicates whether reading
the existing datasource is necessary for this helper function.
cloud-init schema --system calls read_cfg_paths providing
fetch_existing_datasource="trust" prior to calls to
paths.get_ipath().
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Always report failure to host, but report failure to fabric only
outside of _check_if_nic_is_primary() which is expected to fail if
nic is not primary.
Add two types of reportable errors for IMDS metadata:
- add ReportableErrorImdsUrlError() for url errors.
- add ReportableErrorImdsMetadataParsingException() for parsing errors.
Tweak ReportableError repr to be a bit friendlier.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
| |
The code is already in the Dragonfly path, we just need to use it.
LP: #2016350
Sponsored by: The FreeBSD Foundation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Add host_only flag to _report_failure() to allow caller to only
report the failure to host. This is for cases where we don't want
_report_failure() to attempt DHCP or we expect that we may recover
from the reported error (there is no issue reporting multiple times
to host, whereas fabric reports will immediately fail the VM
provisioning).
- Add ReportableErrorDhcpLease() to report lease failures.
- Add ReportableErrorDhcpInterfaceNotFound() to report errors where the
DHCP interface hasn't been found yet.
- Add TestReportFailure class with new test coverage. Will migrate other
_report_failure() tests in the future as they currently depend on
TestAzureDataSource/CiTestCase.
Future work will add the interface name to supporting data, but as that
information is not available with iface=None, another PR will explicitly
add a call to net.find_fallback_nic() to specify it.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
| |
It was only used by Hyper-V which now has a filtering
mechanism that does not require the use of a denylist.
This exposed some issues with tests misspelling "hv_netvsc"
and using unmatched mac addresses. This fixes those to work
with the current filter that does not rely on the driver name.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Azure is introducing a new VF ("MANA") that will initially behave
similarly to mlx4/5 but cannot be denylisted in the same manner.
This is because the synthetic interface (hv_netvsc) will no longer
be required to function in the future which means we must
intelligently filter the VFs out instead of relying solely on the
driver name.
- Isolate filtering logic for Hyper-V's SR-IOV/VFs when used
with synthetic hv_netvsc interfaces.
- Move the filter up to get_interfaces() from
get_interfaces_by_mac_on_linux() to increase coverage of the
filter. With this in place, we should be able to purge the
"blacklist_drivers" across the codebase as it will no longer be
necessary unless there are other paths to be considered.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Much of the filtering logic used in get_interfaces() is duplicated in
find_candidate_nics_on_linux(). Consolidate the two interfaces by
updating find_candidate_nics_on_linux() to use get_interfaces() and
provide boolean toggles to maintain existing behavior.
The following parameters have been added to get_interfaces():
log_filtered_reasons: bool = False
filter_openvswitch_internal: bool = True
filter_vlan: bool = True
filter_without_own_mac: bool = True
filter_zero_mac: bool = True
It may be that these deltas are unwanted or harmless, but the toggles
could be removed independently with more deliberate consideration.
Similarly, logging could be extended if desired for
log_filtered_reasons.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
| |
Systems running systemd-resolved or dnsmasq can utlize more than three
namervers. Older systems will just use the first three and ignore the
rest.
Signed-off-by: Major Hayden <major@redhat.com>
|
|
|
|
|
|
| |
Provide an option to suppress error logging from mount_cb as some
errors can be expected error and handled appropriately by
DataSources. For example: failure to mount NTFS volumes on VMs that
do not have NTFS drivers.
|
|
|
|
|
| |
* Fix logger, use instance rather than module function
* add dunder names
|
|
|
| |
This reverts commit a9cec5daa52aa9fb0a1f17dfc939e3ff61a476b9.
|
|
|
|
|
|
|
|
|
|
|
|
| |
(#2142)
DSA and ED25519 key types are not supported when FIPS is enabled in crypto.
Check if FIPS has been enabled on the system and if so, do not generate those
key types. Presently the check is only available on Linux systems.
LP: 2017761
RHBZ: 2187164
Signed-off-by: Ani Sinha <anisinha@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add success reporting to the host via KVP.
- Move _report_failure_to_host() into kvp module.
- Tweak error description to use result=error instead of
PROVISIONING_ERROR: ...
- Use result=success for the successful ("ready") reports.
- report_x_via_kvp => report_x_to_host for consistency with fabric.
ReportableError.as_description() => as_encoded_report()
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
| |
The `network_data.json` allows the definition of the DNS through the
`services` list at the network level.
See:
- https://opendev.org/openstack/nova/src/commit/700db274c613d6f8f30e5cdc3462beaeb0fda456/nova/tests/unit/network/test_network_info.py#L979-L980
- https://opendev.org/openstack/metalsmith/src/commit/f98dfa61c1d7475b81c20dabbf2c74198c38c793/metalsmith/test/test_network_metadata.py#L52-L90
- https://opendev.org/openstack/nova/commit/4b333b989dfc778a8b61db4a1b8552e988a10471
|
|
|
|
| |
util.get_mount_info() which is called from the
distro base class is linux-specific
|
|
|
|
|
|
|
|
|
|
|
| |
The CLI cloud-init schema now asserts that the leading header comment in user-data files is a valid user-data type. Raise an informative error otherwise about valid user-data types.
For user-data files declared with '## template: jinja', render those files first sourcing jinja variables from /run/cloud-init/instance-data.json or a new --instance-data parameter.
Once the jinja template is rendered, validate schema of the resulting #cloud-config user-data.
This branch also ensures any errors and deprecation warnings are unique.
LP: #1881925
|
|
|
|
|
|
|
| |
Google wants to allow users to make changes on nics while the instance
is stopped. Activate network discovery on every boot.
Additionally, skip the call to `netplan generate` if the rendered
config is the same on subsequent boots.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Because user data and vendor data may contain sensitive information,
this commit ensures that any user data or vendor data written to
instance-data.json gets redacted and is only available to root user.
Also, modify the permissions of cloud-init.log to be 640, so that
sensitive data leaked to the log isn't world readable.
Additionally, remove the logging of user data and vendor data to
cloud-init.log from the Vultr datasource.
LP: #2013967
CVE: CVE-2023-1786
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Azure can report provisioning failures via the Wireserver health
endpoint. However, in the event of networking failures or Wireserver
issues, this report cannot be made and the VM will result in an OS
provisioning timeout and a generic error is presented to the user.
Report the failure via KVP using the "PROVISIONING_REPORT" key so
that the host can relay the provisioning error report to the user
when the VM fails to provision.
The format used is subject to change and/or removal.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of a fixed number of retries, allow up to 5 minutes to fetch
metadata from IMDS. The current approach allows for up to 11 attempts
depending on the path. Given the timeout setting, this can vary from
~11 seconds up to ~32 seconds depending on whether or not read/connection
timeouts are encountered.
Delaying boot on the rare occasion that IMDS is delayed is better than
ignoring the metadata as it ensures the VM is configured as expected.
This is a very conservative timeout and may be reduced in the future.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move isc-dhclient code to dhcp.py
In support of the upcoming deprecation of
isc-dhcp-client, this code refactors current
dhcp code into classes in dhcp.py. The
primary user-visible change should be the
addition of the following log:
dhcp.py[DEBUG]: DHCP client selected: dhclient
This code lays groundwork to enable
alternate implementations to live side by
side in the codebase to be selected with
distro-defined priority fallback. Note that
maybe_perform_dhcp_discovery() now selects
which dhcp client to call, and then runs the
corresponding client's dhcp_discovery()
method. Currently only class IscDhclient is
implemented, however a yet-to-be-implemented
class Dhcpcd exists to test fallback behavior
and this will be implemented in part two of
this series.
Part of this refactor includes shifting
dhclient service management from hardcoded
calls to the distro-defined manage_service()
method in the *BSDs. Future work is required
in this area to support multiple clients via
select_dhcp_client().
|
|
|
|
|
|
| |
Save a few characters by decoding it as utf-8 string rather than using
the bytes representation.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When provisioning failures occur an Azure, a generic description is
used in the report and ultimately returned to the user. To improve
the user experience, report details of the failure in a manner that is
parsable, readable and succinct. The current approach is to use csv
with a custom delimiter ("|") and quote character ("'"). This format
may change in the future.
Gracefully handle reportable errors thrown while crawling metadata and
treat other exceptions as ReportableErrorUnhandledException. Future
work will introduce more reportable errors to handle the expected
failure cases.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
| |
When defining json schema in a63f45f7, we were a bit too strict by
setting `minItems: 1` on the `users:` list.
This schema definition regressed the ability to prevent default_user
creation with user-data. Remove that schema constraint because the
code already supports this case.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Add query_system_uuid() for getting system uuid from dmi in
normalized (lower-cased) form.
- Add byte_swap_system_uuid() to convert a system uuid for gen1
instances to the compute.vmId as presented by IMDS.
- Add convert_system_uuid_to_vm() to convert system uuid to vm
id depending on whether it is gen1 or gen2.
- Add is_vm_gen1() to determine if VM is Azure's gen1 by checking
for available of EFI (used in gen2).
- Add query_vm_id() helper to get VM id without system uuid.
- Move ChassisAssetTag from Azure helpers into identity.
- Update DataSourceAzure._iid() to use this module.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
| |
Historically ds=nocloud-net was a required argument
for the user to pass in to tell cloud-init which mode
to use. This argument, however, is redundant when a
seedfrom argument is passed. Allow the mode to be
automatically determined, so that the user need not
pass a mode configuration to achieve desired behavior.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add support for IPv4 and/or IPv6 provided metadata. Upcoming
features will allow for Instances to be started with only IPv6
connectivity. The datasource must be able to fetch its metadata from
the following endpoints :
* IPv4 on http://169.264.42.42/conf
* IPv6 on http://[fd00:42::42]/conf
This URL may eventually be overridden in the future by a DNS
resolvable URL defined in /etc/cloud/config.d/scaleway.conf
Add support for the configuration of one or many IP addresses when
provided by the `public_ips` metadata key. In such configurations,
instances no longer have a `private_ip`. The previous configuration
method is kept for backward compatibility.
This commit also fixes a bug in the definition of ssh keys from tags.
|
|
|
|
|
|
|
|
|
| |
cloud-init strips new line from "/etc/hostname" on rhel distro when processing
"/var/lib/cloud/data/previous-hostname". Although this does not pose a serious
issue, it is still better if the behavior is similar to other distros like
Ubuntu where /previous-hostname does end with a new line. Fix this issue by
using hostname parser in rhel similar to debian.
Signed-off-by: Ani Sinha <anisinha@redhat.com>
|
|
|
|
|
| |
If an exception occurs during EphemeralIPv4Network setup, any routes
that were setup need to be torn down. This wasn't happening, and this
commit adds the teardown.
|
|
|
|
|
|
|
|
| |
Cloud-init's host key generation mimics that of sshd-keygen.
It used to generate 640 permissions, but going forward it
should be 600. Check sshd version to set the permissions
appropriately.
LP: #2011291
|
|
|
|
|
|
| |
- deprecate ci.ds= and ci.datasource= in favor of ds=
- enable semi-colon-delimited datasource everywhere
- add support for case-insensitive datasource match
- add integration tests
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Daily PPA builds were crashing due to
refactor: stop passing log instances to cc_* handlers (#2016).
When dropping the logger argument, the tuple became a list which
then gets expanded in the call to do_resize() and crashes.
```
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/config/modules.py", line 257, in _run_modules
run_name, mod.handle, func_args, freq=freq
File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 67, in run
return self._runners.run(name, functor, args, freq, clear_on_fail)
File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 172, in run
results = functor(**args)
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_resizefs.py", line 309, in handle
args=(resize_cmd),
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2722, in log_time
ret = func(*args, **kwargs)
TypeError: do_resize() takes 1 positional argument but 2 were given
```
Restore args as a tuple.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some variants of dhclient will exit with non-zero codes on lease
failure. For example, on RHEL 8.7:
```
[cpatterson@test-rhel87 ~]$ sudo /usr/sbin/dhclient -1 -v -lf /tmp/my.lease -pf /tmp/my.pid bridge2nowhere -sf /bin/true
Internet Systems Consortium DHCP Client 4.3.6
Copyright 2004-2017 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on LPF/bridge2nowhere/42:ef:d5:38:1d:19
Sending on LPF/bridge2nowhere/42:ef:d5:38:1d:19
Sending on Socket/fallback
Created duid "\000\004E<\225X\232\304J\337\243\026T\324\243O\270\177".
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 4 (xid=0x777bc142)
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 7 (xid=0x777bc142)
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 13 (xid=0x777bc142)
DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 6 (xid=0x777bc142)
No DHCPOFFERS received.
Unable to obtain a lease on first try. Exiting.
[cpatterson@test-rhel87 ~]$ echo $?
2
```
This results in an unhandled subp.ProcessExecutionError exception.
Catch these failures and re-raise as NoDHCPLeaseError.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Pull out remaining PPS handling bits from _poll_imds() and add two
explicit methods for the overloaded path:
- _wait_for_pps_running_reuse() for running PPS logic.
- _wait_for_pps_unknown_reuse() for unknown and recovery PPS logic.
For consistency:
- Rename _wait_for_all_nics_ready() -> _wait_for_pps_savable_reuse().
- Move reporting ready logic into _wait_for_pps_os_disk_shutdown().
Drop several impacted tests as coverage already exists in
TestProvisioning, and update the rest to handle the +/- 1 DHCP attempt
due to varying assumptions around PPS state and DHCP.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Only set additionalProperties = False on draft4 schema
definition in jsonschema 3.0 or greater
because cloud-init globally registers its draft4 extensions
as the primary validator for any draft4-based schemas in
the same python process.
This affects solutions such as subiquity and
ubuntu-desktop-installer which invoke jsonschema.validate
in the same process at runtime just after calling
cloudinit.schema.get_jsonschema_validator.
The resulting Tracebacks are seen as something like:
jsonschema.exceptions.SchemaError:
{'$ref': '#/definitions/ref_id'} is not valid under any of the
given schema
Background:
cloud-init needs to extend draft4 schema to better
validate and warn 'deprecated' properties in draft4-based
cloud-init schema definitions. Our unittests also attempt
to strictly validate any meta schema definitions for the
cc_* config modules.
To accomplish strict meta schema validation cloud-init makes
a copy of the draft4 meta schema and adds an
'additionalProperties' = True to that schema to raise specific
errors and catch typos in cc_ module schema definitions.
Given that cloud-init at runtime extends and registers
a draft4 schema validator, any external consumers
of jsonschema.validate with draft4-base schemas are
exposed to cloud-init's validator so let's limit our risk
exposure.
For python 2.6.0, we cannot specify make draft4 schema
strict because any "$ref" keys are not yet resolved
to their actual #/defintions/<id> values so the traceback above
will always be generated in 'strict' mode for complex schemas.
This does not affect jsonschema 3.0+ which appears to resolve
schema $refs values before schema validation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When fetching metadata in _check_if_nic_is_primary() the retry count
is 300, but fails out after 10 connection errors. In some cases,
fetching from IMDS may fail with read timeout for more than 10 attempts,
far sooner than the desired 300.
Keeping the existing max_connection_errors = 10 is fine so long
as it is truly a connection error. These generally shouldn't occur
when using the primary NIC.
Always retry on timeout errors (up until desired limit) and count
only connections errors against max_connection_errors.
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
|
|
|
| |
Use the module level Log instances instead of passing log instances
to the cc_* handlers
|
|
|
|
|
|
|
| |
Set permissions if file doesn't exist. Leave them if it does.
LP: #2011783
Co-authored-by: Chad Smith <chad.smith@canonical.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Another physical modem which has duplicate MAC addresses.
Cloud-init needs to ignore the subordinate devices which are
associated with the qmi_wwan driver.
Fixes network rendering for the following modems:
Quectel EG25
Quectel RM510Q-GLHA
Sierra Wireless MC7455
LP: #2008888
|
|
|
|
|
| |
Fedora wasn't previously supported for CA handling. Enabling this
allows the testsuite to pass when ran on a Fedora system. The conf
override is the same as for rhel.
|
|
|
|
|
|
|
|
|
| |
Sending dhclient command failed for InfiniBand ports because
dhcp-client-identifier is not specified.
So, providing this patch to allow send dhcp-client-identifier hardware
with the dhclient command for InfiniBand ports.
Signed-off-by: waleedm <waleedm@nvidia.com>
|
|
|
|
|
| |
Complete the examples that the suggested code work.
The examples in this script are not correct and should be adjusted accordingly.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Do not prompt for CloudName when instance-data.json exists and is valid
YAML.
When instance-data.json exists, general-hooks/cloud_init.py will add the
following fields to bug reports:
CloudName, CloudID, CloudPlatform and CloudSubplatform.
Downstream ubuntu packaging braches deliver:
debian/apport-general-hook.py to
/usr/share/apport/general-hooks/cloud-init.py
Only prompt in during apport bug when the general-hook can't
process instance-data.json.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commit d1ffbea556a06105 enabled skipping python datasource detection on
OpenStack when no other datasources (besides DataSourceNone) can be discovered.
This allowed one to override detection, which is a requirement for OpenStack
Ironic which does not advertise itself to cloud-init.
Since no further datasources can be detected at this stage in the code, this
pattern can be generalized to other datasources to facilitate troubleshooting
or providing a general workaround to runtime detection bugs.
Additionally, this pattern can be extended to kernel commandline datasource
definition. Since kernel commandline is highest priority of the
configurations, it makes sense to override python code datasource
detection as well.
Include an integration test on LXD for this behavior that configures kernel
commandline and reboots to verify that the specified datasource is forced.
|