summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* WIP: split integration and cloud_tests requirementsdepsDaniel Watkins2020-11-033-28/+33
|
* Make some language improvements in growpart documentation (#649)Shane Frasier2020-11-032-5/+7
| | | | | | | | | | | * Fix awkward English in sentence * Add the missing word "the" * Fix misspelling * Add @jsf9k as a contributor Co-authored-by: Rick Harding <rharding@mitechie.com>
* Revert ".travis.yml: use a known-working version of lxd (#643)" (#650)Daniel Watkins2020-11-031-1/+1
| | | | | | This reverts commit f72d0cb7a03cc311700ea8795edf2454172c7414. A new release of lxd has been released to the stable channel, which includes a fix for the issue we were previously working around.
* Fix not sourcing default 50-cloud-init ENI file on Debian (#598)WebSpider2020-11-032-0/+2
| | | | * Include both Ubuntu-style cfg file, and Debian-style directory in ENI * Add WebSpider as contributor
* remove unnecessary reboot from gpart resize (#646)Mina Galić2020-11-021-4/+0
| | | | | a reboot after `gpart resize` hasn't been necessary since ca FreeBSD 9.2 Co-authored-by: Rick Harding <rharding@mitechie.com>
* cloudinit: move dmi functions out of util (#622)Scott Moser2020-11-0228-312/+348
| | | | | | | | | This just separates the reading of dmi values into its own file. Some things of note: * left import of util in dmi.py only for 'is_container' It'd be good if is_container was not in util. * just the use of 'util.is_x86' to dmi.py * open() is used directly rather than load_file.
* integration_tests: various launch improvements (#638)Daniel Watkins2020-11-023-4/+18
| | | | | | | | | * integration_tests: fix passing launch_kwargs to session_cloud.launch * integration_tests: log the launch_kwargs before launching instances * integration_tests: add support for specifying instance name for tests Co-authored-by: Rick Harding <rharding@mitechie.com>
* test_lp1886531: don't assume /etc/fstab exists (#639)Daniel Watkins2020-11-021-1/+1
| | | | | | | As the bug manifested because groovy doesn't ship /etc/fstab, we should not assume that /etc/fstab will be present for us to remove in our bootcmd. Co-authored-by: Rick Harding <rharding@mitechie.com>
* Remove Ubuntu restriction from PR template (#648)James Falcon2020-11-021-1/+1
|
* util: fix mounting of vfat on *BSD (#637)Mina Galić2020-11-022-4/+45
| | | | Fix mounting of vfat filesystems by normalizing the different names for vfat to "msdos" which works across BSDs.
* conftest: improve docstring for disable_subp_usage (#644)Daniel Watkins2020-11-021-12/+22
| | | | | Specifically by fixing references to `util`, adding some missing formatting, and adding information about a corner case of mock usage that `disable_subp_usage` breaks.
* doc: add example query commands to debug Jinja templates (#645)Chad Smith2020-11-021-0/+16
|
* Correct documentation and testcase data for some user-data YAML (#618)dermotbradley2020-10-3012-23/+25
| | | | | | | | | | | | | | | | | | | | | | | | | For cc_users_groups the user setting "expiredate" must be quoted in order for the relevant flag and value to be then passed to the useradd command. It its vaiue is not quoted then it is treated as Python type datetime.date and in `cloudinit/distros/__init__.py` the below "is it a string" condition fails and so no "--expiredate" parameter is passed to useradd and therefore it has no effect: ``` if key in useradd_opts and val and isinstance(val, str): useradd_cmd.extend([useradd_opts[key], val]) ``` For cc_users_groups, the user setting "inactive" does not actually disable accounts, the useradd "--inactive" option actually defines the number of days after password expiry that users can still login. So I have changed the docs to show it taking a quoted value of days (which works with the current code) rather than a boolean value. The quotes are necessary, like expiredate above, so that the value is also passed to the useradd command. For cc_power_state_change.py the "delay" setting value needs to have quotes around it as otherwise its leading plus sign will be stripped off.
* Hetzner: Fix instance_id / SMBIOS serial comparison (#640)Markus Schade2020-10-302-2/+3
| | | | | | | Fixes erroneous string/int comparison introduced in 1431c8a metadata['instance-id'] is an integer but the value read from smbios is a string. The comparision would cause TypeError.
* .travis.yml: use a known-working version of lxd (#643)Daniel Watkins2020-10-301-1/+1
| | | | | The latest lxd 4.7 snap causes failures; while we're diagnosing those and addressing the issues (either upstream or in our code), pin at lxd 4.6 so we can continue to land PRs in cloud-init trunk.
* tools/build-on-freebsd: fix comment explaining purpose of the script (#635)Mina Galić2020-10-291-3/+6
| | | | | | Since there is now an official port for cloud-init in FreeBSD, this script has a different purpose. It's an interface between cloud-init maintainers, and the FreeBSD port maintainers for quickly seeing which dependencies have changed.
* Hetzner: initialize instance_id from system-serial-number (#630)Markus Schade2020-10-292-14/+41
| | | | | | | Hetzner Cloud also provides the instance ID in SMBIOS information. Use it to locally check_instance_id and to compared with instance_id from metadata service. LP: #1885527
* Explicit set IPV6_AUTOCONF and IPV6_FORCE_ACCEPT_RA on static6 (#634)Eduardo Otubo2020-10-295-2/+113
| | | | | | | | | | | | | | | | | | | | | | | | | | The static and static6 subnet types for network_data.json were being ignored by the Openstack handler, this would cause the code to break and not function properly. As of today, if a static6 configuration is chosen, the interface will still eventually be available to receive router advertisements or be set from NetworkManager to wait for them and cycle the interface in negative case. It is safe to assume that if the interface is manually configured to use static ipv6 address, there's no need to wait for router advertisements. This patch will set automatically IPV6_AUTOCONF and IPV6_FORCE_ACCEPT_RA both to "no" in this case. This patch fixes the specific behavior only for RHEL flavor and sysconfig renderer. It also introduces new unit tests for the specific case as well as adjusts some existent tests to be compatible with the new options. This patch also addresses this problem by assigning the appropriate subnet type for each case on the openstack handler. rhbz: #1889635 rhbz: #1889635 Signed-off-by: Eduardo Otubo otubo@redhat.com
* get_interfaces: don't exclude Open vSwitch bridge/bond members (#608)Lukas Märdian2020-10-293-3/+49
| | | | | | | If an OVS bridge was used as the only/primary interface, the 'init' stage failed with a "Not all expected physical devices present" error, leaving the system with a broken SSH setup. LP: #1898997
* Add config modules for controlling IBM PowerVM RMC. (#584)Aman3062020-10-285-0/+414
| | | | | | | | | | | | | Reliable Scalable Cluster Technology (RSCT) is a set of software components that together provide a comprehensive clustering environment(RAS features) for IBM PowerVM based virtual machines. RSCT includes the Resource Monitoring and Control (RMC) subsystem. RMC is a generalized framework used for managing, monitoring, and manipulating resources. RMC runs as a daemon process on individual machines and needs creation of unique node id and restarts during VM boot. LP: #1895979 Co-authored-by: Scott Moser <smoser@brickies.net>
* lp-to-git-users: adding asciiprod (#629)Markus Schade2020-10-281-1/+2
| | | Mapped from lp-markusschade
* Update network config docs to clarify MAC address quoting (#623)dermotbradley2020-10-273-32/+57
| | | | Also update MAC addresses used in testcases to remove quotes where not required and add single quotes where quotes are required.
* gentoo: fix hostname rendering when value has a comment (#611)Manuel Aguilera2020-10-273-4/+33
| | | | | | | | | | | | | | | | | | Gentoo's hostname file format instead of being just the host name is hostname=thename". The old code works fine when the file has no comments but if there is a comment the line ``` gentoo_hostname_config = 'hostname="%s"' % conf ``` can render an invalid hostname file that looks similar to ``` hostname="#This is the host namehello" ``` The fix inserts the hostname in a gentoo friendly way so that it gets handled by HostnameConf as a whole and comments are handled and preserved
* refactor integration testing infrastructure (#610)James Falcon2020-10-263-136/+202
| | | | | | | | | * Separated IntegrationClient into separate cloud and instance abstractions. This makes it easier to control the lifetime of the pycloudlib's cloud and instance abstractions separately. * Created new cloud-specific subclasses accordingly * Moved platform parsing and initialization code into its own file * Created new session-wide autorun fixture to automatically initialize and destroy the dynamic cloud
* stages: don't reset permissions of cloud-init.log every boot (#624)Daniel Watkins2020-10-234-3/+114
| | | | | | | | | | ensure_file needed modification to support doing this, so this commit also includes the following changes: test_util: add tests for util.ensure_file util: add preserve_mode parameter to ensure_file util: add (partial) type annotations to ensure_file LP: #1900837
* docs: Add how to use cloud-localds to boot qemu (#617)Joshua Powers2020-10-221-0/+56
| | | | | | | | | | | | | | | * docs: Add hot to use cloud-localds to boot qemu There is a complete lack of documentation on using cloud-localds with cloud-init to boot an image locally. Drive by, added some more whitepapers, blogs, and videos * fix line length * * add where cloud-localds comes from * add more specific example with metadata and network config * Add link to cloud-utils package
* Drop vestigial update_resolve_conf_file function (#620)Scott Moser2020-10-212-32/+0
| | | | | update_resolve_conf_file is no longer used. The last reference to it was removed in c3680475f9c970, which was itself a "remove dead code" commit.
* cc_mounts: correctly fallback to dd if fallocate fails (#585)Daniel Watkins2020-10-202-4/+37
| | | | | | | | | | `create_swap()` was previously catching and not re-raising the ProcessExecutionError that indicated swap creation failure; this meant that the fallback logic could never be triggered. This commit adds the required re-raise (as well as removing a duplicated log message). LP: #1897099
* .travis.yml: add integration-tests to Travis matrix (#600)Daniel Watkins2020-10-201-0/+81
| | | | | | | | | | This is implemented as a copy/paste of the `citest` integration testing script: the two are not intended to co-exist long-term, so it isn't worth further complicating an already complex part of our Travis configuration for short-term code reuse. The two changes from the `citest` definition: the test framework executed on the last line of `script`, and it is given a `name` so we can easily ignore failures.
* ssh_util: handle non-default AuthorizedKeysFile config (#586)Eduardo Otubo2020-10-202-6/+6
| | | | | | | | | | | | | | | | | | | | | The following commit merged all ssh keys into a default user file `~/.ssh/authorized_keys` in sshd_config had multiple files configured for AuthorizedKeysFile: commit f1094b1a539044c0193165a41501480de0f8df14 Author: Eduardo Otubo <otubo@redhat.com> Date: Thu Dec 5 17:37:35 2019 +0100 Multiple file fix for AuthorizedKeysFile config (#60) This commit ignored the case when sshd_config would have a single file for AuthorizedKeysFile, but a non default configuration, for example `~/.ssh/authorized_keys_foobar`. In this case cloud-init would grab all keys from this file and write a new one, the default `~/.ssh/authorized_keys` causing the bug. rhbz: #1862967 Signed-off-by: Eduardo Otubo <otubo@redhat.com>
* bddeb: new --packaging-branch argument to pull packaging from branch (#576)Paride Legovini2020-10-193-8/+91
| | | | | | | | | bddeb builds a .deb package using the template packaging files in packages/debian/. The new --packaging-branch flag allows to specify a git branch where to pull the packaging (i.e. the debian/ directory) from. This is useful to build a .deb package from master with the very same packaging which is used for the uploads.
* Add more integration tests (#615)lucasmoura2020-10-1911-0/+547
| | | | | | | | | | | | | | | | Translate the following tests from `cloud_tests` to the new integration test framework: * test_runcmd.py * seed_random_data.py * set_hostname.py * set_hostname_fqdn.py * snap.py * ssh_auth_key_fingerprints_disable.py * ssh_auth_key_fingerprints_enable.py * ssh_import_id.py * ssh_keys_generate.py * ssh_keys_provided.py * timezone.py * write_files.py
* DataSourceAzure: write marker file after report ready in preprovisioning (#590)Johnson Shi2020-10-163-22/+90
| | | | | | | | | | | | | | | | | | DataSourceAzure previously writes the preprovisioning reported ready marker file before it goes through the report ready workflow. On certain VM instances, the marker file is successfully written but then reporting ready fails. Upon rare VM reboots by the platform, cloud-init sees that the report ready marker file already exists. The existence of this marker file tells cloud-init not to report ready again (because it mistakenly assumes that it already reported ready in preprovisioning). In this scenario, cloud-init instead erroneously takes the reprovisioning workflow instead of reporting ready again.
* integration_tests: emit settings to log during setup (#601)Daniel Watkins2020-10-162-0/+12
|
* integration_tests: implement citest tests run in Travis (#605)Daniel Watkins2020-10-164-0/+335
| | | | | | | | | | Specifically: * `apt_configure_sources_list` * `ntp_servers` * `set_password_list` * `users_groups` Although not currently run in Travis, `set_password_list_string` was ported over alongside `set_password_list` (as `test_set_password`).
* Add Azure support to integration test framework (#604)James Falcon2020-10-152-1/+2
|
* openstack: consider product_name as valid chassis tag (#580)Adrian Vladu2020-10-153-1/+33
| | | | | | | | | | | | Consider valid product names as valid chassis asset tags when detecting OpenStack platform before crawling for OpenStack metadata. As `ds-identify` tool uses product name as valid chassis asset tags, let's replicate the behaviour in the OpenStack platform detection too. This change should be backwards compatible and a temporary fix for the current limitations on the OpenStack platform detection. LP: #1895976
* azure: clean up and refactor report_diagnostic_event (#563)Johnson Shi2020-10-153-99/+142
| | | This moves logging into `report_diagnostic_event`, to clean up its callsites.
* net: add the ability to blacklist network interfaces based on driver during ↵Anh Vo2020-10-134-40/+86
| | | | enumeration of physical network devices (#591)
* integration_tests: don't error on cloud-init failure (#596)Daniel Watkins2020-10-061-3/+3
| | | | | | pycloudlib's default behaviour is to raise an exception if cloud-init fails to run in an instance being launched. For cloud-init testing, we want our test assertions to flag up failures, so we disable this behaviour for instances we launch.
* integration_tests: improve cloud-init.log assertions (#593)Daniel Watkins2020-10-061-0/+26
|
* conftest.py: remove top-level import of httpretty (#599)Daniel Watkins2020-10-062-3/+11
| | | | This means that the integration tests do not need to install test-requirements.txt in order to successfully import `conftest.py`.
* tox.ini: add integration-tests testenv definition (#595)Daniel Watkins2020-10-051-0/+9
|
* PULL_REQUEST_TEMPLATE.md: empty checkboxes need a space (#597)Daniel Watkins2020-10-051-3/+3
|
* add integration test for LP: #1886531 (#592)Daniel Watkins2020-10-021-0/+27
|
* Initial implementation of integration testing infrastructure (#581)James Falcon2020-10-0110-5/+547
|
* Fix name of ntp and chrony service on CentOS and RHEL. (#589)Scott Moser2020-10-011-0/+8
| | | | | | | | | | | | The service installed by the CentOS and RHEL 'ntp' package is ntpd.service not ntp.service Fix that for those two distros. Also fix chrony service from 'chrony' to 'chronyd'. LP: #1897915
* Adding a PR template (#587)James Falcon2020-09-291-0/+18
|
* Azure parse_network_config uses fallback cfg when generate IMDS network cfg ↵Johnson Shi2020-09-242-120/+164
| | | | | | | | | fails (#549) Azure datasource's `parse_network_config` throws a fatal uncaught exception when an exception is raised during generation of network config from IMDS metadata. This happens when IMDS metadata is invalid/corrupted (such as when it is missing network or interface metadata). This causes the rest of provisioning to fail. This changes `parse_network_config` to be a non-fatal implementation. Additionally, when generating network config from IMDS metadata fails, fall back on generating fallback network config (`_generate_network_config_from_fallback_config`). This also changes fallback network config generation (`_generate_network_config_from_fallback_config`) to blacklist an additional driver: `mlx5_core`.
* features: refresh docs for easier out-of-context reading (#582)Daniel Watkins2020-09-231-9/+21
|