summaryrefslogtreecommitdiff
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* DataSourceAzure: write marker file after report ready in preprovisioning (#590)Johnson Shi2020-10-162-19/+70
| | | | | | | | | | | | | | | | | | DataSourceAzure previously writes the preprovisioning reported ready marker file before it goes through the report ready workflow. On certain VM instances, the marker file is successfully written but then reporting ready fails. Upon rare VM reboots by the platform, cloud-init sees that the report ready marker file already exists. The existence of this marker file tells cloud-init not to report ready again (because it mistakenly assumes that it already reported ready in preprovisioning). In this scenario, cloud-init instead erroneously takes the reprovisioning workflow instead of reporting ready again.
* integration_tests: emit settings to log during setup (#601)Daniel Watkins2020-10-162-0/+12
|
* integration_tests: implement citest tests run in Travis (#605)Daniel Watkins2020-10-164-0/+335
| | | | | | | | | | Specifically: * `apt_configure_sources_list` * `ntp_servers` * `set_password_list` * `users_groups` Although not currently run in Travis, `set_password_list_string` was ported over alongside `set_password_list` (as `test_set_password`).
* Add Azure support to integration test framework (#604)James Falcon2020-10-152-1/+2
|
* openstack: consider product_name as valid chassis tag (#580)Adrian Vladu2020-10-151-0/+30
| | | | | | | | | | | | Consider valid product names as valid chassis asset tags when detecting OpenStack platform before crawling for OpenStack metadata. As `ds-identify` tool uses product name as valid chassis asset tags, let's replicate the behaviour in the OpenStack platform detection too. This change should be backwards compatible and a temporary fix for the current limitations on the OpenStack platform detection. LP: #1895976
* azure: clean up and refactor report_diagnostic_event (#563)Johnson Shi2020-10-151-3/+19
| | | This moves logging into `report_diagnostic_event`, to clean up its callsites.
* net: add the ability to blacklist network interfaces based on driver during ↵Anh Vo2020-10-131-14/+35
| | | | enumeration of physical network devices (#591)
* integration_tests: don't error on cloud-init failure (#596)Daniel Watkins2020-10-061-3/+3
| | | | | | pycloudlib's default behaviour is to raise an exception if cloud-init fails to run in an instance being launched. For cloud-init testing, we want our test assertions to flag up failures, so we disable this behaviour for instances we launch.
* integration_tests: improve cloud-init.log assertions (#593)Daniel Watkins2020-10-061-0/+26
|
* add integration test for LP: #1886531 (#592)Daniel Watkins2020-10-021-0/+27
|
* Initial implementation of integration testing infrastructure (#581)James Falcon2020-10-013-0/+436
|
* Azure parse_network_config uses fallback cfg when generate IMDS network cfg ↵Johnson Shi2020-09-241-57/+73
| | | | | | | | | fails (#549) Azure datasource's `parse_network_config` throws a fatal uncaught exception when an exception is raised during generation of network config from IMDS metadata. This happens when IMDS metadata is invalid/corrupted (such as when it is missing network or interface metadata). This causes the rest of provisioning to fail. This changes `parse_network_config` to be a non-fatal implementation. Additionally, when generating network config from IMDS metadata fails, fall back on generating fallback network config (`_generate_network_config_from_fallback_config`). This also changes fallback network config generation (`_generate_network_config_from_fallback_config`) to blacklist an additional driver: `mlx5_core`.
* Add vendor-data support to seedfrom parameter for NoCloud and OVF (#570)Johann Queuniet2020-09-181-1/+23
|
* create a shutdown_command method in distro classes (#567)Emmanuel Thomé2020-09-151-14/+44
| | | | | | | | | | | | Under FreeBSD, we want to use "shutdown -p" for poweroff. Alpine Linux also has some specificities. We choose to define a method that returns the shutdown command line to use, rather than a method that actually does the shutdown. This makes it easier to have the tests in test_handler_power_state do their verifications. Two tests are added for the special behaviours that are known so far.
* network: Fix type and respect name when rendering vlan in sysconfig. (#541)Eduardo Otubo2020-09-152-4/+81
| | | | | | | | | | | | | | | Prior to this change, vlans were rendered in sysconfig with 'TYPE=Ethernet', and incorrectly rendered the PHYSDEV based on the name of the vlan device rather than the 'link' provided in the network config. The change here fixes: * rendering of TYPE=Ethernet for a vlan * adds a warning if the configured device name is not supported per the RHEL 7 docs "11.5. Naming Scheme for VLAN Interfaces" LP: #1788915 LP: #1826608 RHBZ: #1861871
* Retrieve SSH keys from IMDS first with OVF as a fallback (#509)Thomas Stringer2020-09-102-15/+62
| | | | | | | | | | | | | | | | | * pull ssh keys from imds first and fall back to ovf if unavailable * refactor log and diagnostic messages * refactor the OpenSSLManager instantiation and certificate usage * fix unit test where exception was being silenced for generate cert * fix tests now that certificate is not always generated * add documentation for ssh key retrieval * add ability to check if http client has security enabled * refactor certificate logic to GoalState
* LXD: detach network from profile before deleting it (#542)Paride Legovini2020-08-271-1/+1
| | | | | | | | | | | | | | | | | * LXD: detach network from profile before deleting it When cleaning up the bridge network created by default by LXD as part of the `lxd init` process detach the network its profile before deleting it. LXD will otherwise refuse to delete it with error: Error: The network is currently in use. Discussion with LXD upstream: https://github.com/lxc/lxd/issues/7804. LP: #1776958 * LXD bridge deletion: fail if bridge exists but can't be deleted * LXD bridge deletion: remove useless failure logging
* tox: bump the pylint version to 2.6.0 in the default run (#544)Paride Legovini2020-08-257-25/+42
| | | | | | Changes: tox: bump the pylint version to 2.6.0 in the default run Fix pylint 2.6.0 W0707 warnings (raise-missing-from)
* Azure: Add netplan driver filter when using hv_netvsc driver (#539)James Falcon2020-08-241-0/+17
| | | | | This fixes a long delay during boot of some instances. For Azure instance types using SR-IOV via the Hyper-V netvsc network driver, two network interfaces are created that share the same MAC, but only the virtual device should be configured and used. Updating the netplan configuration to filter on the hv_netvsc driver prevents netplan from trying to figure both devices. LP: #1830740
* Update the list of valid ssh keys. (#487)Ole-Martin Bratteng2020-08-211-20/+231
| | | | | | | | | | | | | Update ssh_util.py with latest list of keys (from openssh-8.3p1/sshkey.c), Added keys: sk-ecdsa-sha2-nistp256-cert-v01@openssh.com sk-ecdsa-sha2-nistp256@openssh.com sk-ssh-ed25519-cert-v01@openssh.com sk-ssh-ed25519@openssh.com ssh-xmss-cert-v01@openssh.com ssh-xmss@openssh.com LP: #1877869
* Pushing cloud-init log to the KVP (#529)Moustafa Moustafa2020-08-201-3/+71
| | | | | | | | | | | | | | | | | | | | | Push the cloud-init.log file (Up to 500KB at once) to the KVP before reporting ready to the Azure platform. Based on the analysis done on a large sample of cloud-init.log files, Here's the statistics collected on the log file size: P50 P90 P95 P99 P99.9 P99.99 137K 423K 537K 3.5MB 6MB 16MB This change limits the size of cloud-init.log file data that gets dumped to KVP to 500KB. So for ~95% of the cases, the whole log file will be dumped and for the remaining ~5%, we will get the last 500KB of the cloud-init.log file. To asses the performance of the 500KB limit, 250 VM were deployed with a 500KB cloud-init.log file and the time taken to compress, encode and dump the entries to KVP was measured. Here's the time in milliseconds percentiles: P50 P99 P999 75.705 232.701 1169.636 Another 250 VMs were deployed with this logic dumping their normal cloud-init.log file to KVP, the same timing was measured as above. Here's the time in milliseconds percentiles: P50 P99 P999 1.88 5.277 6.992 Added excluded_handlers to the report_event function to be able to opt-out from reporting the events of the compressed cloud-init.log file to the cloud-init.log file. The KVP break_down logic had a bug, where it will reuse the same key for all the split chunks of KVP which results in overwriting the split KVPs by the last one when consumed by Hyper-V. I added the split chunk index as a differentiator to the KVP key. The Hyper-V consumes the KVPs from the KVP file as chunks whose key is 512KB and value is 2048KB but the Azure platform expects the value to be 1024KB, thus I introduced the Azure value limit.
* Add Alpine Linux support. (#535)dermotbradley2020-08-196-52/+419
| | | | | | | | Add new module cc_apk_configure for creating Alpine /etc/apk/repositories file. Modify cc_ca_certs, cc_ntp, cc_power_state_change, and cc_resolv_conf for Alpine. Add Alpine template files for Chrony and Busybox NTP support. Add Alpine template file for /etc/hosts.
* Detect kernel version before swap file creation (#428)Eduardo Otubo2020-08-182-0/+123
| | | | | | | | | | According to man page `man 8 swapon', "Preallocated swap files are supported on XFS since Linux 4.18". This patch checks for kernel version before attepting to create swapfile, using dd for XFS only on kernel versions <= 4.18 or btrfs. Add new func util.kernel_version which returns a tuple of ints (major, minor) Signed-off-by: Eduardo Otubo otubo@redhat.com
* Refactor Azure report ready code (#468)Johnson Shi2020-08-131-58/+333
| | | This PR refactors Azure report ready code to include more robust tests and telemetry.
* test_util: parametrize devlist tests (#523)James Falcon2020-08-071-32/+24
|
* Recognize LABEL_FATBOOT labels (#513)James Falcon2020-08-072-83/+130
| | | | | | Update DataSourceNoCloud and ds-identify to recognize LABEL_FATBOOT labels from blkid. Also updated associated tests. LP: #1841466
* azure: disable bouncing hostname when setting hostname fails (#494)Anh Vo2020-07-221-0/+11
| | | DataSourceAzure: Gracefully handle the case of set hostname failure during provisioning
* VMware: Support parsing DEFAULT-RUN-POST-CUST-SCRIPT (#441)xiaofengw-vmware2020-07-212-0/+96
| | | | | | | | | | | Add support for VMware's vCD configuration setting DEFAULT-RUN-POST-CUST-SCRIPT. When set True, it will default vms to run post customization scripts if the VM has not been configured in VMTools with "enable-custom-scripts" set False. Add datasource documentation with a bit more context about this interaction on VMware products. With this fix, the behavior will be: * If VM administrator doesn't want others to execute a script on this VM, VMtools can set "enable-custom-scripts" to false from the utility "vmware-toolbox-cmd". * If VM administrator doesn't set value to "enable-custom-scripts", then by default this script is disabled for security purpose. * For VMware's vCD product , the preference is to enable the script if "enable-custom-scripts" is not set. vCD will generate a configuration file with "DEFAULT-RUN-POST-CUST-SCRIPT" set to true. This flag works for both VMware customization engine and cloud-init.
* cc_ca_certs.py: fix blank line problem when removing CAs and adding new one ↵dermotbradley2020-07-151-0/+22
| | | | | | | | | | | | | (#483) Problem: When cc_ca_certs configuration has both "remove-defaults: true" and also specifies one, or more, new trusted CAs to add then the resultant /etc/ca-certificates.conf file's 1st line is blank. As noted in comments in the existing cc_ca_certs.py code blank lines in this file cause problems. Fix: Before adding the cloud-init CA filename to this file first check the size of the file - if is is empty (as all existing CAs have been deleted) then write only the cloud-init CA filename to the file rather than appending it to the file.
* cloudinit: remove global disable of pylint W0107 and fix errors (#489)Daniel Watkins2020-07-156-13/+0
| | | | | | | | * cloudinit: remove global disable of pylint W0107 and fix errors This includes removing a test class which contained no tests but wasn't detected as empty because of an errant pass statement. * .pylintrc: update disable comment to match arguments
* cloudinit: remove global disable of pylint W0105 and fix errors (#480)Daniel Watkins2020-07-132-6/+7
| | | | This includes a fix to a test that had a string concatenation issue, and so was only testing a prefix of what was intended.
* Fix two minor warnings (#475)Daniel Watkins2020-07-131-1/+2
|
* test_data: fix faulty patch (#476)Daniel Watkins2020-07-101-1/+3
| | | | | | | I've been seeing intermittent failures of this test, and I tracked it down to something to do with`test_features.py`: running this test after `test_features.py` causes the failure, but the inverse does not. This fixed patch ensures that the test will pass regardless of ordering.
* cc_mounts: handle missing fstab (#484)Ryan Harper2020-07-101-0/+12
| | | | | | Do not fail if /etc/fstab is not present. Some images, like container rootfs may not include this file by default. LP: #1886531
* LXD cloud_tests: support more lxd image formats (#482)Paride Legovini2020-07-091-8/+25
| | | | | | | | Update lxd_export method to detect and handle different image formats cloud_tests will only support the "split" type images which exports a compressed (xz) tarball of metadata and a rootfs (of different formats). For non-split image formats (single tarball with metadata + rootfs) we now raise an exception indicating that the requested image is not supported at this time.
* tests: use markers to configure disable_subp_usage (#473)Daniel Watkins2020-07-022-3/+2
| | | | | | | | | This is an improvement over indirect parameterisation for a few reasons: * The test code is much easier to read, the mark names are much more intuitive than the indirect parameterisation invocation, and there's less boilerplate to boot * The fixture no longer has to overload the single parameter that fixtures can take with multiple meanings
* Disable ec2 mirror for non aws instances (#390)lucasmoura2020-06-304-81/+145
| | | | | For versions before 20.2, we allowed the use of ec2 mirrors if the datasource availability_zone matches one of the ec2 regions. We are now updating that behavior to allow allow the use of ec2 mirrors on ec2 instances or if the user directly passes an an ec2 mirror url through #cloud-config apt directives. LP: #1456277
* cloud_tests: don't pass --python-version to read-dependencies (#465)Daniel Watkins2020-06-301-1/+1
| | | We dropped that parameter in 4d26848
* networking: refactor is_physical from cloudinit.net (#457)Daniel Watkins2020-06-301-54/+95
| | | | | As the first refactor PR, this also includes the initial structure for tests. LP: #1884619
* RbxCloud: Add support for FreeBSD (#464)Adam Dobrawy2020-06-291-0/+16
| | | | | | | Changes are made that simplify code and aim to properly support FreeBSD: - use `util.find_devs_with` instead call directly `blkid`, because on FreeBSD is not supported well and `util.find_devs_with` have solution for FreeBSD for that - introduction of an additional name on FAT file system, which is used in FreeBSD - drop shell to use default value, because FreeBSD – by default – does not have `/bin/bash`
* Add schema for cc_chef module (#375)lucasmoura2020-06-291-1/+2
| | | | | | | Create a schema object for the chef module and validate this schema in the handle function of the module. Some of the config keys description, so I tried looking at the code and chef documentation to provide an information to the user. However, I don't know if I have the best description for all fields. For example, for the key show_time I could not find an accurate description of what it did, so I used what was in our code base to infer what it should do. LP: #1858888
* Hetzner: support reading user-data that is base64 encoded. (#448)Scott Moser2020-06-221-0/+23
| | | | | | | | | | | | | | | Hetzner cloud only supports user-data as a string (presumably utf-8). In order to allow users on Hetzner to provide binary data to cloud-init, we will attempt to base64decode the userdata. The change here adds a 'maybe_b64decode' function that will decode data if and only if is base64 encoded. The reason for not using util.b64d is that we do not want the return value decoded to a string, and util.b64d will do that if it can. Additionally we call decode with validate=True which oddly is not the default. LP: #1884071
* util: add ensure_dir_exists parameter to write_file (#443)Daniel Watkins2020-06-191-0/+11
| | | | This allows us to disable the `ensure_dir` call when it isn't appropriate.
* printing the error stream of the dhclient process before killing it (#369)Moustafa Moustafa2020-06-192-2/+2
| | | This introduces a way to log the dhclient error stream, and uses it for the Azure datasource (where we have a specific requirement for this data to be logged).
* util: rename write_file's copy_mode parameter to preserve_mode (#439)Daniel Watkins2020-06-171-6/+6
| | | | | When updating the docstring to include it, I realised that the current name is somewhat misleading; this makes it a little easier to understand, I think.
* test: fix all flake8 E126 errors (#425)Joshua Powers2020-06-107-45/+73
|
* Fixes KeyError for bridge with no "parameters:" setting (#423)Brian Candler2020-06-091-0/+10
| | | | | Reason: commit ded1ec8 introduced a regression whereby a bridge with no "parameters:" setting caused a KeyError exception. LP: #1879673
* test: move conftest.py to top-level, to cover tests/ also (#414)Daniel Watkins2020-06-082-2/+7
| | | | | | | | | | | | | | * test_opennebula: convert TestParseShellConfig to a pytest test And allow it to run bash. (We aren't aiming to convert TestCase tests to pytest tests as a rule. In this case, I needed to change its implementation to limit subp usage, and I chose pytest over CiTestCase.) * test: move conftest.py to top-level, to cover tests/ also This gives us a single conftest.py which is shared by all tests in the project.
* Move subp into its own module. (#416)Scott Moser2020-06-0849-466/+272
| | | | | | | | | | | | | | | | | This was painful, but it finishes a TODO from cloudinit/subp.py. It moves the following from util to subp: ProcessExecutionError subp which target_path I moved subp_blob_in_tempfile into cc_chef, which is its only caller. That saved us from having to deal with it using write_file and temp_utils from subp (which does not import any cloudinit things now). It is arguable that 'target_path' could be moved to a 'path_utils' or something, but in order to use it from subp and also from utils, we had to get it out of utils.
* New feature flag functionality and fix includes failing silently (#367)James Falcon2020-06-041-0/+25
| | | | | | | | | | | | | Build time feature flags are now defined in cloudinit/features.py. Feature flags can be added to toggle configuration options or deprecated features. Feature flag overrides can be placed in cloudinit/feature_overrides.py. Further documentation can be found in HACKING.rst. Additionally, updated default behavior to exit with an exception if #include can't retrieve resources as expected. This behavior can be toggled with a feature flag. LP: #1734939
View Lorry Controller Status