summaryrefslogtreecommitdiff
path: root/tempest/scenario
Commit message (Collapse)AuthorAgeFilesLines
* Allow passing validation_resources in scenarioDan Smith2023-05-041-4/+8
| | | | | | | | | | | | In the scenario manager, we need to pass through validation parameters if they are passed to us. If not, we need to do the needful for SSHABLE targets, but some tests need to pass in their own. Note that the scenario manager create_server() already has a validatable parameter, but it does *not* pass that through to create_test_server(), so we also need to fix that up for those cases to avoid tripping over the assertion requirement that it be provided. Change-Id: I44b3deba70e3f33f1287a6b3f28c21da5492cc04
* Allow wait_until=SSHABLE in scenario managerDan Smith2023-05-024-8/+58
| | | | | | | | | | | | | If we can, we should always wait for servers to be ssh'able before we try to do things to them, especially attach volumes. This improves stability of the tests, at the expense of performance. This also makes some volume-having scenario tests wait for SSHABLE. The ones that don't are already using ssh to do things like create and check timestamps, so they need not be changed to still have this behavior. Change-Id: I169d8d119758e4f5e79c42cfd458577a67b13fc5
* Merge "Introduce a change that will allow more clients to make volumes"Zuul2023-03-131-14/+28
|\
| * Introduce a change that will allow more clients to make volumesMilana Levy2023-03-131-14/+28
| | | | | | | | | | | | | | This change was written so that a new volume could be created by another client other than the primary admin. This change also let this clients to attach/detach volumes. Change-Id: I122a08aaf273a1f786e13737f31f2a0ab9c7bfdb
* | Use assertLessEqual for downtime validationyatinkarel2023-02-171-1/+1
|/ | | | | | | | | | | | | | | | The test was added as part of [1]. It should succeed when downtime is within the limit but it fails as below even if downtime is not higher than the allowed limit:- AssertionError: 5.0 not less than 5 : Downtime \ of 5.0 seconds is higher than expected '5' Fix it by using assertLessEqual so it fails only if downtime is higher than the allowed limit. [1] https://review.opendev.org/c/openstack/tempest/+/828686 Change-Id: I1bf87bd33aecf004d9cb650110f06454584f600c
* Minimize the tests footprint in multinode jobGhanshyam Mann2023-02-115-9/+14
| | | | | | | | | | | | multinode job run all the tests including multinode and non multinode tests. But we do not need to run all the non multinode tests in this job instead smoke tests along with multinode tests should be enough to run. This make multinode jobs to run only smoke and multinode tests. For that, we need to tag the multinode tests with 'multinode' attr. Relavant-Bug: #2004780 Change-Id: I7e87d1db3ef3a00b3d27f0904d0af6a270e03837
* Mark test_minimum_basic_instance_hard_reboot_after_vol_snap_deletion as slow ↵Ghanshyam Mann2023-02-031-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | test We are seeing more timeout in tests (and so does sometimes it end up to job timeout) in various jobs. A few examples are - miltinode job - https://zuul.openstack.org/build/084729c5a1fe46e091a17a50441e99d5/log/job-output.txt - stable/yoga job - https://zuul.opendev.org/t/openstack/build/9b79f54d3f6c4e05ae6619fdac7ad95a Two tests in doubt and taking more time is test_minimum_basic_instance_hard_reboot_after_vol_snap_deletion - taking 258.358054s - https://zuul.openstack.org/build/084729c5a1fe46e091a17a50441e99d5/log/job-output.txt#34173 - taking 187.411266s - https://zuul.opendev.org/t/openstack/build/9b79f54d3f6c4e05ae6619fdac7ad95a/log/job-output.txt#25738 test_minimum_basic_scenario - taking 309.109043s - https://zuul.opendev.org/t/openstack/build/d068cb494d234fe7b79dc5ae6fd6ae69/log/job-output.txt#24052 marking these test as slot test and monitor if there is another slow test we can find. Related-Bug: #2004780 Change-Id: I0aff3507b3bf3498ab0ecd548bb57cdcd97ec11a
* Merge "Restore IP addresses configuration after spoofing MAC address"Zuul2023-01-251-2/+9
|\
| * Restore IP addresses configuration after spoofing MAC addressSlawek Kaplonski2023-01-231-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the test_port_security_macspoofing_port test, NIC on one of the vms is set to DOWN, MAC of this NIC is changed to the spoofed one and NIC is then bring back to UP. For some reason it works fine in Cirros 0.5.x but not in 0.6.0 and newer as after bringinig interface back to be UP there is also need to restore configured previously IP addresses. This patch adds check of IPs configured on that NIC before it is switch to DOWN and later restores the same IPs configuration when NIC is UP again. Related-Bug: #2003063 Change-Id: I05d2118125195a387163ad1f0177fd9dfc916238
* | Introduce @serial test execution decoratorBalazs Gibizer2023-01-181-142/+0
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tempest provides a LockFixture to avoid two potentially interfering tests to run in parallel. However, this solution does not scale when we want to separate a set of tests from many other test cases. For example, host aggregate and availability zone testing needs compute hosts without any nova servers to be able to test moving computes between aggregates but a lot of other tests are creating nova servers. To fully separate these aggregate tests from the rest of the tempest test cases, this patch proposes a @serial class decorator to mark a test class to be run totally independently of any other test classes. Under the hood, the @serial decorator is implemented with a tempest-wide interprocess read-write lock. The serial test classes always take the write lock, while the non-serial classes take the read lock. The lock allows in many readers OR a single writer. So the serial tests are run independently from the rest. To minimize the time a serial test blocks other tempest tests run in parallel, this patch also introduced a serial_tests test directory to store the serial tests. The current test ordering in a fresh env uses alphabetical order so the serial tests will run at the end of the execution not randomly in the middle. The gate uses fresh VMs for every run so we can rely on this optimization there. In local envs where tests are re-run, the subsequent runs will be ordered at runtime by stestr. Therfore, a longer runtime might be observed due to locking, but the correctness of the test execution is still kept. Related-Bug: #821732 Change-Id: I0181517edab75f586464a38c4811417f888783b1
* Added functionality to wait for port creationEliad Cohen2022-11-281-3/+6
| | | | | | | | | Added a waiter that compares the current status of the port to a target value and returns the port when that is achieved. Closes-Bug: 1845486 Change-Id: Ibc5434244f3758c60ddefa96d78476948a47ed13
* Clean up scenario floating IP waitingArtom Lifshitz2022-07-271-41/+0
| | | | | | | | | | | | | In changes I0f7e1c9096dc1903903fb31c5e854f07800efbfd and I05e079237aec332925a88476e45071b34dc722c6 we started waiting for a floating IP to appear in the Nova 'show server details' API before continuing with the test. This left some useless test code left over that needlessly asserted that the floating IP was showing up (or had disappeared, in the case of disassociation) from the Nova 'show server details' API after the waiter had already checked the same thing. This patch cleans this up. Change-Id: I692402cf0d831c524ca5f49daf4aa9b0d19d41b5
* Merge "Add LUKS v2 tests"Zuul2022-07-252-8/+37
|\
| * Add LUKS v2 testsGorka Eguileor2022-06-292-8/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We currently have cryptsetup and LUKS v1 tests, but we are missing LUKS v2 tests. Code paths for LUKS v1 and LUKS v2 are different, because Nova uses libvirt to decode LUKS v1 volumes but os-brick to decode LUKS v2 volumes. This means that not only the attach process is different, but also some features such as extend volume, so one of them may work while the other doesn't. This patch adds LUKS v2 encryption tests for all backends except for RBD volumes, because they are not connected to the host but directly through libvirt, so they can only use LUKS v1. Depends-On: Ia1411f11ec4bf44af6a42d5f96c8a0903846ed66 Depends-On: I351f1a7769c9f915e4cd280f05a8b8b87f40df84 Related-Bug: #1967157 Change-Id: I40c518c3a3e760867efd6b6338fa9c16945b28c4
* | Merge "Wait for floating IP assocation in test_minimum_basic test"Zuul2022-07-251-0/+2
|\ \
| * | Wait for floating IP assocation in test_minimum_basic testArtom Lifshitz2022-07-251-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In bug 1923194 Tempest wasn't giving Nova a change to update its network info cache after attaching a floating IP, leading to failures when the floating IP association was asserted via the Nova API. The fix was to add a waiter before doing the assertion. We were not using that waiter in the scenario floating IP association code, leading to bug 1980255. This patch starts doing that. Related-bug: 1980255 Change-Id: I05e079237aec332925a88476e45071b34dc722c6
* | | Remove unstable_test decoratorRoman Popelka2022-07-131-1/+0
|/ / | | | | | | | | | | | | | | | | | | Removes unstable_test decorator from test_server_connectivity_cold_migration_revert test since the related bug seems to be gone and the test passes in most of the jobs. Closes-Bug: 1836595 Change-Id: Ifc691971c71d89615d0e789cf6df884d2224047d
* | Fix create_server_snapshot for microversionBenny Kopilov2022-06-301-1/+3
|/ | | | | | | | | Location returned till microversion 2.44 image_id returned from microversion 2.45 Added support in manager for both Change-Id: I5ef9233d9ba2437198bac1357782a9639024dc79
* Merge "Add another scenario testcase test_minimum_basic"Zuul2022-06-242-18/+150
|\
| * Add another scenario testcase test_minimum_basicBenny Kopilov2021-12-132-18/+150
| | | | | | | | | | | | | | | | | | | | | | | | | | Current test used as a basic check for multiple services - Changed the nova reboot to HARD in order to refresh connection for attahced devices - Created a volume snapshot and delete it before reboot instance - Write data to the attached volume and read post instance reboot. Related-Bug: #1860913 Change-Id: I4151715954b3a12e6c790836d4b7bd50c0407720
* | Merge "Validate network downtime during live migration"Zuul2022-06-011-0/+19
|\ \
| * | Validate network downtime during live migrationRoman Safronov2022-05-191-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the ability to measure network downtime during live migration process. A fixture has been added to start and stop a background pinger process and also read status from it. The downtime measurement has 0.2 seconds granularity. In order to reduce overall traffic the ping payload size set to minimal value. Change-Id: I83c6a5d49f5d4da05deb677907e5048ecdd2242b
* | | Add verification when restoring backupsSofia Enriquez2022-05-261-2/+6
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | If you want to restore a backup, specifying only the backup ID, you will get a new volume. However, if you pass the volume_id argument, you are targeting the restore of an existing volume. The problem I am having is that restore_backup() adds a second call to addCleanup to an existing volume and this results in a 400 HTTP error. To fix this problem, I add a simple check when we want to use an existing volume without calling cleanup. Change-Id: I521f69091cdad60edfe4ed85368df5decc0a7f55
* | Allows to skip wait for volume createBenny Kopilov2022-04-131-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current code does not support skipping wait for volume, tempest plugins require test without waiting and still use all existing flow code. Examples: Create multiple volume in a batch without waiting Create multiple volumes and expecting for error state Instead of duplicating code its better to set a flag, wait_until default value is available, when its None we skip waiting inside create volume Change-Id: I7fe90e26f773b3a128e3d1bff1b89a3ef665eebb
* | Allow rulesets modification in secgroupsRoman Popelka2022-04-011-18/+21
| | | | | | | | | | | | | | | | | | | | | | As part of tempest-scenario-manager-cleanup[1] effort it would be good to allow plugins to define their own security group rulesets as they're not always the same as tempest's (f.e. manila tempest plugin[2]) [1]https://etherpad.opendev.org/p/tempest-scenario-manager-cleanup [2]https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/tests/scenario/manager.py#L1001 Change-Id: I3cb3a6f67ea507d9e721a31fcba78f8344c630ca
* | Fix compute_unified decorationBenny Kopilov2022-03-171-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Current test class decorate the class with skip. The problem is that when feature is disabled the skip section returns a function type. The ServersQuotaTest class inherits from a function type instead type (class) and it fails with TypeError: function() argument We dont see the exception because on false we dont try to run testtool version is 2.5.0 Change-Id: I3ff2e59ca1eaae4f9b1eb1fb7148cd3efcbcd8f2
* | Merge "Tests for nova unified quotas"Zuul2022-03-051-0/+166
|\ \
| * | Tests for nova unified quotasmelanie witt2022-03-041-0/+166
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a sequence of unified limit updates and verifications of enforcement for servers, vcpu, ram, and disk. This is safe to run in parallel with other tests because only project scoped limits are being tested in this scenario. Related to blueprint unified-limits-nova Change-Id: I37d3896a037e2d4d1004abc52f6e93fd0025f981
* | | Fix TypeError in QoSBandwidthAndPacketRateTestsyatinkarel2022-02-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [1] Added random int to flavor name but that resulted into TypeError: can only concatenate str (not "int") to str. Concatenate using % operator to fix it. Also make tempest-full-ussuri-py3 job non-voting temporary to unblock gate, can be reverted once [2] is fixed. [1] https://review.opendev.org/c/openstack/tempest/+/826946 [2] https://launchpad.net/bugs/1959600 Related-Bug: #1959467 Change-Id: I76e15b6b52bb44fec550ead510186c644c4a1c7f
* | | Append random int to new flavor name in QoSBandwidthAndPacketRateTestsmelanie witt2022-01-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In a couple of the tests a new flavor is created by appending the string 'extra' to the existing flavor name. The existing flavor name is however CONF.compute.flavor_ref, which isn't unique. If two tests that create the flavor run at the same time, there will be a 409 collision and one test will fail. This appends a random int to the new flavor name to prevent conflicts. Closes-Bug: #1959467 Change-Id: I9928e2be9c749f69eea9a9d4f9e8fe03ef4ef612
* | | Add test coverage for bug 1943724Balazs Gibizer2022-01-141-14/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that bug 1943724 is fixed in neutron we can extend the tempest test coverage to show that changing the direction of the QoS minimum guaranteed bandwidth rule on a bound port is supported. Related-Bug: #1943724 Change-Id: I09e773baedcffc8af5a88e0be1df38d14633ed74
* | | Merge "Test Network QoS minimum packet rate scenarios"Zuul2022-01-121-0/+498
|\ \ \ | |_|/ |/| |
| * | Test Network QoS minimum packet rate scenariosBalazs Gibizer2021-11-291-0/+498
| | | | | | | | | | | | | | | | | | | | | | | | | | | Depends-On: https://review.opendev.org/c/openstack/neutron/+/805391 Depens-On: https://review.opendev.org/c/openstack/nova/+/811396 blueprint: qos-minimum-guaranteed-packet-rate Change-Id: I4cb12a23f33d4feef6c800d23a0501d1aebb4f17
* | | Moving API microversion fixture in resource_setupGhanshyam Mann2021-12-111-7/+4
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | API microversion fixture is used to set the API microversion on service client and reset it back after test execution. That is done in setUp() method and due to that API microversion is not set on service client during resource_setup() method and any API call will be requested on base version. To sync the complete test execution of test class with requested API microversion, this commit moves the API microversion setting in resource_setup() method. Closes-Bug: #1881863 Change-Id: Ia4b884a79cc0e50057dd57090737332bd76da37e
* | Merge "Add floating IP waiter"Zuul2021-11-231-32/+5
|\ \
| * | Add floating IP waiterArtom Lifshitz2021-10-141-32/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In bug 1923194, a floating IP is associated to a server. Upon performing the operation, Neutron sends Nova a network-vif-changed external event, causing Nova to update its network info cache. Until Nova does this, the new floating IP is not reflected in `server show`. Tempest's `server show` is racing with this process, causing intermittent failures when the new floating IP does not show up in `server show` in time. This patch adds a new waiter that waits for a floating IP to either appear to disappear in the `server show` output, and modifies two tests to use the new helper. Closes bug: 1923194 Change-Id: I0f7e1c9096dc1903903fb31c5e854f07800efbfd
* | | Consolidate client usage in MinBwAllocationPlacementTestBalazs Gibizer2021-11-021-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | Let's only use admin client for the operations that are really needing it. blueprint: qos-minimum-guaranteed-packet-rate Change-Id: I4cc9eb76a1318c17a55d8d10d21290eac4690548
* | | Refactor existing QoS min bw testsBalazs Gibizer2021-11-021-25/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As a preparation to test QoS minimum bandwidth and minimum packet rate together this patch refactors the existing MinBwAllocationPlacementTest test class to allow generalization. blueprint: qos-minimum-guaranteed-packet-rate Change-Id: Ie589e114d08cc8bb5a524d6b242f1b29fda43983
* | | Prepare MinBwAllocationPlacementTest for new neutron API extBalazs Gibizer2021-10-271-3/+14
|/ / | | | | | | | | | | | | | | | | | | The port-resource-request-groups neutron API extension proposed in I99a49b107b1872ddf83d1d8497a26a8d728feb07 changes the format of the port.binding:profile.allocation value. The existing tests are depending on the format so they are adapted now to accept both format. blueprint: qos-minimum-guaranteed-packet-rate Change-Id: I7c5ecfe1015dba4ad0b70bbdcacc079503c50e65
* | Merge "Revert "Skip test_qos_min_bw_allocation_basic when not supported""Zuul2021-09-181-5/+2
|\ \
| * | Revert "Skip test_qos_min_bw_allocation_basic when not supported"Balazs Gibizer2021-09-141-5/+2
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 0976ae4ee20c3faf7b2add3893ec3f8cbfa55db1. That commit introduced a skip of MinBwAllocationPlacementTest tests if no bandwidth allocation is possible to avoid failing these tests in OVN jobs. However that is a wrong solutions as it would skip the test also in OVS jobs if we regress the bandwidth inventory reporting in neutron (or regress allocation candidate handling in placement). A better fix is not to enable the tempest test flag on OVN jobs: $TEMPEST_CONFIG: network-feature-enabled: qos_placement_physnet The existing tests already skipped if this is not configured. This also means that the generic tempest-* job definitions should not configure the above flag as those tempest jobs will run by default with OVN. So they are cleaned up along with the OVS specific neutron configuration. This means that jobs that was inherit from tempest-multinode-full-py3 and tempest-full-py3 and reconfigre the job to run with OVS instead of the default OVN needs to change to configure the OVS specific network config and enable the qos tests. This will be done in project specific patches. The stable tempest jobs are OK as they are still running with OVS by default. Depends-On: https://review.opendev.org/c/openstack/neutron/+/806170 Change-Id: I13fd2dc023b806ac74ab5c8d86cc99f807f0f702
* | [OVN] Remove OVN metadata ports from network port listRodolfo Alonso Hernandez2021-09-091-2/+5
|/ | | | | | | | | | | | | | | In "_check_network_internal_connectivity", all "network:" and "compute:" ports belonging to a network are retrieved. OVN metadata ports "network:distributed" should be filtered out because are not a DHCP nor L3 ports. This port type was introduced in Neutron in [1][2]. [1]https://review.opendev.org/c/openstack/neutron-lib/+/732141 [2]https://review.opendev.org/c/openstack/neutron/+/732364 Partial-Bug: #1942913 Change-Id: I7db94247c55f9ac39e3e30a4195cdcda6ed657d7
* Add tests for glance unified quotasDan Smith2021-07-161-0/+435
| | | | | | | Related to blueprint glance-unified-quotas Depends-On: https://review.opendev.org/c/openstack/devstack/+/788056 Change-Id: I08ccf5627ba98173507791c10fef0e7917880bc3
* Fix unstability in test_subnet_detailsccamposr2021-03-291-0/+7
| | | | | | | | | | | | | | | | | | | | | when the image has some dns preconfigured in /etc/resolv.conf, for example: resolv.conf nameserver 192.168.122.1 the test failed due to: testtools.matchers._impl.MismatchError: {'1.2.3.4'} != {'1.2.3.4', '192.168.122.1'}: Looking for servers: ['1.2.3.4']. Retrieved DNS nameservers: ['192.168.122.1', '1.2.3.4'] From host: 10.0.0.234 DHCP renewal is forced in the server that will cleanup the file /etc/resolv.conf from any other preconfigured DNS entry. When the DHCP server pushes the new configuration, the Neutron network DNS entries are applied Closes-Bug: #1914229 Change-Id: I9bd8b47642891a1fe42c48a0b9fe50cf9bc4e03b
* Move flavor client in cls.setup_compute_client()Ghanshyam Mann2021-03-251-2/+2
| | | | | | | | flavor client is compute service client so moving this to other compute clients. Implements: blueprint tempest-scenario-manager-stable Change-Id: Icdbdbd388858c6accbd4f241c9ee404797c2f0e3
* Rename create_networks to correct name as per its functionalityGhanshyam Mann2021-03-255-8/+10
| | | | | | | | | | | | | create_networks() method create network, subnet, router and connect it to router but its name is sounds like it will create more than one network. This also confusing with existing create_network() method (without 's'). Renaming this to setup_network_subnet_with_router() which is what this function does. Implements: blueprint tempest-scenario-manager-stable Change-Id: I63b3991210fd86bcc830af5beed28324797c3209
* Move floating IP operation from nova-network to neutronGhanshyam Mann2021-03-254-124/+109
| | | | | | | | | | | | | | | | | | nova-network is no more available in nova and all network request needs to move to neutron. Scenario tests are something we should be testing with the latest interface not the deprecated one so we can remove the nova netowrk specific service clients and methods. This commit moves the floating ip operation from nova-network to neutron. nova network create_floating_ip used to associate the FIP to server also so with neutron create floating ip we need to associate the FIP to server in separate method. Implements: blueprint tempest-scenario-manager-stable Change-Id: I6ad260731eedc2e1f56fe67b3a9e19fed8d4a38e
* Remove nova-network from scenario testsSoniya Vyas2021-03-231-219/+150
| | | | | | | | | | | | | | | nova-network is no more available in nova and all network request needs to move to neutron. Scenario tests are something we should be testing with the latest interface not the deprecated one so we can remove the nova netowrk specific service clients and methods. Implements: blueprint tempest-scenario-manager-stable Signed-off by: Soniya Vyas<svyas@redhat.com> Co-Authored-By: Ghanshyam Mann <gmann@ghanshyammann.com> Change-Id: I3cc88cf0a8728448fa9bd05c68784553660e07f8
* Merge "Remove deprecated scenario img_dir option"Zuul2021-03-221-7/+2
|\
| * Remove deprecated scenario img_dir optionMartin Kopec2021-01-251-7/+2
| | | | | | | | | | | | | | | | It's time to remove deprecated CONF.scenario.img_dir which got deprecated in the 25.0.0 tag. Change-Id: I86ad219265a67a1c016e8efe04677993a0437ede Closes-Bug: #1393881
View Lorry Controller Status