diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-08-19 09:08:42 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-08-19 09:08:42 +0000 |
commit | b76ae638462ab0f673e5915986070518dd3f9ad3 (patch) | |
tree | bdab0533383b52873be0ec0eb4d3c66598ff8b91 /doc/development/testing_guide | |
parent | 434373eabe7b4be9593d18a585fb763f1e5f1a6f (diff) | |
download | gitlab-ce-b76ae638462ab0f673e5915986070518dd3f9ad3.tar.gz |
Add latest changes from gitlab-org/gitlab@14-2-stable-eev14.2.0-rc42
Diffstat (limited to 'doc/development/testing_guide')
8 files changed, 217 insertions, 21 deletions
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index e153fa9f334..ba7312b760f 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -911,14 +911,16 @@ describe '#show', :snowplow do expect_snowplow_event( category: 'Experiment', action: 'start', - standard_context: { namespace: group, project: project } + namespace: group, + project: project ) expect_snowplow_event( category: 'Experiment', action: 'sent', property: 'property', label: 'label', - standard_context: { namespace: group, project: project } + namespace: group, + project: project ) end end @@ -972,11 +974,16 @@ range of inputs, might look like this: describe "#==" do using RSpec::Parameterized::TableSyntax + let(:one) { 1 } + let(:two) { 2 } + where(:a, :b, :result) do - 1 | 1 | true - 1 | 2 | false - true | true | true - true | false | false + 1 | 1 | true + 1 | 2 | false + true | true | true + true | false | false + ref(:one) | ref(:one) | true # let variables must be referenced using `ref` + ref(:one) | ref(:two) | false end with_them do @@ -989,11 +996,13 @@ describe "#==" do end ``` +<!-- vale gitlab.Spelling = NO --> + WARNING: -Only use simple values as input in the `where` block. Using -<!-- vale gitlab.Spelling = NO --> procs, stateful +Only use simple values as input in the `where` block. Using procs, stateful objects, FactoryBot-created objects, and similar items can lead to [unexpected results](https://github.com/tomykaira/rspec-parameterized/issues/8). + <!-- vale gitlab.Spelling = YES --> ### Prometheus tests diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index 15520d8a6b1..74c02d19d0a 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -338,6 +338,16 @@ Page::Project::Pipeline::Show.perform do |pipeline| end ``` +### Use `eventually_` matchers for expectations that require waiting + +When something requires waiting to be matched, use `eventually_` matchers with clear wait duration definition. + +`Eventually` matchers use the following naming pattern: `eventually_${rspec_matcher_name}`. They are defined in [eventually_matcher.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/spec/support/matchers/eventually_matcher.rb). + +```ruby +expect { async_value }.to eventually_eq(value).within(max_duration: 120, max_attempts: 60, reload_page: page) +``` + ### Create negatable matchers to speed `expect` checks However, sometimes we want to check that something is _not_ as we _don't_ want it to be. In other diff --git a/doc/development/testing_guide/end_to_end/environment_selection.md b/doc/development/testing_guide/end_to_end/environment_selection.md deleted file mode 100644 index 2192d9c4ed4..00000000000 --- a/doc/development/testing_guide/end_to_end/environment_selection.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'execution_context_selection.md' ---- - -This file was moved to [another location](execution_context_selection.md). - -<!-- This redirect file can be deleted after <2021-08-14>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index eca649b73a5..f4b01c64385 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -91,7 +91,7 @@ subgraph "`gitlab-org/gitlab-qa-mirror` pipeline" 1. The result of the [`gitlab-org/gitlab-qa-mirror` pipeline](https://gitlab.com/gitlab-org/gitlab-qa-mirror) is being propagated upstream (through polling from upstream pipelines), through [`gitlab-org/build/omnibus-gitlab-mirror`](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror), back to the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) merge request. -Please note, we plan to [add more specific information](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/156) +We plan to [add more specific information](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/156) about the tests included in each job/scenario that runs in `gitlab-org/gitlab-qa-mirror`. NOTE: diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md index 7f541f1be3f..3a016c0e95c 100644 --- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md +++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md @@ -16,7 +16,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec | `:elasticsearch` | The test requires an Elasticsearch service. It is used by the [instance-level scenario](https://gitlab.com/gitlab-org/gitlab-qa#definitions) [`Test::Integration::Elasticsearch`](https://gitlab.com/gitlab-org/gitlab/-/blob/72b62b51bdf513e2936301cb6c7c91ec27c35b4d/qa/qa/ee/scenario/test/integration/elasticsearch.rb) to include only tests that require Elasticsearch. | | `:except` | The test is to be run in their typical execution contexts _except_ as specified. See [test execution context selection](execution_context_selection.md) for more information. | | `:geo` | The test requires two GitLab Geo instances - a primary and a secondary - to be spun up. | -| `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements-for-configuring-a-gitaly-cluster). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. | +| `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. | | `:github` | The test requires a GitHub personal access token. | | `:group_saml` | The test requires a GitLab instance that has SAML SSO enabled at the group level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. | | `:instance_saml` | The test requires a GitLab instance that has SAML SSO enabled at the instance level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. | diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md index f200d6c682a..46a3053c267 100644 --- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md +++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md @@ -432,7 +432,7 @@ To run the LDAP tests on your local with TLS enabled, follow these steps: `127.0.0.1 gitlab.test` - You can then run tests against GitLab in a Docker container on `https://gitlab.test`. Please note that the TLS certificate [checked into the GitLab-QA repository](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab) is configured for this domain. + You can then run tests against GitLab in a Docker container on `https://gitlab.test`. The TLS certificate [checked into the GitLab-QA repository](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/tls_certificates/gitlab) is configured for this domain. 1. Run the OpenLDAP container with TLS enabled. Change the path to [`gitlab-qa/fixtures/ldap`](https://gitlab.com/gitlab-org/gitlab-qa/-/tree/9ffb9ad3be847a9054967d792d6772a74220fb42/fixtures/ldap) directory to your local checkout path: ```shell diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index d8f3a18577f..3af806d8f57 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -51,7 +51,7 @@ which have to be stubbed. ### Differences to Karma -- Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/26982). +- Jest runs in a Node.js environment, not in a browser. [An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/26982) for running Jest tests in a browser. - Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. See also its [limitations](#limitations-of-jsdom) below. - Jest does not have access to Webpack loaders or aliases. The aliases used by Jest are defined in its [own configuration](https://gitlab.com/gitlab-org/gitlab/-/blob/master/jest.config.js). @@ -423,6 +423,55 @@ it('does something', () => { }); ``` +### Mocking the current location in Jest + +NOTE: +The value of `window.location.href` is reset before every test to avoid earlier +tests affecting later ones. + +If your tests require `window.location.href` to take a particular value, use +the `setWindowLocation` helper: + +```javascript +import setWindowLocation from 'helpers/set_window_location'; + +it('passes', () => { + setWindowLocation('https://gitlab.test/foo?bar=true'); + + expect(window.location).toMatchObject({ + hostname: 'gitlab.test', + pathname: '/foo', + search: '?bar=true', + }); +}); +``` + +To modify only the hash, use either the `setWindowLocation` helper, or assign +directly to `window.location.hash`, e.g.: + +```javascript +it('passes', () => { + window.location.hash = '#foo'; + + expect(window.location.href).toBe('http://test.host/#foo'); +}); +``` + +If your tests need to assert that certain `window.location` methods were +called, use the `useMockLocationHelper` helper: + +```javascript +import { useMockLocationHelper } from 'helpers/mock_window_location_helper'; + +useMockLocationHelper(); + +it('passes', () => { + window.location.reload(); + + expect(window.location.reload).toHaveBeenCalled(); +}); +``` + ### Waiting in tests Sometimes a test needs to wait for something to happen in the application before it continues. diff --git a/doc/development/testing_guide/testing_migrations_guide.md b/doc/development/testing_guide/testing_migrations_guide.md index d54ca0d3c64..757a70fb4e0 100644 --- a/doc/development/testing_guide/testing_migrations_guide.md +++ b/doc/development/testing_guide/testing_migrations_guide.md @@ -127,7 +127,90 @@ reversible_migration do |migration| end ``` -### Example database migration test +### Custom matchers for post-deployment migrations + +We have some custom matchers in +[`spec/support/matchers/background_migrations_matchers.rb`](https://gitlab.com/gitlab-org/gitlab/blob/v14.1.0-ee/spec/support/matchers/background_migrations_matchers.rb) +to verify background migrations were correctly scheduled from a post-deployment migration, and +receive the correct number of arguments. + +All of them use the internal matcher `be_background_migration_with_arguments`, which verifies that +the `#perform` method on your migration class doesn't crash when receiving the provided arguments. + +#### `be_scheduled_migration` + +Verifies that a Sidekiq job was queued with the expected class and arguments. + +This matcher usually makes sense if you're queueing jobs manually, rather than going through our helpers. + +```ruby +# Migration +BackgroundMigrationWorker.perform_async('MigrationClass', args) + +# Spec +expect('MigrationClass').to be_scheduled_migration(*args) +``` + +#### `be_scheduled_migration_with_multiple_args` + +Verifies that a Sidekiq job was queued with the expected class and arguments. + +This works the same as `be_scheduled_migration`, except that the order is ignored when comparing +array arguments. + +```ruby +# Migration +BackgroundMigrationWorker.perform_async('MigrationClass', ['foo', [3, 2, 1]]) + +# Spec +expect('MigrationClass').to be_scheduled_migration_with_multiple_args('foo', [1, 2, 3]) +``` + +#### `be_scheduled_delayed_migration` + +Verifies that a Sidekiq job was queued with the expected delay, class, and arguments. + +This can also be used with `queue_background_migration_jobs_by_range_at_intervals` and related helpers. + +```ruby +# Migration +BackgroundMigrationWorker.perform_in(delay, 'MigrationClass', args) + +# Spec +expect('MigrationClass').to be_scheduled_delayed_migration(delay, *args) +``` + +#### `have_scheduled_batched_migration` + +Verifies that a `BatchedMigration` record was created with the expected class and arguments. + +The `*args` are additional arguments passed to the `MigrationClass`, while `**kwargs` are any other +attributes to be verified on the `BatchedMigration` record (Example: `interval: 2.minutes`). + +```ruby +# Migration +queue_batched_background_migration( + 'MigrationClass', + table_name, + column_name, + *args, + **kwargs +) + +# Spec +expect('MigrationClass').to have_scheduled_batched_migration( + table_name: table_name, + column_name: column_name, + job_arguments: args, + **kwargs +) +``` + +### Examples of migration tests + +Migration tests depend on what the migration does exactly, the most common types are data migrations and scheduling background migrations. + +#### Example of a data migration test This spec tests the [`db/post_migrate/20170526185842_migrate_pipeline_stages.rb`](https://gitlab.com/gitlab-org/gitlab-foss/blob/v11.6.5/db/post_migrate/20170526185842_migrate_pipeline_stages.rb) @@ -181,6 +264,59 @@ RSpec.describe MigratePipelineStages do end ``` +#### Example of a background migration scheduling test + +To test these you usually have to: + +- Create some records. +- Run the migration. +- Verify that the expected jobs were scheduled, with the correct set + of records, the correct batch size, interval, etc. + +The behavior of the background migration itself needs to be verified in a [separate +test for the background migration class](#example-background-migration-test). + +This spec tests the +[`db/post_migrate/20210701111909_backfill_issues_upvotes_count.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/v14.1.0-ee/db/post_migrate/20210701111909_backfill_issues_upvotes_count.rb) +post-deployment migration. You can find the complete spec in +[`spec/migrations/backfill_issues_upvotes_count_spec.rb`](https://gitlab.com/gitlab-org/gitlab/blob/v14.1.0-ee/spec/spec/migrations/backfill_issues_upvotes_count_spec.rb). + +```ruby +require 'spec_helper' +require_migration! + +RSpec.describe BackfillIssuesUpvotesCount do + let(:migration) { described_class.new } + let(:issues) { table(:issues) } + let(:award_emoji) { table(:award_emoji) } + + let!(:issue1) { issues.create! } + let!(:issue2) { issues.create! } + let!(:issue3) { issues.create! } + let!(:issue4) { issues.create! } + let!(:issue4_without_thumbsup) { issues.create! } + + let!(:award_emoji1) { award_emoji.create!( name: 'thumbsup', awardable_type: 'Issue', awardable_id: issue1.id) } + let!(:award_emoji2) { award_emoji.create!( name: 'thumbsup', awardable_type: 'Issue', awardable_id: issue2.id) } + let!(:award_emoji3) { award_emoji.create!( name: 'thumbsup', awardable_type: 'Issue', awardable_id: issue3.id) } + let!(:award_emoji4) { award_emoji.create!( name: 'thumbsup', awardable_type: 'Issue', awardable_id: issue4.id) } + + it 'correctly schedules background migrations', :aggregate_failures do + stub_const("#{described_class.name}::BATCH_SIZE", 2) + + Sidekiq::Testing.fake! do + freeze_time do + migrate! + + expect(described_class::MIGRATION).to be_scheduled_migration(issue1.id, issue2.id) + expect(described_class::MIGRATION).to be_scheduled_migration(issue3.id, issue4.id) + expect(BackgroundMigrationWorker.jobs.size).to eq(2) + end + end + end +end +``` + ## Testing a non-`ActiveRecord::Migration` class To test a non-`ActiveRecord::Migration` test (a background migration), |