summaryrefslogtreecommitdiff
path: root/doc/development/testing_guide
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2022-05-19 07:33:21 +0000
committerGitLab Bot <gitlab-bot@gitlab.com>2022-05-19 07:33:21 +0000
commit36a59d088eca61b834191dacea009677a96c052f (patch)
treee4f33972dab5d8ef79e3944a9f403035fceea43f /doc/development/testing_guide
parenta1761f15ec2cae7c7f7bbda39a75494add0dfd6f (diff)
downloadgitlab-ce-36a59d088eca61b834191dacea009677a96c052f.tar.gz
Add latest changes from gitlab-org/gitlab@15-0-stable-eev15.0.0-rc42
Diffstat (limited to 'doc/development/testing_guide')
-rw-r--r--doc/development/testing_guide/end_to_end/beginners_guide.md2
-rw-r--r--doc/development/testing_guide/end_to_end/feature_flags.md22
-rw-r--r--doc/development/testing_guide/end_to_end/index.md6
-rw-r--r--doc/development/testing_guide/end_to_end/rspec_metadata_tests.md4
-rw-r--r--doc/development/testing_guide/frontend_testing.md35
-rw-r--r--doc/development/testing_guide/img/k9s.pngbin117900 -> 0 bytes
-rw-r--r--doc/development/testing_guide/review_apps.md3
-rw-r--r--doc/development/testing_guide/testing_migrations_guide.md12
8 files changed, 53 insertions, 31 deletions
diff --git a/doc/development/testing_guide/end_to_end/beginners_guide.md b/doc/development/testing_guide/end_to_end/beginners_guide.md
index 27a87d25170..39a3e3445ea 100644
--- a/doc/development/testing_guide/end_to_end/beginners_guide.md
+++ b/doc/development/testing_guide/end_to_end/beginners_guide.md
@@ -49,7 +49,7 @@ For information about the distribution of tests per level in GitLab, see
- Review how often the feature changes. Stable features that don't change very often
might not be worth covering with end-to-end tests if they are already covered
in lower level tests.
-- Finally, discuss the proposed test with the developer(s) involved in implementing
+- Finally, discuss the proposed test with the developers involved in implementing
the feature and the lower-level tests.
WARNING:
diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md
index 47ebef37a4d..b4ec9e8ccd3 100644
--- a/doc/development/testing_guide/end_to_end/feature_flags.md
+++ b/doc/development/testing_guide/end_to_end/feature_flags.md
@@ -23,7 +23,7 @@ Please be sure to include the `feature_flag` tag so that the test can be skipped
`name`
- Format: `feature_flag: { name: 'feature_flag_name' }`
-- Used only for informational purposes at this time. It should be included to help quickly determine what
+- Used only for informational purposes at this time. It should be included to help quickly determine what
feature flag is under test.
`scope`
@@ -31,28 +31,28 @@ feature flag is under test.
- Format: `feature_flag: { name: 'feature_flag_name', scope: :project }`
- When `scope` is set to `:global`, the test will be **skipped on all live .com environments**. This is to avoid issues with feature flag changes affecting other tests or users on that environment.
- When `scope` is set to any other value (such as `:project`, `:group` or `:user`), or if no `scope` is specified, the test will only be **skipped on canary and production**.
-This is due to the fact that admin access is not available there.
+This is due to the fact that administrator access is not available there.
**WARNING:** You are strongly advised to first try and [enable feature flags only for a group, project, user](../../feature_flags/index.md#feature-actors),
or [feature group](../../feature_flags/index.md#feature-groups).
-- If a global feature flag must be used, it is strongly recommended to apply `scope: :global` to the `feature_flag` metadata. This is, however, left up to the SET's discretion to determine the level of risk.
- - For example, a test uses a global feature flag that only affects a small area of the application and is also needed to check for critical issues on live environments.
- In such a scenario, it would be riskier to skip running the test. For cases like this, `scope` can be left out of the metadata so that it can still run in live environments
- with admin access, such as staging.
+- If a global feature flag must be used, it is strongly recommended to apply `scope: :global` to the `feature_flag` metadata. This is, however, left up to the SET's discretion to determine the level of risk.
+ - For example, a test uses a global feature flag that only affects a small area of the application and is also needed to check for critical issues on live environments.
+ In such a scenario, it would be riskier to skip running the test. For cases like this, `scope` can be left out of the metadata so that it can still run in live environments
+ with administrator access, such as staging.
-**Note on `requires_admin`:** This tag should still be applied if there are other actions within the test that require admin access that are unrelated to updating a
+**Note on `requires_admin`:** This tag should still be applied if there are other actions within the test that require administrator access that are unrelated to updating a
feature flag (ex: creating a user via the API).
The code below would enable a feature flag named `:feature_flag_name` for the project
created by the test:
```ruby
-RSpec.describe "with feature flag enabled", feature_flag: {
- name: 'feature_flag_name',
- scope: :project
+RSpec.describe "with feature flag enabled", feature_flag: {
+ name: 'feature_flag_name',
+ scope: :project
} do
-
+
let(:project) { Resource::Project.fabricate_via_api! }
before do
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index 1e7cba9d247..9730115fd9f 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -147,11 +147,11 @@ as well as these:
| Variable | Description |
|-|-|
| `QA_SCENARIO` | The scenario to run (default `Test::Instance::Image`) |
-| `QA_TESTS` | The test(s) to run (no default, which means run all the tests in the scenario). Use file paths as you would when running tests via RSpec, for example, `qa/specs/features/ee/browser_ui` would include all the `EE` UI tests. |
+| `QA_TESTS` | The tests to run (no default, which means run all the tests in the scenario). Use file paths as you would when running tests via RSpec, for example, `qa/specs/features/ee/browser_ui` would include all the `EE` UI tests. |
| `QA_RSPEC_TAGS` | The RSpec tags to add (no default) |
-For now [manual jobs with custom variables don't use the same variable
-when retried](https://gitlab.com/gitlab-org/gitlab/-/issues/31367), so if you want to run the same test(s) multiple times,
+For now, [manual jobs with custom variables don't use the same variable
+when retried](https://gitlab.com/gitlab-org/gitlab/-/issues/31367), so if you want to run the same tests multiple times,
specify the same variables in each `custom-parallel` job (up to as
many of the 10 available jobs that you want to run).
diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
index 45161404c73..0163f2e648c 100644
--- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
+++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
@@ -21,8 +21,8 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
| `:github` | The test requires a GitHub personal access token. |
| `:group_saml` | The test requires a GitLab instance that has SAML SSO enabled at the group level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |
| `:instance_saml` | The test requires a GitLab instance that has SAML SSO enabled at the instance level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |
-| `:integrations` | This aims to test the available [integrations](../../../user/project/integrations/overview.md#integrations-listing). The test requires Docker to be installed in the run context. It will provision the containers and can be run against a local instance or using the `gitlab-qa` scenario `Test::Integration::Integrations` |
-| `:service_ping_disabled` | The test interacts with the GitLab configuration service ping at the instance level to turn admin setting service ping checkbox on or off. This tag will have the test run only in the `service_ping_disabled` job and must be paired with the `:orchestrated` and `:requires_admin` tags. |
+| `:integrations` | This aims to test the available [integrations](../../../user/project/integrations/index.md#available-integrations). The test requires Docker to be installed in the run context. It will provision the containers and can be run against a local instance or using the `gitlab-qa` scenario `Test::Integration::Integrations` |
+| `:service_ping_disabled` | The test interacts with the GitLab configuration service ping at the instance level to turn Admin Area setting service ping checkbox on or off. This tag will have the test run only in the `service_ping_disabled` job and must be paired with the `:orchestrated` and `:requires_admin` tags. |
| `:jira` | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) provisions the Jira Server in a Docker container when the `Test::Integration::Jira` test scenario is run. |
| `:kubernetes` | The test includes a GitLab instance that is configured to be run behind an SSH tunnel, allowing a TLS-accessible GitLab. This test also includes provisioning of at least one Kubernetes cluster to test against. _This tag is often be paired with `:orchestrated`._ |
| `:ldap_no_server` | The test requires a GitLab instance to be configured to use LDAP. To be used with the `:orchestrated` tag. It does not spin up an LDAP server at orchestration time. Instead, it creates the LDAP server at runtime. |
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index d03a4976a8c..d91c53823e2 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -297,7 +297,7 @@ it('tests a promise rejection', async () => {
You can also simply return a promise from the test function.
Using the `done` and `done.fail` callbacks is discouraged when working with
-promises. They should only be used when testing callback-based code.
+promises. They should not be used.
**Bad**:
@@ -466,18 +466,22 @@ it('waits for an Ajax call', () => {
#### Vue rendering
-To wait until a Vue component is re-rendered, use either of the equivalent
-[`Vue.nextTick()`](https://vuejs.org/v2/api/#Vue-nextTick) or `vm.$nextTick()`.
+Use [`nextTick()`](https://vuejs.org/v2/api/#Vue-nextTick) to wait until a Vue component is
+re-rendered.
**in Jest:**
```javascript
-it('renders something', () => {
+import { nextTick } from 'vue';
+
+// ...
+
+it('renders something', async () => {
wrapper.setProps({ value: 'new value' });
- return wrapper.vm.$nextTick().then(() => {
- expect(wrapper.text()).toBe('new value');
- });
+ await nextTick();
+
+ expect(wrapper.text()).toBe('new value');
});
```
@@ -487,15 +491,17 @@ If the application triggers an event that you need to wait for in your test, reg
the assertions:
```javascript
-it('waits for an event', done => {
+it('waits for an event', () => {
eventHub.$once('someEvent', eventHandler);
someFunction();
- function eventHandler() {
- expect(something).toBe('done');
- done();
- }
+ return new Promise((resolve) => {
+ function expectEventHandler() {
+ expect(something).toBe('done');
+ resolve();
+ }
+ });
});
```
@@ -807,11 +813,14 @@ The following are examples of tests that work for Jest:
```javascript
it('uses some HTML element', () => {
- loadFixtures('some/page.html'); // loads spec/frontend/fixtures/some/page.html and adds it to the DOM
+ loadHTMLFixture('some/page.html'); // loads spec/frontend/fixtures/some/page.html and adds it to the DOM
const element = document.getElementById('#my-id');
// ...
+
+ // Jest does not clean up the DOM automatically
+ resetHTMLFixture();
});
```
diff --git a/doc/development/testing_guide/img/k9s.png b/doc/development/testing_guide/img/k9s.png
deleted file mode 100644
index 34585b2a43a..00000000000
--- a/doc/development/testing_guide/img/k9s.png
+++ /dev/null
Binary files differ
diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md
index f5483a4b79c..ff4b77dec2c 100644
--- a/doc/development/testing_guide/review_apps.md
+++ b/doc/development/testing_guide/review_apps.md
@@ -16,6 +16,7 @@ For any of the following scenarios, the `start-review-app-pipeline` job would be
- for merge requests with frontend changes
- for merge requests with changes to `{,ee/,jh/}{app/controllers}/**/*`
- for merge requests with changes to `{,ee/,jh/}{app/models}/**/*`
+- for merge requests with changes to `{,ee/,jh/}lib/{,ee/,jh/}gitlab/**/*`
- for merge requests with QA changes
- for scheduled pipelines
- the MR has the `pipeline:run-review-app` label set
@@ -198,7 +199,7 @@ subgraph "CNG-mirror pipeline"
issue with a link to your merge request. Note that the deployment failure can
reveal an actual problem introduced in your merge request (that is, this isn't
necessarily a transient failure)!
-- If the `review-qa-smoke` or `review-qa-reliable` job keeps failing (note that we already retry them once),
+- If the `review-qa-smoke` or `review-qa-reliable` job keeps failing,
please check the job's logs: you could discover an actual problem introduced in
your merge request. You can also download the artifacts to see screenshots of
the page at the time the failures occurred. If you don't find the cause of the
diff --git a/doc/development/testing_guide/testing_migrations_guide.md b/doc/development/testing_guide/testing_migrations_guide.md
index 4092c1a2f6d..d71788e21f3 100644
--- a/doc/development/testing_guide/testing_migrations_guide.md
+++ b/doc/development/testing_guide/testing_migrations_guide.md
@@ -227,6 +227,18 @@ expect('MigrationClass').to have_scheduled_batched_migration(
)
```
+#### `be_finalize_background_migration_of`
+
+Verifies that a migration calls `finalize_background_migration` with the expected background migration class.
+
+```ruby
+# Migration
+finalize_background_migration('MigrationClass')
+
+# Spec
+expect(described_class).to be_finalize_background_migration_of('MigrationClass')
+```
+
### Examples of migration tests
Migration tests depend on what the migration does exactly, the most common types are data migrations and scheduling background migrations.