summaryrefslogtreecommitdiff
path: root/doc/development/testing_guide
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2021-06-16 18:25:58 +0000
committerGitLab Bot <gitlab-bot@gitlab.com>2021-06-16 18:25:58 +0000
commita5f4bba440d7f9ea47046a0a561d49adf0a1e6d4 (patch)
treefb69158581673816a8cd895f9d352dcb3c678b1e /doc/development/testing_guide
parentd16b2e8639e99961de6ddc93909f3bb5c1445ba1 (diff)
downloadgitlab-ce-a5f4bba440d7f9ea47046a0a561d49adf0a1e6d4.tar.gz
Add latest changes from gitlab-org/gitlab@14-0-stable-eev14.0.0-rc42
Diffstat (limited to 'doc/development/testing_guide')
-rw-r--r--doc/development/testing_guide/best_practices.md56
-rw-r--r--doc/development/testing_guide/ci.md4
-rw-r--r--doc/development/testing_guide/end_to_end/beginners_guide.md6
-rw-r--r--doc/development/testing_guide/end_to_end/feature_flags.md4
-rw-r--r--doc/development/testing_guide/end_to_end/img/gl-capybara_V13_12.pngbin19201 -> 6777 bytes
-rw-r--r--doc/development/testing_guide/end_to_end/img/gl-chemlab_V13_12.pngbin17753 -> 6580 bytes
-rw-r--r--doc/development/testing_guide/end_to_end/index.md7
-rw-r--r--doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md4
-rw-r--r--doc/development/testing_guide/flaky_tests.md6
-rw-r--r--doc/development/testing_guide/frontend_testing.md22
-rw-r--r--doc/development/testing_guide/review_apps.md16
-rw-r--r--doc/development/testing_guide/testing_levels.md4
-rw-r--r--doc/development/testing_guide/testing_rake_tasks.md8
13 files changed, 96 insertions, 41 deletions
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index c3125f52cf2..c44e26927fe 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -851,6 +851,47 @@ using expectations, or dependency injection along with stubs, to avoid the need
for modifications. If you have no other choice, an `around` block like the global
variables example can be used, but avoid this if at all possible.
+#### Elasticsearch specs
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61171) in GitLab 14.0.
+
+Specs that require Elasticsearch must be marked with the `:elastic` trait. This
+creates and deletes indices between examples to ensure a clean index, so that there is no room
+for polluting the tests with nonessential data.
+Most tests for Elasticsearch logic relate to:
+
+- Creating data in Postgres and waiting for it to be indexed in Elasticsearch.
+- Searching for that data.
+- Ensuring that the test gives the expected result.
+
+There are some exceptions, such as checking for structural changes rather than individual records in an index.
+
+The `:elastic_with_delete_by_query` trait was added to reduce run time for pipelines by creating and deleting indices
+at the start and end of each context only. The [Elasticsearch DeleteByQuery API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html)
+is used to delete data in all indices in between examples to ensure a clean index.
+
+Note that Elasticsearch indexing uses [`Gitlab::Redis::SharedState`](../../../ee/development/redis.md#gitlabrediscachesharedstatequeues).
+Therefore, the Elasticsearch traits dynamically use the `:clean_gitlab_redis_shared_state` trait.
+You do NOT need to add `:clean_gitlab_redis_shared_state` manually.
+
+Specs using Elasticsearch require that you:
+
+- Create data in Postgres and then index it into Elasticsearch.
+- Enable Application Settings for Elasticsearch (which is disabled by default).
+
+To do so, use:
+
+```ruby
+before do
+ stub_ee_application_setting(elasticsearch_search: true, elasticsearch_indexing: true)
+end
+```
+
+Additionally, you can use the `ensure_elasticsearch_index!` method to overcome the asynchronous nature of Elasticsearch.
+It uses the [Elasticsearch Refresh API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html#refresh-api-desc)
+to make sure all operations performed on an index since the last refresh are available for search. This method is typically
+called after loading data into Postgres to ensure the data is indexed and searchable.
+
#### Test Snowplow events
WARNING:
@@ -954,6 +995,7 @@ Only use simple values as input in the `where` block. Using
objects, FactoryBot-created objects, and similar items can lead to
[unexpected results](https://github.com/tomykaira/rspec-parameterized/issues/8).
<!-- vale gitlab.Spelling = YES -->
+
### Prometheus tests
Prometheus metrics may be preserved from one test run to another. To ensure that metrics are
@@ -1034,6 +1076,16 @@ expect(json_string).to be_valid_json
expect(json_string).to be_valid_json.and match_schema(schema)
```
+#### `be_one_of(collection)`
+
+The inverse of `include`, tests that the `collection` includes the expected
+value:
+
+```ruby
+expect(:a).to be_one_of(%i[a b c])
+expect(:z).not_to be_one_of(%i[a b c])
+```
+
### Testing query performance
Testing query performance allows us to:
@@ -1097,7 +1149,7 @@ module Spec
module Helpers
module CycleAnalyticsHelpers
def create_commit_referencing_issue(issue, branch_name: random_git_name)
- project.repository.add_branch(user, branch_name, 'master')
+ project.repository.add_branch(user, branch_name, 'main')
create_commit("Commit for ##{issue.iid}", issue.project, user, branch_name)
end
end
@@ -1154,7 +1206,7 @@ let(:project) { create(:project, :repository) }
```
Where you can, consider using the `:custom_repo` trait instead of `:repository`.
-This allows you to specify exactly what files appear in the `master` branch
+This allows you to specify exactly what files appear in the `main` branch
of the project's repository. For example:
```ruby
diff --git a/doc/development/testing_guide/ci.md b/doc/development/testing_guide/ci.md
index 7318f767219..e3fccdcee34 100644
--- a/doc/development/testing_guide/ci.md
+++ b/doc/development/testing_guide/ci.md
@@ -12,7 +12,7 @@ Our current CI parallelization setup is as follows:
1. The `retrieve-tests-metadata` job in the `prepare` stage ensures we have a
`knapsack/report-master.json` file:
- - The `knapsack/report-master.json` file is fetched from the latest `master` pipeline which runs `update-tests-metadata`
+ - The `knapsack/report-master.json` file is fetched from the latest `main` pipeline which runs `update-tests-metadata`
(for now it's the 2-hourly scheduled master pipeline), if it's not here we initialize the file with `{}`.
1. Each `[rspec|rspec-ee] [unit|integration|system|geo] n m` job are run with
`knapsack rspec` and should have an evenly distributed share of tests:
@@ -31,7 +31,7 @@ After that, the next pipeline uses the up-to-date `knapsack/report-master.json`
## Monitoring
-The GitLab test suite is [monitored](../performance.md#rspec-profiling) for the `master` branch, and any branch
+The GitLab test suite is [monitored](../performance.md#rspec-profiling) for the `main` branch, and any branch
that includes `rspec-profile` in their name.
## CI setup
diff --git a/doc/development/testing_guide/end_to_end/beginners_guide.md b/doc/development/testing_guide/end_to_end/beginners_guide.md
index 7cde2cad300..e0f0e9e7089 100644
--- a/doc/development/testing_guide/end_to_end/beginners_guide.md
+++ b/doc/development/testing_guide/end_to_end/beginners_guide.md
@@ -41,11 +41,11 @@ Does sufficient test coverage exist at the unit, feature, or integration levels?
If you answered *yes*, then you *don't* need an end-to-end test.
For information about the distribution of tests per level in GitLab, see
-[Testing Levels](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/testing_guide/testing_levels.md).
+[Testing Levels](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md).
- See the
- [How to test at the correct level?](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/testing_guide/testing_levels.md#how-to-test-at-the-correct-level)
- section of the [Testing levels](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/testing_guide/testing_levels.md) document.
+ [How to test at the correct level?](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md#how-to-test-at-the-correct-level)
+ section of the [Testing levels](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/testing_levels.md) document.
- Review how often the feature changes. Stable features that don't change very often
might not be worth covering with end-to-end tests if they are already covered
in lower level tests.
diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md
index e3719393d41..c9acb2e9371 100644
--- a/doc/development/testing_guide/end_to_end/feature_flags.md
+++ b/doc/development/testing_guide/end_to_end/feature_flags.md
@@ -72,7 +72,7 @@ Runtime::Feature.enable(:feature_flag_name)
It's also possible to run an entire scenario with a feature flag enabled, without having to edit
existing tests or write new ones.
-Please see the [QA README](https://gitlab.com/gitlab-org/gitlab/tree/master/qa#running-tests-with-a-feature-flag-enabled)
+Please see the [QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa#running-tests-with-a-feature-flag-enabled)
for details.
## Confirming that end-to-end tests pass with a feature flag enabled
@@ -81,7 +81,7 @@ End-to-end tests should pass with a feature flag enabled before it is enabled on
If a test enables a feature flag as describe above, it is sufficient to run the `package-and-qa` job in a merge request containing the relevant changes.
Or, if the feature flag and relevant changes have already been merged, you can confirm that the tests
-pass on `master`. The end-to-end tests run on `master` every two hours, and the results are posted to a [Test
+pass on `main`. The end-to-end tests run on `main` every two hours, and the results are posted to a [Test
Session Report, which is available in the testcase-sessions project](https://gitlab.com/gitlab-org/quality/testcase-sessions/-/issues?label_name%5B%5D=found%3Amaster).
If the relevant tests do not enable the feature flag themselves, you can check if the tests will need
diff --git a/doc/development/testing_guide/end_to_end/img/gl-capybara_V13_12.png b/doc/development/testing_guide/end_to_end/img/gl-capybara_V13_12.png
index 9ceccd39025..d5a2522ed82 100644
--- a/doc/development/testing_guide/end_to_end/img/gl-capybara_V13_12.png
+++ b/doc/development/testing_guide/end_to_end/img/gl-capybara_V13_12.png
Binary files differ
diff --git a/doc/development/testing_guide/end_to_end/img/gl-chemlab_V13_12.png b/doc/development/testing_guide/end_to_end/img/gl-chemlab_V13_12.png
index 489a043f52e..55eecaf8adf 100644
--- a/doc/development/testing_guide/end_to_end/img/gl-chemlab_V13_12.png
+++ b/doc/development/testing_guide/end_to_end/img/gl-chemlab_V13_12.png
Binary files differ
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index e6da4771e55..6ab288b0525 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -100,7 +100,8 @@ You may have noticed that we use `gitlab-org/build/omnibus-gitlab-mirror` instea
This is due to technical limitations in the GitLab permission model: the ability to run a pipeline
against a protected branch is controlled by the ability to push/merge to this branch.
This means that for developers to be able to trigger a pipeline for the default branch in
-`gitlab-org/omnibus-gitlab`/`gitlab-org/gitlab-qa`, they would need to have Maintainer permission in those projects.
+`gitlab-org/omnibus-gitlab`/`gitlab-org/gitlab-qa`, they would need to have the
+[Maintainer role](../../../user/permissions.md) for those projects.
For security reasons and implications, we couldn't open up the default branch to all the Developers.
Hence we created these mirrors where Developers and Maintainers are allowed to push/merge to the default branch.
This problem was discovered in <https://gitlab.com/gitlab-org/gitlab-qa/-/issues/63#note_107175160> and the "mirror"
@@ -179,7 +180,7 @@ of the test scenarios you can run via the orchestrator](https://gitlab.com/gitla
On the other hand, if you would like to run against a local development GitLab
environment, you can use the [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/).
-Please refer to the instructions in the [QA README](https://gitlab.com/gitlab-org/gitlab/tree/master/qa/README.md#how-can-i-use-it)
+Please refer to the instructions in the [QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/README.md#how-can-i-use-it)
and the section below.
### Running tests that require special setup
@@ -192,7 +193,7 @@ In order to write new tests, you first need to learn more about GitLab QA
architecture. See the [documentation about it](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md).
Once you decided where to put [test environment orchestration scenarios](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/lib/gitlab/qa/scenario) and
-[instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features), take a look at the [GitLab QA README](https://gitlab.com/gitlab-org/gitlab/tree/master/qa/README.md),
+[instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features), take a look at the [GitLab QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/README.md),
the [GitLab QA orchestrator README](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md), and [the already existing
instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features).
diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
index 549ab95a5d1..859b8f950e3 100644
--- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
+++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
## Jenkins spec
-The [`jenkins_build_status_spec`](https://gitlab.com/gitlab-org/gitlab/blob/163c8a8c814db26d11e104d1cb2dcf02eb567dbe/qa/qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb) spins up a Jenkins instance in a Docker container based on an image stored in the [GitLab-QA container registry](https://gitlab.com/gitlab-org/gitlab-qa/container_registry).
+The [`jenkins_build_status_spec`](https://gitlab.com/gitlab-org/gitlab/-/blob/163c8a8c814db26d11e104d1cb2dcf02eb567dbe/qa/qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb) spins up a Jenkins instance in a Docker container based on an image stored in the [GitLab-QA container registry](https://gitlab.com/gitlab-org/gitlab-qa/container_registry).
The Docker image it uses is preconfigured with some base data and plugins.
The test then configures the GitLab plugin in Jenkins with a URL of the GitLab instance that are used
to run the tests. Unfortunately, the GitLab Jenkins plugin does not accept ports so `http://localhost:3000` would
@@ -47,7 +47,7 @@ Jenkins is available on `http://localhost:8080`.
Admin username is `admin` and password is `password`.
-It is worth noting that this is not an orchestrated test. It is [tagged with the `:orchestrated` meta](https://gitlab.com/gitlab-org/gitlab/blob/163c8a8c814db26d11e104d1cb2dcf02eb567dbe/qa/qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb#L5)
+It is worth noting that this is not an orchestrated test. It is [tagged with the `:orchestrated` meta](https://gitlab.com/gitlab-org/gitlab/-/blob/163c8a8c814db26d11e104d1cb2dcf02eb567dbe/qa/qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb#L5)
only to prevent it from running in the pipelines for live environments such as Staging.
### Troubleshooting
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index 6b1c7a7eb58..bfcd68dbaf3 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -13,7 +13,7 @@ eventually.
## Quarantined tests
-When a test frequently fails in `master`,
+When a test frequently fails in `main`,
[a ~"master:broken" issue](https://about.gitlab.com/handbook/engineering/workflow/#broken-master)
should be created.
If the test cannot be fixed in a timely fashion, there is an impact on the
@@ -53,10 +53,10 @@ Quarantined tests are run on the CI in dedicated jobs that are allowed to fail:
## Automatic retries and flaky tests detection
On our CI, we use [RSpec::Retry](https://github.com/NoRedInk/rspec-retry) to automatically retry a failing example a few
-times (see [`spec/spec_helper.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/spec_helper.rb) for the precise retries count).
+times (see [`spec/spec_helper.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/spec_helper.rb) for the precise retries count).
We also use a home-made `RspecFlaky::Listener` listener which records flaky
-examples in a JSON report file on `master` (`retrieve-tests-metadata` and
+examples in a JSON report file on `main` (`retrieve-tests-metadata` and
`update-tests-metadata` jobs).
This was originally implemented in: <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13021>.
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index 911fbd43989..8573fa81718 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -54,7 +54,7 @@ which have to be stubbed.
- Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/26982).
- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. See also its [limitations](#limitations-of-jsdom) below.
- Jest does not have access to Webpack loaders or aliases.
- The aliases used by Jest are defined in its [own configuration](https://gitlab.com/gitlab-org/gitlab/blob/master/jest.config.js).
+ The aliases used by Jest are defined in its [own configuration](https://gitlab.com/gitlab-org/gitlab/-/blob/master/jest.config.js).
- All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/timer-mocks).
- `rewire` is not required because Jest supports mocking modules. See also [Manual Mocks](https://jestjs.io/docs/manual-mocks).
- No [context object](https://jasmine.github.io/tutorials/your_first_suite#section-The_%3Ccode%3Ethis%3C/code%3E_keyword) is passed to tests in Jest.
@@ -83,13 +83,13 @@ Running `yarn jest-debug` runs Jest in debug mode, allowing you to debug/inspect
### Timeout error
The default timeout for Jest is set in
-[`/spec/frontend/test_setup.js`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/test_setup.js).
+[`/spec/frontend/test_setup.js`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/frontend/test_setup.js).
If your test exceeds that time, it fails.
If you cannot improve the performance of the tests, you can increase the timeout
for a specific test using
-[`setTestTimeout`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/__helpers__/timeout.js).
+[`setTestTimeout`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/frontend/__helpers__/timeout.js).
```javascript
import { setTestTimeout } from 'helpers/timeout';
@@ -249,7 +249,7 @@ it('exists', () => {
wrapper.findByText(/Click Me/i)
// Good (especially for unit tests)
- wrapper.find(FooComponent);
+ wrapper.findComponent(FooComponent);
wrapper.find('input[name=foo]');
wrapper.find('[data-testid="my-foo-id"]');
wrapper.findByTestId('my-foo-id'); // with shallowMountExtended or mountExtended – check below
@@ -281,7 +281,7 @@ Example:
```javascript
it('exists', () => {
- wrapper.find(FooComponent);
+ wrapper.findComponent(FooComponent);
});
```
@@ -386,7 +386,7 @@ Sometimes we have to test time-sensitive code. For example, recurring events tha
#### `setTimeout()` / `setInterval()` in application
If the application itself is waiting for some time, mock await the waiting. In Jest this is already
-[done by default](https://gitlab.com/gitlab-org/gitlab/blob/a2128edfee799e49a8732bfa235e2c5e14949c68/jest.config.js#L47)
+[done by default](https://gitlab.com/gitlab-org/gitlab/-/blob/a2128edfee799e49a8732bfa235e2c5e14949c68/jest.config.js#L47)
(see also [Jest Timer Mocks](https://jestjs.io/docs/timer-mocks)). In Karma you can use the
[Jasmine mock clock](https://jasmine.github.io/api/2.9/Clock.html).
@@ -748,7 +748,7 @@ Jest supports [manual module mocks](https://jestjs.io/docs/manual-mocks) by plac
(e.g. `app/assets/javascripts/ide/__mocks__`). **Don't do this.** We want to keep all of our test-related code in one place (the `spec/` folder).
If a manual mock is needed for a `node_modules` package, use the `spec/frontend/__mocks__` folder. Here's an example of
-a [Jest mock for the package `monaco-editor`](https://gitlab.com/gitlab-org/gitlab/blob/b7f914cddec9fc5971238cdf12766e79fa1629d7/spec/frontend/__mocks__/monaco-editor/index.js#L1).
+a [Jest mock for the package `monaco-editor`](https://gitlab.com/gitlab-org/gitlab/-/blob/b7f914cddec9fc5971238cdf12766e79fa1629d7/spec/frontend/__mocks__/monaco-editor/index.js#L1).
If a manual mock is needed for a CE module, place it in `spec/frontend/mocks/ce`.
@@ -759,12 +759,12 @@ If a manual mock is needed for a CE module, place it in `spec/frontend/mocks/ce`
#### Manual mock examples
-- [`mocks/axios_utils`](https://gitlab.com/gitlab-org/gitlab/blob/bd20aeb64c4eed117831556c54b40ff4aee9bfd1/spec/frontend/mocks/ce/lib/utils/axios_utils.js#L1) -
+- [`mocks/axios_utils`](https://gitlab.com/gitlab-org/gitlab/-/blob/bd20aeb64c4eed117831556c54b40ff4aee9bfd1/spec/frontend/mocks/ce/lib/utils/axios_utils.js#L1) -
This mock is helpful because we don't want any unmocked requests to pass any tests. Also, we are able to inject some test helpers such as `axios.waitForAll`.
-- [`__mocks__/mousetrap/index.js`](https://gitlab.com/gitlab-org/gitlab/blob/cd4c086d894226445be9d18294a060ba46572435/spec/frontend/__mocks__/mousetrap/index.js#L1) -
+- [`__mocks__/mousetrap/index.js`](https://gitlab.com/gitlab-org/gitlab/-/blob/cd4c086d894226445be9d18294a060ba46572435/spec/frontend/__mocks__/mousetrap/index.js#L1) -
This mock is helpful because the module itself uses AMD format which webpack understands, but is incompatible with the jest environment. This mock doesn't remove
any behavior, only provides a nice es6 compatible wrapper.
-- [`__mocks__/monaco-editor/index.js`](https://gitlab.com/gitlab-org/gitlab/blob/b7f914cddec9fc5971238cdf12766e79fa1629d7/spec/frontend/__mocks__/monaco-editor/index.js) -
+- [`__mocks__/monaco-editor/index.js`](https://gitlab.com/gitlab-org/gitlab/-/blob/b7f914cddec9fc5971238cdf12766e79fa1629d7/spec/frontend/__mocks__/monaco-editor/index.js) -
This mock is helpful because the Monaco package is completely incompatible in a Jest environment. In fact, webpack requires a special loader to make it work. This mock
makes this package consumable by Jest.
@@ -1109,7 +1109,7 @@ See also [Notes on testing Vue components](../fe_guide/vue.md#testing-vue-compon
## Test helpers
-Test helpers can be found in [`spec/frontend/__helpers__`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/frontend/__helpers__).
+Test helpers can be found in [`spec/frontend/__helpers__`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/frontend/__helpers__).
If you introduce new helpers, place them in that directory.
### Vuex Helper: `testAction`
diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md
index c4194be23a4..cf757aad870 100644
--- a/doc/development/testing_guide/review_apps.md
+++ b/doc/development/testing_guide/review_apps.md
@@ -81,6 +81,8 @@ subgraph "CNG-mirror pipeline"
- Since we're using [the official GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab/), this means
you get a dedicated environment for your branch that's very close to what
it would look in production.
+ - Each review app is deployed to its own Kubernetes namespace. The namespace is based on the Review App slug that is
+ unique to each branch.
1. Once the [`review-deploy`](https://gitlab.com/gitlab-org/gitlab/-/jobs/467724810) job succeeds, you should be able to
use your Review App thanks to the direct link to it from the MR widget. To log
into the Review App, see "Log into my Review App?" below.
@@ -132,6 +134,9 @@ the QA smoke suite.
You can also manually start the `review-qa-all`: it runs the full QA suite.
+After the end-to-end test runs have finished, [Allure reports](https://github.com/allure-framework/allure2) are generated and published by
+the `allure-report-qa-smoke` and `allure-report-qa-all` jobs. A comment with links to the reports are added to the merge request.
+
## Performance Metrics
On every [pipeline](https://gitlab.com/gitlab-org/gitlab/pipelines/125315730) in the `qa` stage, the
@@ -141,12 +146,7 @@ browser performance testing using a
## Cluster configuration
-### Node pools
-
-The `review-apps` cluster is currently set up with
-the following node pools:
-
-- `e2-highcpu-16` (16 vCPU, 16 GB memory) pre-emptible nodes with autoscaling
+The cluster is configured via Terraform in the [`engineering-productivity-infrastructure`](https://gitlab.com/gitlab-org/quality/engineering-productivity-infrastructure) project.
Node pool image type must be `Container-Optimized OS (cos)`, not `Container-Optimized OS with Containerd (cos_containerd)`,
due to this [known issue on GitLab Runner Kubernetes executor](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4755)
@@ -200,7 +200,7 @@ the GitLab handbook information for the [shared 1Password account](https://about
1. Click on the `KUBECTL` dropdown, then `Exec` -> `task-runner`.
1. Replace `-c task-runner -- ls` with `-it -- gitlab-rails console` from the
default command or
- - Run `kubectl exec --namespace review-apps review-qa-raise-e-12chm0-task-runner-d5455cc8-2lsvz -it -- gitlab-rails console` and
+ - Run `kubectl exec --namespace review-qa-raise-e-12chm0 review-qa-raise-e-12chm0-task-runner-d5455cc8-2lsvz -it -- gitlab-rails console` and
- Replace `review-qa-raise-e-12chm0-task-runner-d5455cc8-2lsvz`
with your Pod's name.
@@ -218,7 +218,7 @@ the GitLab handbook information for the [shared 1Password account](https://about
## Diagnosing unhealthy Review App releases
If [Review App Stability](https://app.periscopedata.com/app/gitlab/496118/Engineering-Productivity-Sandbox?widget=6690556&udv=785399)
-dips this may be a signal that the `review-apps-ce/ee` cluster is unhealthy.
+dips this may be a signal that the `review-apps` cluster is unhealthy.
Leading indicators may be health check failures leading to restarts or majority failure for Review App deployments.
The [Review Apps Overview dashboard](https://console.cloud.google.com/monitoring/classic/dashboards/6798952013815386466?project=gitlab-review-apps&timeDomain=1d)
diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md
index abacb9a0c87..3a4a28702c7 100644
--- a/doc/development/testing_guide/testing_levels.md
+++ b/doc/development/testing_guide/testing_levels.md
@@ -48,7 +48,7 @@ records should use stubs/doubles as much as possible.
| `config/` | `spec/config/` | RSpec | |
| `config/initializers/` | `spec/initializers/` | RSpec | |
| `config/routes.rb`, `config/routes/` | `spec/routing/` | RSpec | |
-| `config/puma.example.development.rb`, `config/unicorn.rb.example` | `spec/rack_servers/` | RSpec | |
+| `config/puma.example.development.rb` | `spec/rack_servers/` | RSpec | |
| `db/` | `spec/db/` | RSpec | |
| `db/{post_,}migrate/` | `spec/migrations/` | RSpec | More details in the [Testing Rails migrations guide](testing_migrations_guide.md). |
| `Gemfile` | `spec/dependencies/`, `spec/sidekiq/` | RSpec | |
@@ -486,7 +486,7 @@ Note that:
- data needed for the tests can only be created using the GUI or the API
- expectations can only be made against the browser page and API responses
-Every new feature should come with a [test plan](https://gitlab.com/gitlab-org/gitlab/tree/master/.gitlab/issue_templates/Test%20plan.md).
+Every new feature should come with a [test plan](https://gitlab.com/gitlab-org/gitlab/-/tree/master/.gitlab/issue_templates/Test%20plan.md).
| Tests path | Testing engine | Notes |
| ---------- | -------------- | ----- |
diff --git a/doc/development/testing_guide/testing_rake_tasks.md b/doc/development/testing_guide/testing_rake_tasks.md
index dc754721e24..30d193de2f2 100644
--- a/doc/development/testing_guide/testing_rake_tasks.md
+++ b/doc/development/testing_guide/testing_rake_tasks.md
@@ -11,19 +11,21 @@ in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use
`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures
a few other things to make testing Rake tasks easier.
-At a minimum, requiring the Rake helper redirects `stdout`, include the
-runtime task helpers, and include the `RakeHelpers` Spec support module.
+At a minimum, requiring the Rake helper includes the runtime task helpers, and
+includes the `RakeHelpers` Spec support module.
The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make
executing tasks simple. See `spec/support/helpers/rake_helpers.rb` for all available
methods.
+`$stdout` can be redirected by adding `:silence_stdout`.
+
Example:
```ruby
require 'rake_helper'
-describe 'gitlab:shell rake tasks' do
+describe 'gitlab:shell rake tasks', :silence_stdout do
before do
Rake.application.rake_require 'tasks/gitlab/shell'