summaryrefslogtreecommitdiff
path: root/doc/development/testing_guide
diff options
context:
space:
mode:
Diffstat (limited to 'doc/development/testing_guide')
-rw-r--r--doc/development/testing_guide/best_practices.md74
-rw-r--r--doc/development/testing_guide/contract/index.md2
-rw-r--r--doc/development/testing_guide/end_to_end/feature_flags.md4
-rw-r--r--doc/development/testing_guide/end_to_end/index.md8
-rw-r--r--doc/development/testing_guide/end_to_end/page_objects.md2
-rw-r--r--doc/development/testing_guide/end_to_end/rspec_metadata_tests.md3
-rw-r--r--doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md4
-rw-r--r--doc/development/testing_guide/end_to_end/troubleshooting.md2
-rw-r--r--doc/development/testing_guide/flaky_tests.md10
-rw-r--r--doc/development/testing_guide/frontend_testing.md61
-rw-r--r--doc/development/testing_guide/img/testing_triangle.pngbin11836 -> 32902 bytes
11 files changed, 118 insertions, 52 deletions
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index b6bf3c7805a..aee3e2871c2 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -425,11 +425,10 @@ results are available, and not just the first failure.
when you need an ID/IID/access level that doesn't actually exists. Using 123, 1234,
or even 999 is brittle as these IDs could actually exist in the database in the
context of a CI run.
-- All top-level `RSpec.describe` blocks should have [`feature_category`](https://about.gitlab.com/categories.json) metadata set.
- Consider splitting the file in the case there are identified multiple feature categories in same file.
- If no `feature_category` is identified then use `not_owned`. This information is used in flaky test
- issues created in order to identify the group owning the feature.
- Eg: `RSpec.describe Admin::Geo::SettingsController, :geo, feature_category: :geo_replication do`.
+
+### Feature category metadata
+
+You must [set feature category metadata for each RSpec example](../feature_categorization/index.md#rspec-examples).
### Coverage
@@ -845,6 +844,8 @@ it 'is overdue' do
travel_to(3.days.from_now) do
expect(issue).to be_overdue
end
+
+ travel_back # Returns the current time back to its original state
end
```
@@ -923,6 +924,21 @@ sequence-generated column. To avoid accidental conflicts, specs should also
avoid manually specifying any values in these kinds of columns. Instead, leave
them unspecified, and look up the value after the row is created.
+##### TestProf in migration specs
+
+Because of what is described above, migration specs can't be run inside
+a database transaction. Our test suite uses
+[TestProf](https://github.com/test-prof/test-prof) to improve the runtime of the
+test suite, but `TestProf` uses database transactions to perform these optimizations.
+For this reason, we can't use `TestProf` methods in our migration specs.
+These are the methods that should not be used and should be replaced with
+default RSpec methods instead:
+
+- `let_it_be`: use `let` or `let!` instead.
+- `let_it_be_with_reload`: use `let` or `let!` instead.
+- `let_it_be_with_refind`: use `let` or `let!` instead.
+- `before_all`: use `before` or `before(:all)` instead.
+
#### Redis
GitLab stores two main categories of data in Redis: cached items, and Sidekiq
@@ -1327,7 +1343,7 @@ Testing query performance allows us to:
`QueryRecorder` allows profiling and testing of the number of database queries
performed in a given block of code.
-See the [`QueryRecorder`](../query_recorder.md) section for more details.
+See the [`QueryRecorder`](../database/query_recorder.md) section for more details.
#### GitalyClient
@@ -1439,7 +1455,7 @@ or [cause the universe to implode](../contributing/verify/index.md#do-not-cause-
### Factories
-GitLab uses [factory_bot](https://github.com/thoughtbot/factory_bot) as a test fixture replacement.
+GitLab uses [`factory_bot`](https://github.com/thoughtbot/factory_bot) as a test fixture replacement.
- Factory definitions live in `spec/factories/`, named using the pluralization
of their corresponding model (`User` factories are defined in `users.rb`).
@@ -1451,11 +1467,51 @@ GitLab uses [factory_bot](https://github.com/thoughtbot/factory_bot) as a test f
resulting record to pass validation.
- When instantiating from a factory, don't supply attributes that aren't
required by the test.
-- Prefer [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition),
+- Use [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition),
[explicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#explicit-definition), or
[inline](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#inline-definition) associations
- over `create` / `build` for association setup in callbacks.
+ instead of `create` / `build` for association setup in callbacks.
See [issue #262624](https://gitlab.com/gitlab-org/gitlab/-/issues/262624) for further context.
+
+ When creating factories with a [`has_many`](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#has_many-associations) and `belongs_to` association, use the `instance` method to refer to the object being built.
+ This prevents [creation of unnecessary records](https://gitlab.com/gitlab-org/gitlab/-/issues/378183) by using [interconnected associations](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#interconnected-associations).
+
+ For example, if we have the following classes:
+
+ ```ruby
+ class Car < ApplicationRecord
+ has_many :wheels, inverse_of: :car, foreign_key: :car_id
+ end
+
+ class Wheel < ApplicationRecord
+ belongs_to :car, foreign_key: :car_id, inverse_of: :wheel, optional: false
+ end
+ ```
+
+ We can create the following factories:
+
+ ```ruby
+ FactoryBot.define do
+ factory :car do
+ transient do
+ wheels_count { 2 }
+ end
+
+ wheels do
+ Array.new(wheels_count) do
+ association(:wheel, car: instance)
+ end
+ end
+ end
+ end
+
+ FactoryBot.define do
+ factory :wheel do
+ car { association :car }
+ end
+ end
+ ```
+
- Factories don't have to be limited to `ActiveRecord` objects.
[See example](https://gitlab.com/gitlab-org/gitlab-foss/commit/0b8cefd3b2385a21cfed779bd659978c0402766d).
- Factories and their traits should produce valid objects that are [verified by specs](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/models/factories_spec.rb).
diff --git a/doc/development/testing_guide/contract/index.md b/doc/development/testing_guide/contract/index.md
index 8412a260c7d..08a21e58a52 100644
--- a/doc/development/testing_guide/contract/index.md
+++ b/doc/development/testing_guide/contract/index.md
@@ -26,6 +26,8 @@ The contracts themselves are stored in [`/spec/contracts/contracts`](https://git
Before running the consumer tests, go to `spec/contracts/consumer` and run `npm install`. To run all the consumer tests, you just need to run `npm test -- /specs`. Otherwise, to run a specific spec file, replace `/specs` with the specific spec filename.
+You can also run tests from the root directory of the project, using the command `yarn jest:contract`.
+
### Run the provider tests
Before running the provider tests, make sure your GDK (GitLab Development Kit) is fully set up and running. You can follow the setup instructions detailed in the [GDK repository](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/main). To run the provider tests, you use Rake tasks that can be found in [`./lib/tasks/contracts`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/tasks/contracts). To get a list of all the Rake tasks related to the provider tests, run `bundle exec rake -T contracts`. For example:
diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md
index 6d826e170f6..e473b158087 100644
--- a/doc/development/testing_guide/end_to_end/feature_flags.md
+++ b/doc/development/testing_guide/end_to_end/feature_flags.md
@@ -30,7 +30,7 @@ feature flag is under test.
- Format: `feature_flag: { name: 'feature_flag_name', scope: :project }`
- When `scope` is set to `:global`, the test will be **skipped on all live .com environments**. This is to avoid issues with feature flag changes affecting other tests or users on that environment.
-- When `scope` is set to any other value (such as `:project`, `:group` or `:user`), or if no `scope` is specified, the test will only be **skipped on canary, production, and preprod**.
+- When `scope` is set to any other value (such as `:project`, `:group` or `:user`), or if no `scope` is specified, the test will only be **skipped on canary, production, and pre-production**.
This is due to the fact that administrator access is not available there.
**WARNING:** You are strongly advised to first try and [enable feature flags only for a group, project, user](../../feature_flags/index.md#feature-actors),
@@ -42,7 +42,7 @@ or [feature group](../../feature_flags/index.md#feature-groups).
with administrator access, such as staging.
**Note on `requires_admin`:** This tag should still be applied if there are other actions within the test that require administrator access that are unrelated to updating a
-feature flag (ex: creating a user via the API).
+feature flag (like creating a user via the API).
The code below would enable a feature flag named `:feature_flag_name` for the project
created by the test:
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index 8ffe044c4d8..55d725ba4ae 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -238,7 +238,7 @@ Each type of scheduled pipeline generates a static link for the latest test repo
- [`production`](https://storage.googleapis.com/gitlab-qa-allure-reports/production-full/master/index.html)
- [`production-sanity`](https://storage.googleapis.com/gitlab-qa-allure-reports/production-sanity/master/index.html)
-## How do I run the tests?
+## How do you run the tests?
If you are not [testing code in a merge request](#testing-code-in-merge-requests),
there are two main options for running the tests. If you want to run
@@ -255,12 +255,12 @@ and the section below.
Learn how to perform [tests that require special setup or consideration to run on your local environment](running_tests_that_require_special_setup.md).
-## How do I write tests?
+## How do you write tests?
In order to write new tests, you first need to learn more about GitLab QA
architecture. See the [documentation about it](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md).
-Once you decided where to put [test environment orchestration scenarios](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/lib/gitlab/qa/scenario) and
+After you've decided where to put [test environment orchestration scenarios](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/lib/gitlab/qa/scenario) and
[instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features), take a look at the [GitLab QA README](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/README.md),
the [GitLab QA orchestrator README](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md),
and [the already existing instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features).
@@ -283,7 +283,7 @@ Continued reading:
- [Execution context selection](execution_context_selection.md)
- [Troubleshooting](troubleshooting.md)
-## Where can I ask for help?
+## Where can you ask for help?
You can ask question in the `#quality` channel on Slack (GitLab internal) or
you can find an issue you would like to work on in
diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md
index 5fbf2ffbcde..6a599ce9a50 100644
--- a/doc/development/testing_guide/end_to_end/page_objects.md
+++ b/doc/development/testing_guide/end_to_end/page_objects.md
@@ -155,7 +155,7 @@ In our case, `data-qa-selector="login_field"`, `data-qa-selector="password_field
Things to note:
-- The name of the element and the `qa_selector` must match and be snake_cased
+- The name of the element and the `qa_selector` must match and be snake cased
- If the element appears on the page unconditionally, add `required: true` to the element. See
[Dynamic element validation](dynamic_element_validation.md)
- You may see `.qa-selector` classes in existing Page Objects. We should prefer the [`data-qa-selector`](#data-qa-selector-vs-qa-selector)
diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
index a7d1ece77b2..c1389b3ac0e 100644
--- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
+++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
@@ -15,7 +15,8 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `:elasticsearch` | The test requires an Elasticsearch service. It is used by the [instance-level scenario](https://gitlab.com/gitlab-org/gitlab-qa#definitions) [`Test::Integration::Elasticsearch`](https://gitlab.com/gitlab-org/gitlab/-/blob/72b62b51bdf513e2936301cb6c7c91ec27c35b4d/qa/qa/ee/scenario/test/integration/elasticsearch.rb) to include only tests that require Elasticsearch. |
| `:except` | The test is to be run in their typical execution contexts _except_ as specified. See [test execution context selection](execution_context_selection.md) for more information. |
-| `:feature_flag` | The test uses a feature flag and therefore requires an administrator account to run. When `scope` is set to `:global`, the test will be skipped on all live .com environments. Otherwise, it will be skipped only on Canary, Production, and Preprod. See [testing with feature flags](../../../development/testing_guide/end_to_end/feature_flags.md) for more details. |
+| `:feature_flag` | The test uses a feature flag and therefore requires an administrator account to run. When `scope` is set to `:global`, the test will be skipped on all live .com environments. Otherwise, it will be skipped only on Canary, Production, and Pre-production. See [testing with feature flags](../../../development/testing_guide/end_to_end/feature_flags.md) for more details. |
+| `:framework` | The test makes sanity assertions around the QA framework itself |
| `:geo` | The test requires two GitLab Geo instances - a primary and a secondary - to be spun up. |
| `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. |
| `:github` | The test requires a GitHub personal access token. |
diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
index 81e1c7d5dc0..4a947e59d5f 100644
--- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
+++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
@@ -15,8 +15,8 @@ The project also has instructions for forking and building the images automatica
Some extra environment variables for the location of the forked repository are also needed.
-- `QA_THIRD_PARTY_DOCKER_REGISTRY` (the container registry where the repository/images are hosted, eg `registry.gitlab.com`)
-- `QA_THIRD_PARTY_DOCKER_REPOSITORY` (the base repository path where the images are hosted, eg `registry.gitlab.com/<project path>`)
+- `QA_THIRD_PARTY_DOCKER_REGISTRY` (the container registry where the repository/images are hosted, for example `registry.gitlab.com`)
+- `QA_THIRD_PARTY_DOCKER_REPOSITORY` (the base repository path where the images are hosted, for example `registry.gitlab.com/<project path>`)
- `QA_THIRD_PARTY_DOCKER_USER` (a username that has access to the container registry for this repository)
- `QA_THIRD_PARTY_DOCKER_PASSWORD` (a password/token for the username to authenticate with)
diff --git a/doc/development/testing_guide/end_to_end/troubleshooting.md b/doc/development/testing_guide/end_to_end/troubleshooting.md
index e0925cb71f4..b4c47b90f0b 100644
--- a/doc/development/testing_guide/end_to_end/troubleshooting.md
+++ b/doc/development/testing_guide/end_to_end/troubleshooting.md
@@ -59,7 +59,7 @@ Net::ReadTimeout: Net::ReadTimeout with #<TCPSocket:(closed)>
```
This error can happen if GitLab runs on an address that does not resolve from
-`localhost`. For example, if you set GDK's `hostname`
+`localhost`. For example, if you set the GDK `hostname`
[to a specific local IP address](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/run_qa_against_gdk.md#run-qa-tests-against-your-gdk-setup),
you must use that IP address instead of `localhost` in the command.
For example, if your IP is `192.168.0.12`:
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index cc62a0ebf03..ef5b75d166f 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -31,7 +31,7 @@ it's reset to a pristine test after each test.
inconsistent state, so that following tests might not know about certain columns.
- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/issues/368500): A test modifies data that is
used by a following test.
-- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103434#note_1172316521): A test for a database query passes in a fresh database, but in a
+- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103434#note_1172316521): A test for a database query passes in a fresh database, but in a
CI/CD pipeline where the database is used to process previous test sequences, the test fails. This likely
means that the query itself needs to be updated to work in a non-clean database.
@@ -56,6 +56,7 @@ the problem.
- [Example 1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10148/diffs): Without
specifying `ORDER BY`, database will not give deterministic ordering, or data race happening
in the tests.
+- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106936/diffs).
### Dataset-specific
@@ -75,7 +76,7 @@ difficult to achieve locally.
any table has more than 500 columns. It could pass in the merge request, but fail later in
`master` if the order of tests changes.
- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91016/diffs): A test asserts
- that trying to find a record with an unexisting ID retuns an error message. The test uses an
+ that trying to find a record with an nonexistent ID returns an error message. The test uses an
hardcoded ID that's supposed to not exist (e.g. `42`). If the test is run early in the test
suite, it might pass as not enough records were created before it, but as soon as it would run
later in the suite, there could be a record that actually has the ID `42`, hence the test would
@@ -104,6 +105,7 @@ or the app.
**Description:** The DOM selector used in the test is unreliable.
**Difficulty to reproduce:** Moderate to difficult. Depending on whether the DOM selector is duplicated, or appears after a delay etc.
+Adding a delay in API or controller could help reproducing the issue.
**Resolution:** It really depends on the problem here. It could be to wait for requests to finish, to scroll down the page etc.
@@ -207,10 +209,10 @@ The `rspec/flaky/report-suite.json` report is:
- [Sporadic RSpec failures due to `PG::UniqueViolation`](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/28307#note_24958837): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9846>
- Follow-up: <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10688>
- [Capybara.reset_session! should be called before requests are blocked](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/33779): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12224>
-- FFaker generates funky data that tests are not ready to handle (and tests should be predictable so that's bad!):
+- ffaker generates funky data that tests are not ready to handle (and tests should be predictable so that's bad!):
- [Make `spec/mailers/notify_spec.rb` more robust](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/20121): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10015>
- [Transient failure in `spec/requests/api/commits_spec.rb`](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/27988#note_25342521): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9944>
- - [Replace FFaker factory data with sequences](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/29643): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10184>
+ - [Replace ffaker factory data with sequences](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/29643): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10184>
- [Transient failure in spec/finders/issues_finder_spec.rb](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30211#note_26707685): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10404>
### Order-dependent flaky tests
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index 041b0f0a4f4..2fa5fdeab7d 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -90,7 +90,7 @@ function getFahrenheit(celsius) {
}
```
-It does not make sense to test our `getFahrenheit` function because underneath it does nothing else but invoking the library function, and we can expect that one is working as intended. (Simplified, I know)
+It does not make sense to test our `getFahrenheit` function because underneath it does nothing else but invoking the library function, and we can expect that one is working as intended.
Let's take a short look into Vue land. Vue is a critical part of the GitLab JavaScript codebase. When writing specs for Vue components, a common gotcha is to actually end up testing Vue provided functionality, because it appears to be the easiest thing to test. Here's an example taken from our codebase.
@@ -522,7 +522,7 @@ it('waits for an event', () => {
### Ensuring that tests are isolated
-Tests are normally architected in a pattern which requires a recurring setup and breakdown of the component under test. This is done by making use of the `beforeEach` and `afterEach` hooks.
+Tests are normally architected in a pattern which requires a recurring setup of the component under test. This is often achieved by making use of the `beforeEach` hook.
Example
@@ -532,16 +532,22 @@ Example
beforeEach(() => {
wrapper = mount(Component);
});
+```
+
+With [enableAutoDestroy](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/100389), it is no longer necessary to manually call `wrapper.destroy()`.
+However, some mocks, spies, and fixtures do need to be torn down, and we can leverage the `afterEach` hook.
+
+Example
+
+```javascript
+ let wrapper;
afterEach(() => {
- wrapper.destroy();
+ fakeApollo = null;
+ store = null;
});
```
-When looking at this initially you'd suspect that the component is setup before each test and then broken down afterwards, providing isolation between tests.
-
-This is however not entirely true as the `destroy` method does not remove everything which has been mutated on the `wrapper` object. For functional components, destroy only removes the rendered DOM elements from the document.
-
### Jest best practices
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34209) in GitLab 13.2.
@@ -713,7 +719,7 @@ unit testing by mocking out modules that cannot be easily consumed in our test e
> Instead, consider using [`jest.mock(..)`](https://jestjs.io/docs/jest-object#jestmockmodulename-factory-options)
> (or a similar mocking function) in the relevant spec file.
-#### Where should I put manual mocks?
+#### Where should you put manual mocks?
Jest supports [manual module mocks](https://jestjs.io/docs/manual-mocks) by placing a mock in a `__mocks__/` directory next to the source module
(for example, `app/assets/javascripts/ide/__mocks__`). **Don't do this.** We want to keep all of our test-related code in one place (the `spec/` folder).
@@ -1159,7 +1165,7 @@ By now you've probably heard of [Jest snapshot tests](https://jestjs.io/docs/sna
To use them within GitLab, there are a few guidelines that should be highlighted:
- Treat snapshots as code
-- Don't think of a snapshot file as a Blackbox
+- Don't think of a snapshot file as a black box
- Care for the output of the snapshot, otherwise, it's not providing any real value. This will usually involve reading the generated snapshot file as you would read any other piece of code
Think of a snapshot test as a simple way to store a raw `String` representation of what you've put into the item being tested. This can be used to evaluate changes in a component, a store, a complex piece of generated output, etc. You can see more in the list below for some recommended `Do's and Don'ts`.
@@ -1169,7 +1175,7 @@ Jest provides a great set of docs on [best practices](https://jestjs.io/docs/sna
### How does a snapshot work?
-A snapshot is purely a stringified version of what you ask to be tested on the lefthand side of the function call. This means any kind of changes you make to the formatting of the string has an impact on the outcome. This process is done by leveraging serializers for an automatic transform step. For Vue this is already taken care of by leveraging the `vue-jest` package, which offers the proper serializer.
+A snapshot is purely a stringified version of what you ask to be tested on the left-hand side of the function call. This means any kind of changes you make to the formatting of the string has an impact on the outcome. This process is done by leveraging serializers for an automatic transform step. For Vue this is already taken care of by leveraging the `vue-jest` package, which offers the proper serializer.
Should the outcome of your spec be different from what is in the generated snapshot file, you'll be notified about it by a failing test in your test suite.
@@ -1448,13 +1454,10 @@ Before executing any page interaction when navigating or making asynchronous cal
#### Elements interaction
-There are a lot of different ways to find and interact with elements. For example, you could use the basic `find` method with the `selector` and `text` parameter and then use the `.click` method
-
-```ruby
- find('.gl-tab-nav-item', text: 'Tests').click
-```
+There are a lot of different ways to find and interact with elements.
+For best practises, refer to the [UI testing](best_practices.md#ui-testing) section.
-Alternatively, you could use `click_button` with a string of text that is found within the button, which is a more semantically meaningful way of clicking the element.
+To click a button, use `click_button` with the string of text found in the button:
```ruby
click_button 'Text inside the button element'
@@ -1474,20 +1477,20 @@ You can use `fill_in` to fill input / form elements. The first argument is the s
Alternatively, you can use the `find` selector paired with `send_keys` to add keys in a field without removing previous text, or `set` which completely replaces the value of the input element.
-All of these are valid selectors and methods. Pick whichever suits your needs and look around as there are many more useful ones!
+You can find a more comprehensive list of actions in the [feature tests actions](best_practices.md#actions) documentation.
#### Assertions
To assert anything in a page, you can always access `page` variable, which is automatically defines and actually means the page document. This means you can expect the `page` to have certain components like selectors or content. Here are a few examples:
```ruby
- # Finding an element by ID
- expect(page).to have_selector('#js-pipeline-graph')
+ # Finding a button
+ expect(page).to have_button('Submit review')
```
```ruby
# Finding by text
- expect(page).to have_content('build')
+ expect(page).to have_text('build')
```
```ruby
@@ -1496,20 +1499,20 @@ To assert anything in a page, you can always access `page` variable, which is au
```
```ruby
- # Finding by CSS selector. This is a last resort.
- # For example, when you cannot add attributes on the desired element.
- expect(page).to have_css('.js-icon-retry')
+ # Find by data-testid
+ # Like CSS selector, this is acceptable when there isn't a specific matcher available.
+ expect(page).to have_css('[data-testid="pipeline-multi-actions-dropdown"]')
```
```ruby
- # Find by data-testid
- # Like CSS selector, this is acceptable when there isn't a specific matcher available.
- expect(page).to have_selector('[data-testid="pipeline-multi-actions-dropdown"]')
+ # Finding by CSS selector. This is a last resort.
+ # For example, when you cannot add attributes on the desired element.
+ expect(page).to have_css('.js-icon-retry')
```
```ruby
# You can combine any of these selectors with `not_to` instead
- expect(page).not_to have_selector('#js-pipeline-graph')
+ expect(page).not_to have_button('Submit review')
```
```ruby
@@ -1529,11 +1532,13 @@ You can also create a sub-block to look into, to:
- Make sure an element is found within the right boundaries.
```ruby
- page.within('#js-pipeline-graph') do
+ page.within('[data-testid="pipeline-multi-actions-dropdown"]') do
...
end
```
+You can find a more comprehensive list of matchers in the [feature tests matchers](best_practices.md#matchers) documentation.
+
#### Feature flags
By default, every feature flag is enabled **regardless of the YAML definition or the flags you've set manually in your GDK**. To test when a feature flag is disabled, you must manually stub the flag, ideally in a `before do` block.
diff --git a/doc/development/testing_guide/img/testing_triangle.png b/doc/development/testing_guide/img/testing_triangle.png
index 7a9a848c2ee..3ac4955eaff 100644
--- a/doc/development/testing_guide/img/testing_triangle.png
+++ b/doc/development/testing_guide/img/testing_triangle.png
Binary files differ
View Lorry Controller Status