summaryrefslogtreecommitdiff
path: root/doc/development/testing_guide
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2020-01-21 14:21:10 +0000
committerGitLab Bot <gitlab-bot@gitlab.com>2020-01-21 14:21:10 +0000
commitcb0d23c455b73486fd1015f8ca9479b5b7e3585d (patch)
treed7dc129a407fd74266d2dc561bebf24665197c2f /doc/development/testing_guide
parentc3e911be175c0aabfea1eb030f9e0ef23f5f3887 (diff)
downloadgitlab-ce-cb0d23c455b73486fd1015f8ca9479b5b7e3585d.tar.gz
Add latest changes from gitlab-org/gitlab@12-7-stable-ee
Diffstat (limited to 'doc/development/testing_guide')
-rw-r--r--doc/development/testing_guide/best_practices.md3
-rw-r--r--doc/development/testing_guide/end_to_end/best_practices.md2
-rw-r--r--doc/development/testing_guide/end_to_end/index.md18
-rw-r--r--doc/development/testing_guide/end_to_end/page_objects.md6
-rw-r--r--doc/development/testing_guide/end_to_end/quick_start_guide.md2
-rw-r--r--doc/development/testing_guide/end_to_end/style_guide.md18
-rw-r--r--doc/development/testing_guide/flaky_tests.md6
-rw-r--r--doc/development/testing_guide/frontend_testing.md6
-rw-r--r--doc/development/testing_guide/img/k9s.pngbin364038 -> 117900 bytes
9 files changed, 34 insertions, 27 deletions
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index 356e3f7a227..a845b5a26e8 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -67,9 +67,10 @@ When using spring and guard together, use `SPRING=1 bundle exec guard` instead t
- Don't supply the `:each` argument to hooks since it's the default.
- On `before` and `after` hooks, prefer it scoped to `:context` over `:all`
- When using `evaluate_script("$('.js-foo').testSomething()")` (or `execute_script`) which acts on a given element,
- use a Capyabara matcher beforehand (e.g. `find('.js-foo')`) to ensure the element actually exists.
+ use a Capybara matcher beforehand (e.g. `find('.js-foo')`) to ensure the element actually exists.
- Use `focus: true` to isolate parts of the specs you want to run.
- Use [`:aggregate_failures`](https://relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures) when there is more than one expectation in a test.
+- For [empty test description blocks](https://github.com/rubocop-hq/rspec-style-guide#it-and-specify), use `specify` rather than `it do` if the test is self-explanatory.
### System / Feature tests
diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md
index fc00fcea67e..e060312e05f 100644
--- a/doc/development/testing_guide/end_to_end/best_practices.md
+++ b/doc/development/testing_guide/end_to_end/best_practices.md
@@ -14,7 +14,7 @@ Now, realize that almost all tests need the user to be logged in, and that we ne
Now, multiply the number of tests per 2 seconds, and as your test suite grows, the time to run it grows with it, and this is not sustainable.
-An alternative to perform a login in a cheaper way would be having an endpoint (available only for testing) where we could pass the user's credentials as encrypted values as query strings, and then we would be redirected to the logged in home page if the credentials are valid. Let's say that, on average, this process takes only 200 miliseconds.
+An alternative to perform a login in a cheaper way would be having an endpoint (available only for testing) where we could pass the user's credentials as encrypted values as query strings, and then we would be redirected to the logged in home page if the credentials are valid. Let's say that, on average, this process takes only 200 milliseconds.
You see the point right?
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index 19885f5756f..96141a5d68d 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -15,27 +15,27 @@ a black-box testing framework for the API and the UI.
### Testing nightly builds
-We run scheduled pipeline each night to test nightly builds created by Omnibus.
+We run scheduled pipelines each night to test nightly builds created by Omnibus.
You can find these nightly pipelines at `https://gitlab.com/gitlab-org/quality/nightly/pipelines`
(need Developer access permissions). Results are reported in the `#qa-nightly` Slack channel.
### Testing staging
-We run scheduled pipeline each night to test staging.
+We run scheduled pipelines each night to test staging.
You can find these nightly pipelines at `https://gitlab.com/gitlab-org/quality/staging/pipelines`
-(need developer access permissions). Results are reported in the `#qa-staging` Slack channel.
+(need Developer access permissions). Results are reported in the `#qa-staging` Slack channel.
### Testing code in merge requests
-#### Using the `package-and-qa-manual` job
+#### Using the `package-and-qa` job
It is possible to run end-to-end tests for a merge request, eventually being run in
a pipeline in the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/) project,
-by triggering the `package-and-qa-manual` manual action in the `test` stage (not
+by triggering the `package-and-qa` manual action in the `test` stage (not
available for forks).
-**This runs end-to-end tests against a custom Omnibus package built from your
-merge request's changes.**
+**This runs end-to-end tests against a custom CE and EE (with an Ultimate license)
+Omnibus package built from your merge request's changes.**
Manual action that starts end-to-end tests is also available in merge requests
in [Omnibus GitLab][omnibus-gitlab].
@@ -53,7 +53,7 @@ graph LR
B2[`Trigger-qa` stage<br>`Trigger:qa-test` job] -.->|2. Triggers a gitlab-qa pipeline and wait for it to be done| A3
subgraph "gitlab-foss/gitlab pipeline"
- A1[`test` stage<br>`package-and-qa-manual` job]
+ A1[`test` stage<br>`package-and-qa` job]
end
subgraph "omnibus-gitlab pipeline"
@@ -61,7 +61,7 @@ subgraph "omnibus-gitlab pipeline"
end
subgraph "gitlab-qa pipeline"
- A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa-manual` job<br>and post the result on the original commit tested| A1
+ A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa` job<br>and post the result on the original commit tested| A1
end
```
diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md
index 554995fa2e2..f1e0de0c792 100644
--- a/doc/development/testing_guide/end_to_end/page_objects.md
+++ b/doc/development/testing_guide/end_to_end/page_objects.md
@@ -40,7 +40,7 @@ the time it would take to build packages and test everything.
That is why when someone changes `t.text_field :login` to
`t.text_field :username` in the _new session_ view we won't know about this
change until our GitLab QA nightly pipeline fails, or until someone triggers
-`package-and-qa-manual` action in their merge request.
+`package-and-qa` action in their merge request.
Obviously such a change would break all tests. We call this problem a _fragile
tests problem_.
@@ -171,8 +171,8 @@ and we should prefer the `data-qa-selector` method of definition.
> Introduced in GitLab 12.5
-A common occurrence in automated testing is selecting a single "one-of-many" element.
-In a list of several items, how do you differentiate what you are selecting on?
+A common occurrence in automated testing is selecting a single "one-of-many" element.
+In a list of several items, how do you differentiate what you are selecting on?
The most common workaround for this is via text matching. Instead, a better practice is
by matching on that specific element by a unique identifier, rather than by text.
diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md
index fb820ac22a2..be00129a2bc 100644
--- a/doc/development/testing_guide/end_to_end/quick_start_guide.md
+++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md
@@ -445,7 +445,7 @@ end
By defining the `resource_web_url(resource)` method, we override the one from the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb#L44) module. We do that to avoid failing the test due to this particular resource not exposing a `web_url` property.
-By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the publich API, we raise a `NotImplementedError` instead.
+By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the public API, we raise a `NotImplementedError` instead.
By defining the `api_post_path` method, we allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project.
diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md
index 9088e9e9bfb..7f4616f394b 100644
--- a/doc/development/testing_guide/end_to_end/style_guide.md
+++ b/doc/development/testing_guide/end_to_end/style_guide.md
@@ -54,18 +54,20 @@ We follow a simple formula roughly based on hungarian notation.
*Formula*: `element :<descriptor>_<type>`
- `descriptor`: The natural-language description of what the element is. On the login page, this could be `username`, or `password`.
-- `type`: A physical control on the page that can be seen by a user.
+- `type`: A generic control on the page that can be seen by a user.
- `_button`
- - `_link`
- - `_tab`
- - `_dropdown`
- - `_field`
- `_checkbox`
+ - `_container`: an element that includes other elements, but doesn't present visible content itself. E.g., an element that has a third-party editor inside it, but which isn't the editor itself and so doesn't include the editor's content.
+ - `_content`: any element that contains text, images, or any other content displayed to the user.
+ - `_dropdown`
+ - `_field`: a text input element.
+ - `_link`
+ - `_modal`: a popup modal dialog, e.g., a confirmation prompt.
+ - `_placeholder`: a temporary element that appears while content is loading. For example, the elements that are displayed instead of discussions while the discussions are being fetched.
- `_radio`
- - `_content`
+ - `_tab`
-*Note: This list is a work in progress. This list will eventually be the end-all enumeration of all available types.
- I.e., any element that does not end with something in this list is bad form.*
+*Note: If none of the listed types are suitable, please open a merge request to add an appropriate type to the list.*
### Examples
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index 3a96f8204fc..5628ca633f6 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -76,7 +76,7 @@ This was originally implemented in: <https://gitlab.com/gitlab-org/gitlab-foss/m
### Feature tests
-- [Be sure to create all the data the test need before starting exercize](https://gitlab.com/gitlab-org/gitlab-foss/issues/32622#note_31128195): <https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/12059>
+- [Be sure to create all the data the test need before starting exercise](https://gitlab.com/gitlab-org/gitlab-foss/issues/32622#note_31128195): <https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/12059>
- [Bis](https://gitlab.com/gitlab-org/gitlab-foss/issues/34609#note_34048715): <https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/12604>
- [Bis](https://gitlab.com/gitlab-org/gitlab-foss/issues/34698#note_34276286): <https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/12664>
- [Assert against the underlying database state instead of against a page's content](https://gitlab.com/gitlab-org/gitlab-foss/issues/31437): <https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/10934>
@@ -98,6 +98,10 @@ This was originally implemented in: <https://gitlab.com/gitlab-org/gitlab-foss/m
- Memory is through the roof! (TL;DR: Load images but block images requests!): <https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/12003>
+#### Capybara expectation times out
+
+- [Test imports a project (via Sidekiq) that is growing over time, leading to timeouts when the import takes longer than 60 seconds](https://gitlab.com/gitlab-org/gitlab/merge_requests/22599)
+
## Resources
- [Flaky Tests: Are You Sure You Want to Rerun Them?](http://semaphoreci.com/blog/2017/04/20/flaky-tests.html)
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index 8934f0d4b65..56b059063e0 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -315,7 +315,7 @@ export, one is be generated by the babel plugin). The second parameter is the
name of the import you wish to change. The result of the function is a Spy
object which can be treated like any other Jasmine spy object.
-Further documentation on the babel rewire pluign API can be found on
+Further documentation on the babel rewire plugin API can be found on
[its repository Readme doc](https://github.com/speedskater/babel-plugin-rewire#babel-plugin-rewire).
#### Waiting in tests
@@ -532,7 +532,7 @@ In order to ensure that a clean wrapper object and DOM are being used in each te
});
```
-See also the [Vue Test Utils documention on `destroy`](https://vue-test-utils.vuejs.org/api/wrapper/#destroy).
+See also the [Vue Test Utils documentation on `destroy`](https://vue-test-utils.vuejs.org/api/wrapper/#destroy).
#### Migrating flaky Karma tests to Jest
@@ -649,7 +649,7 @@ it('uses some HTML element', () => {
HTML and JSON fixtures are generated from backend views and controllers using RSpec (see `spec/frontend/fixtures/*.rb`).
For each fixture, the content of the `response` variable is stored in the output file.
-This variable gets automagically set if the test is marked as `type: :request` or `type: :controller`.
+This variable gets automatically set if the test is marked as `type: :request` or `type: :controller`.
Fixtures are regenerated using the `bin/rake frontend:fixtures` command but you can also generate them individually,
for example `bin/rspec spec/frontend/fixtures/merge_requests.rb`.
When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`.
diff --git a/doc/development/testing_guide/img/k9s.png b/doc/development/testing_guide/img/k9s.png
index c4b222f0b64..34585b2a43a 100644
--- a/doc/development/testing_guide/img/k9s.png
+++ b/doc/development/testing_guide/img/k9s.png
Binary files differ