diff options
Diffstat (limited to 'doc/development/testing_guide/end_to_end')
5 files changed, 24 insertions, 22 deletions
diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index fc00fcea67e..e060312e05f 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -14,7 +14,7 @@ Now, realize that almost all tests need the user to be logged in, and that we ne Now, multiply the number of tests per 2 seconds, and as your test suite grows, the time to run it grows with it, and this is not sustainable. -An alternative to perform a login in a cheaper way would be having an endpoint (available only for testing) where we could pass the user's credentials as encrypted values as query strings, and then we would be redirected to the logged in home page if the credentials are valid. Let's say that, on average, this process takes only 200 miliseconds. +An alternative to perform a login in a cheaper way would be having an endpoint (available only for testing) where we could pass the user's credentials as encrypted values as query strings, and then we would be redirected to the logged in home page if the credentials are valid. Let's say that, on average, this process takes only 200 milliseconds. You see the point right? diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 19885f5756f..96141a5d68d 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -15,27 +15,27 @@ a black-box testing framework for the API and the UI. ### Testing nightly builds -We run scheduled pipeline each night to test nightly builds created by Omnibus. +We run scheduled pipelines each night to test nightly builds created by Omnibus. You can find these nightly pipelines at `https://gitlab.com/gitlab-org/quality/nightly/pipelines` (need Developer access permissions). Results are reported in the `#qa-nightly` Slack channel. ### Testing staging -We run scheduled pipeline each night to test staging. +We run scheduled pipelines each night to test staging. You can find these nightly pipelines at `https://gitlab.com/gitlab-org/quality/staging/pipelines` -(need developer access permissions). Results are reported in the `#qa-staging` Slack channel. +(need Developer access permissions). Results are reported in the `#qa-staging` Slack channel. ### Testing code in merge requests -#### Using the `package-and-qa-manual` job +#### Using the `package-and-qa` job It is possible to run end-to-end tests for a merge request, eventually being run in a pipeline in the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/) project, -by triggering the `package-and-qa-manual` manual action in the `test` stage (not +by triggering the `package-and-qa` manual action in the `test` stage (not available for forks). -**This runs end-to-end tests against a custom Omnibus package built from your -merge request's changes.** +**This runs end-to-end tests against a custom CE and EE (with an Ultimate license) +Omnibus package built from your merge request's changes.** Manual action that starts end-to-end tests is also available in merge requests in [Omnibus GitLab][omnibus-gitlab]. @@ -53,7 +53,7 @@ graph LR B2[`Trigger-qa` stage<br>`Trigger:qa-test` job] -.->|2. Triggers a gitlab-qa pipeline and wait for it to be done| A3 subgraph "gitlab-foss/gitlab pipeline" - A1[`test` stage<br>`package-and-qa-manual` job] + A1[`test` stage<br>`package-and-qa` job] end subgraph "omnibus-gitlab pipeline" @@ -61,7 +61,7 @@ subgraph "omnibus-gitlab pipeline" end subgraph "gitlab-qa pipeline" - A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa-manual` job<br>and post the result on the original commit tested| A1 + A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa` job<br>and post the result on the original commit tested| A1 end ``` diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md index 554995fa2e2..f1e0de0c792 100644 --- a/doc/development/testing_guide/end_to_end/page_objects.md +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -40,7 +40,7 @@ the time it would take to build packages and test everything. That is why when someone changes `t.text_field :login` to `t.text_field :username` in the _new session_ view we won't know about this change until our GitLab QA nightly pipeline fails, or until someone triggers -`package-and-qa-manual` action in their merge request. +`package-and-qa` action in their merge request. Obviously such a change would break all tests. We call this problem a _fragile tests problem_. @@ -171,8 +171,8 @@ and we should prefer the `data-qa-selector` method of definition. > Introduced in GitLab 12.5 -A common occurrence in automated testing is selecting a single "one-of-many" element. -In a list of several items, how do you differentiate what you are selecting on? +A common occurrence in automated testing is selecting a single "one-of-many" element. +In a list of several items, how do you differentiate what you are selecting on? The most common workaround for this is via text matching. Instead, a better practice is by matching on that specific element by a unique identifier, rather than by text. diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index fb820ac22a2..be00129a2bc 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -445,7 +445,7 @@ end By defining the `resource_web_url(resource)` method, we override the one from the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb#L44) module. We do that to avoid failing the test due to this particular resource not exposing a `web_url` property. -By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the publich API, we raise a `NotImplementedError` instead. +By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the public API, we raise a `NotImplementedError` instead. By defining the `api_post_path` method, we allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project. diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index 9088e9e9bfb..7f4616f394b 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -54,18 +54,20 @@ We follow a simple formula roughly based on hungarian notation. *Formula*: `element :<descriptor>_<type>` - `descriptor`: The natural-language description of what the element is. On the login page, this could be `username`, or `password`. -- `type`: A physical control on the page that can be seen by a user. +- `type`: A generic control on the page that can be seen by a user. - `_button` - - `_link` - - `_tab` - - `_dropdown` - - `_field` - `_checkbox` + - `_container`: an element that includes other elements, but doesn't present visible content itself. E.g., an element that has a third-party editor inside it, but which isn't the editor itself and so doesn't include the editor's content. + - `_content`: any element that contains text, images, or any other content displayed to the user. + - `_dropdown` + - `_field`: a text input element. + - `_link` + - `_modal`: a popup modal dialog, e.g., a confirmation prompt. + - `_placeholder`: a temporary element that appears while content is loading. For example, the elements that are displayed instead of discussions while the discussions are being fetched. - `_radio` - - `_content` + - `_tab` -*Note: This list is a work in progress. This list will eventually be the end-all enumeration of all available types. - I.e., any element that does not end with something in this list is bad form.* +*Note: If none of the listed types are suitable, please open a merge request to add an appropriate type to the list.* ### Examples |