diff options
Diffstat (limited to 'doc/development/testing_guide/end_to_end')
| -rw-r--r-- | doc/development/testing_guide/end_to_end/index.md | 55 | ||||
| -rw-r--r-- | doc/development/testing_guide/end_to_end/quick_start_guide.md | 17 |
2 files changed, 27 insertions, 45 deletions
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index e2622cec6e2..443b7b06a24 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -9,8 +9,8 @@ together. ## How do we test GitLab? -We use [Omnibus GitLab][omnibus-gitlab] to build GitLab packages and then we -test these packages using the [GitLab QA orchestrator][gitlab-qa] tool, which is +We use [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) to build GitLab packages and then we +test these packages using the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa) tool, which is a black-box testing framework for the API and the UI. ### Testing nightly builds @@ -38,7 +38,7 @@ available for forks). Omnibus package built from your merge request's changes.** Manual action that starts end-to-end tests is also available in merge requests -in [Omnibus GitLab][omnibus-gitlab]. +in [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab). Below you can read more about how to use it and how does it work. @@ -68,14 +68,15 @@ subgraph "gitlab-qa-mirror pipeline" 1. Developer triggers a manual action, that can be found in CE / EE merge requests. This starts a chain of pipelines in multiple projects. -1. The script being executed triggers a pipeline in [Omnibus GitLab Mirror][omnibus-gitlab-mirror] +1. The script being executed triggers a pipeline in + [Omnibus GitLab Mirror](https://gitlab.com/gitlab-org/omnibus-gitlab-mirror) and waits for the resulting status. We call this a _status attribution_. -1. GitLab packages are being built in the [Omnibus GitLab][omnibus-gitlab] +1. GitLab packages are being built in the [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) pipeline. Packages are then pushed to its Container Registry. 1. When packages are ready, and available in the registry, a final step in the - [Omnibus GitLab][omnibus-gitlab] pipeline, triggers a new + [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) pipeline, triggers a new GitLab QA pipeline (those with access can view them at `https://gitlab.com/gitlab-org/gitlab-qa-mirror/pipelines`). It also waits for a resulting status. 1. GitLab QA pulls images from the registry, spins-up containers and runs tests @@ -139,26 +140,23 @@ many of the 10 available jobs that you want to run). On every pipeline during the `test` stage, the `review-qa-smoke` job is automatically started: it runs the QA smoke suite against the -[Review App][review-apps]. +[Review App](../review_apps.md). You can also manually start the `review-qa-all`: it runs the full QA suite -against the [Review App][review-apps]. +against the [Review App](../review_apps.md). **This runs end-to-end tests against a Review App based on [the official GitLab -Helm chart][helm-chart], itself deployed with custom -[Cloud Native components][cng] built from your merge request's changes.** +Helm chart](https://gitlab.com/gitlab-org/charts/gitlab/), itself deployed with custom +[Cloud Native components](https://gitlab.com/gitlab-org/build/CNG) built from your merge request's changes.** -See [Review Apps][review-apps] for more details about Review Apps. - -[helm-chart]: https://gitlab.com/gitlab-org/charts/gitlab/ -[cng]: https://gitlab.com/gitlab-org/build/CNG +See [Review Apps](../review_apps.md) for more details about Review Apps. ## How do I run the tests? If you are not [testing code in a merge request](#testing-code-in-merge-requests), there are two main options for running the tests. If you simply want to run the existing tests against a live GitLab instance or against a pre-built docker image -you can use the [GitLab QA orchestrator][gitlab-qa-readme]. See also [examples +you can use the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md). See also [examples of the test scenarios you can run via the orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#examples). On the other hand, if you would like to run against a local development GitLab @@ -173,12 +171,12 @@ Learn how to perform [tests that require special setup or consideration to run o ## How do I write tests? In order to write new tests, you first need to learn more about GitLab QA -architecture. See the [documentation about it][gitlab-qa-architecture]. +architecture. See the [documentation about it](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md). -Once you decided where to put [test environment orchestration scenarios] and -[instance-level scenarios], take a look at the [GitLab QA README][instance-qa-readme], -the [GitLab QA orchestrator README][gitlab-qa-readme], and [the already existing -instance-level scenarios][instance-level scenarios]. +Once you decided where to put [test environment orchestration scenarios](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/lib/gitlab/qa/scenario) and +[instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features), take a look at the [GitLab QA README](https://gitlab.com/gitlab-org/gitlab/tree/master/qa/README.md), +the [GitLab QA orchestrator README](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md), and [the already existing +instance-level scenarios](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features). Continued reading: @@ -193,18 +191,5 @@ Continued reading: You can ask question in the `#quality` channel on Slack (GitLab internal) or you can find an issue you would like to work on in -[the `gitlab` issue tracker][gitlab-issues], or -[the `gitlab-qa` issue tracker][gitlab-qa-issues]. - -[omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab -[omnibus-gitlab-mirror]: https://gitlab.com/gitlab-org/omnibus-gitlab-mirror -[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa -[gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md -[review-apps]: ../review_apps.md -[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md -[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario -[gitlab-issues]: https://gitlab.com/gitlab-org/gitlab/issues?label_name[]=QA&label_name[]=test -[test environment orchestration scenarios]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/lib/gitlab/qa/scenario -[instance-level scenarios]: https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/specs/features -[instance-qa-readme]: https://gitlab.com/gitlab-org/gitlab/tree/master/qa/README.md -[instance-qa-examples]: https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa +[the `gitlab` issue tracker](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=QA&label_name%5B%5D=test), or +[the `gitlab-qa` issue tracker](https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario). diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index c721c934033..0ae3f375284 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -22,10 +22,10 @@ If you don't exactly understand what we mean by **not everything needs to happen - [2.](#2-test-skeleton) Creating the skeleton of the test file (`*_spec.rb`) - [3.](#3-test-cases-mvc) The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases' logic - [4.](#4-extracting-duplicated-code) Extracting duplicated code into methods -- [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects] +- [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects](page_objects.md) - [6.](#6-optimization) Optimizing the test suite - [7.](#7-resources) Using and implementing resources -- [8.](#8-page-objects) Moving element definitions and methods to [Page Objects] +- [8.](#8-page-objects) Moving element definitions and methods to [Page Objects](page_objects.md) ### 0. Are end-to-end tests needed? @@ -126,7 +126,7 @@ end > Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. > -> The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects] before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. +> The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects](page_objects.md) before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. Below are the steps that the test covers: @@ -294,7 +294,7 @@ In the `before` block we create all the application state needed for the tests t > A project is created in the background by creating the `issue` resource. > -> When creating the [Resources], notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. +> When creating the [Resources](resources.md), notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. > > What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. > @@ -358,7 +358,7 @@ To address point 1, we changed the test implementation from two `it` blocks into **Note:** When writing this document, some code that is now merged to master was not implemented yet, but we left them here for the readers to understand the whole process of end-to-end test creation. -You can think of [Resources] as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. +You can think of [Resources](resources.md) as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc. @@ -468,7 +468,7 @@ Page Objects are used in end-to-end tests for maintenance reasons, where a page' > Page Objects are auto-loaded in the [`qa/qa.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/qa/qa.rb) file and available in all the test files (`*_spec.rb`). -Take a look at the [Page Objects] documentation. +Take a look at the [Page Objects](page_objects.md) documentation. Now, let's go back to our example. @@ -571,7 +571,7 @@ The `text_of_labels_block` method is a simple method that returns the `:labels_b #### Updates in the view (*.html.haml) and `dropdowns_helper.rb` files -Now let's change the view and the `dropdowns_helper` files to add the selectors that relate to the [Page Objects]. +Now let's change the view and the `dropdowns_helper` files to add the selectors that relate to the [Page Objects](page_objects.md). In [`app/views/shared/issuable/_sidebar.html.haml:105`](https://gitlab.com/gitlab-org/gitlab/blob/7ca12defc7a965987b162a6ebef302f95dc8867f/app/views/shared/issuable/_sidebar.html.haml#L105), add a `data: { qa_selector: 'edit_link_labels' }` data attribute. @@ -619,6 +619,3 @@ This method receives an element (`name`) and the `keys` that it will send to tha As you might remember, in the Issue Page Object we call this method like this: `send_keys_to_element(:dropdown_input_field, [label, :enter])`. With that, you should be able to start writing end-to-end tests yourself. *Congratulations!* - -[Page Objects]: page_objects.md -[Resources]: resources.md |