From 4793b2818bbac5fb7144e4e878e450584200d8d5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9my=20Coutable?= Date: Mon, 25 Mar 2019 12:27:03 +0100 Subject: Mention review-qa-all in the end-to-end tests documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Rémy Coutable --- doc/development/testing_guide/end_to_end_tests.md | 31 ++++++++++++++++++++--- doc/development/testing_guide/review_apps.md | 4 +-- doc/development/testing_guide/smoke.md | 14 +++++++--- doc/development/testing_guide/testing_levels.md | 4 ++- 4 files changed, 43 insertions(+), 10 deletions(-) diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md index 7010250b33c..9837ea515a3 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -27,10 +27,15 @@ Results are reported in the `#qa-staging` Slack channel. ### Testing code in merge requests +#### Using the `package-and-qa` job + It is possible to run end-to-end tests for a merge request, eventually being run in a pipeline in the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/) project, -by triggering the `package-and-qa` manual action in the `test` stage (which should -be present in a merge request widget, unless the merge request comes from a fork). +by triggering the `package-and-qa` manual action in the `test` stage (not +available for forks). + +**This runs end-to-end tests against a custom Omnibus package built from your +merge request's changes.** Manual action that starts end-to-end tests is also available in merge requests in [Omnibus GitLab][omnibus-gitlab]. @@ -85,7 +90,25 @@ subgraph gitlab-qa pipeline 1. The result of the [GitLab QA pipeline][gitlab-qa-pipelines] is being propagated upstream, through Omnibus, back to the CE / EE merge request. -#### How do I write tests? +#### Using the `review-qa-all` jobs + +On every pipeline during the `test` stage, the `review-qa-smoke` job is +automatically started: it runs the QA smoke suite against the +[Review App][review-apps]. + +You can also manually start the `review-qa-all`: it runs the full QA suite +against the [Review App][review-apps]. + +**This runs end-to-end tests against a Review App based on [the official GitLab +Helm chart][helm-chart], itself deployed with custom +[Cloud Native components][cng] built from your merge request's changes.** + +See [Review Apps][review-apps] for more details about Review Apps. + +[helm-chart]: https://gitlab.com/charts/gitlab/ +[cng]: https://gitlab.com/gitlab-org/build/CNG + +## How do I write tests? In order to write new tests, you first need to learn more about GitLab QA architecture. See the [documentation about it][gitlab-qa-architecture]. @@ -105,9 +128,11 @@ you can find an issue you would like to work on in [omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab [gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa +[gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines [gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md [quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines [quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines +[review-apps]: ./review_apps.md [gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md [gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario [gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index fda3ff57316..ffc71051377 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -90,8 +90,8 @@ subgraph GCP `gitlab-review-apps` project ## QA runs On every [pipeline][gitlab-pipeline] during the `test` stage, the -`review-qa-smoke` job is automatically started: it runs the smoke QA suite. -You can also manually start the `review-qa-all`: it runs the full QA suite. +`review-qa-smoke` job is automatically started: it runs the QA smoke suite. +You can also manually start the `review-qa-all`: it runs the QA full suite. Note that both jobs first wait for the `review-deploy` job to be finished. diff --git a/doc/development/testing_guide/smoke.md b/doc/development/testing_guide/smoke.md index 3360031c220..30d861d7d68 100644 --- a/doc/development/testing_guide/smoke.md +++ b/doc/development/testing_guide/smoke.md @@ -7,13 +7,19 @@ functionality is working. Currently, our suite consists of this basic functionality coverage: -- User Login (Standard Auth) -- Project Creation -- Issue Creation -- Merge Request Creation +- User standard authentication +- SSH Key creation and addition to a user +- Project simple creation +- Project creation with Auto-DevOps enabled +- Issue creation +- Merge Request creation +- Snippet creation Smoke tests have the `:smoke` RSpec metadata. +See [End-to-end Testing](./end_to_end_tests.md) for more details about +end-to-end tests. + --- [Return to Testing documentation](index.md) diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 5d46833a1e2..352651fe91b 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -160,7 +160,7 @@ Every new feature should come with a [test plan]. > See [end-to-end tests](end_to_end_tests.md) for more information. Note that `qa/spec` contains unit tests of the QA framework itself, not to be -confused with the application's [unit tests](#unit-tests) or +confused with the application's [unit tests](#unit-tests) or [end-to-end tests](#black-box-tests-at-the-system-level-aka-end-to-end-tests). [multiple pieces]: ../architecture.md#components @@ -234,6 +234,8 @@ you should write an integration test using Jasmine. [big]: https://twitter.com/timbray/status/822470746773409794 [picture]: https://twitter.com/withzombies/status/829716565834752000 [tests-cost]: https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e +[RSpec]: https://github.com/rspec/rspec-rails#feature-specs +[Capybara]: https://github.com/teamcapybara/capybara --- -- cgit v1.2.1