summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorLin Jen-Shin <godfat@godfat.org>2019-03-25 14:19:22 +0000
committerLin Jen-Shin <godfat@godfat.org>2019-03-25 14:19:22 +0000
commit6f15d027d37d5b08fd70d0c8393d26e5d63b632f (patch)
treed26b172a607fd7ab11707ec6720dc61e364ada1e
parent1f1e965be6bbc89f71984ce07fbf1b98d606c184 (diff)
parent4793b2818bbac5fb7144e4e878e450584200d8d5 (diff)
downloadgitlab-ce-6f15d027d37d5b08fd70d0c8393d26e5d63b632f.tar.gz
Merge branch 'docs-improve-end-to-ends-tests-to-mention-review-qa-all-job' into 'master'
Mention review-qa-all in the end-to-end tests documentation See merge request gitlab-org/gitlab-ce!26541
-rw-r--r--doc/development/testing_guide/end_to_end_tests.md31
-rw-r--r--doc/development/testing_guide/review_apps.md4
-rw-r--r--doc/development/testing_guide/smoke.md14
-rw-r--r--doc/development/testing_guide/testing_levels.md4
4 files changed, 43 insertions, 10 deletions
diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md
index 7010250b33c..9837ea515a3 100644
--- a/doc/development/testing_guide/end_to_end_tests.md
+++ b/doc/development/testing_guide/end_to_end_tests.md
@@ -27,10 +27,15 @@ Results are reported in the `#qa-staging` Slack channel.
### Testing code in merge requests
+#### Using the `package-and-qa` job
+
It is possible to run end-to-end tests for a merge request, eventually being run in
a pipeline in the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/) project,
-by triggering the `package-and-qa` manual action in the `test` stage (which should
-be present in a merge request widget, unless the merge request comes from a fork).
+by triggering the `package-and-qa` manual action in the `test` stage (not
+available for forks).
+
+**This runs end-to-end tests against a custom Omnibus package built from your
+merge request's changes.**
Manual action that starts end-to-end tests is also available in merge requests
in [Omnibus GitLab][omnibus-gitlab].
@@ -85,7 +90,25 @@ subgraph gitlab-qa pipeline
1. The result of the [GitLab QA pipeline][gitlab-qa-pipelines] is being
propagated upstream, through Omnibus, back to the CE / EE merge request.
-#### How do I write tests?
+#### Using the `review-qa-all` jobs
+
+On every pipeline during the `test` stage, the `review-qa-smoke` job is
+automatically started: it runs the QA smoke suite against the
+[Review App][review-apps].
+
+You can also manually start the `review-qa-all`: it runs the full QA suite
+against the [Review App][review-apps].
+
+**This runs end-to-end tests against a Review App based on [the official GitLab
+Helm chart][helm-chart], itself deployed with custom
+[Cloud Native components][cng] built from your merge request's changes.**
+
+See [Review Apps][review-apps] for more details about Review Apps.
+
+[helm-chart]: https://gitlab.com/charts/gitlab/
+[cng]: https://gitlab.com/gitlab-org/build/CNG
+
+## How do I write tests?
In order to write new tests, you first need to learn more about GitLab QA
architecture. See the [documentation about it][gitlab-qa-architecture].
@@ -105,9 +128,11 @@ you can find an issue you would like to work on in
[omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab
[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa
+[gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines
[gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md
[quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines
[quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines
+[review-apps]: ./review_apps.md
[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md
[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario
[gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test
diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md
index fda3ff57316..ffc71051377 100644
--- a/doc/development/testing_guide/review_apps.md
+++ b/doc/development/testing_guide/review_apps.md
@@ -90,8 +90,8 @@ subgraph GCP `gitlab-review-apps` project
## QA runs
On every [pipeline][gitlab-pipeline] during the `test` stage, the
-`review-qa-smoke` job is automatically started: it runs the smoke QA suite.
-You can also manually start the `review-qa-all`: it runs the full QA suite.
+`review-qa-smoke` job is automatically started: it runs the QA smoke suite.
+You can also manually start the `review-qa-all`: it runs the QA full suite.
Note that both jobs first wait for the `review-deploy` job to be finished.
diff --git a/doc/development/testing_guide/smoke.md b/doc/development/testing_guide/smoke.md
index 3360031c220..30d861d7d68 100644
--- a/doc/development/testing_guide/smoke.md
+++ b/doc/development/testing_guide/smoke.md
@@ -7,13 +7,19 @@ functionality is working.
Currently, our suite consists of this basic functionality coverage:
-- User Login (Standard Auth)
-- Project Creation
-- Issue Creation
-- Merge Request Creation
+- User standard authentication
+- SSH Key creation and addition to a user
+- Project simple creation
+- Project creation with Auto-DevOps enabled
+- Issue creation
+- Merge Request creation
+- Snippet creation
Smoke tests have the `:smoke` RSpec metadata.
+See [End-to-end Testing](./end_to_end_tests.md) for more details about
+end-to-end tests.
+
---
[Return to Testing documentation](index.md)
diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md
index 5d46833a1e2..352651fe91b 100644
--- a/doc/development/testing_guide/testing_levels.md
+++ b/doc/development/testing_guide/testing_levels.md
@@ -160,7 +160,7 @@ Every new feature should come with a [test plan].
> See [end-to-end tests](end_to_end_tests.md) for more information.
Note that `qa/spec` contains unit tests of the QA framework itself, not to be
-confused with the application's [unit tests](#unit-tests) or
+confused with the application's [unit tests](#unit-tests) or
[end-to-end tests](#black-box-tests-at-the-system-level-aka-end-to-end-tests).
[multiple pieces]: ../architecture.md#components
@@ -234,6 +234,8 @@ you should write an integration test using Jasmine.
[big]: https://twitter.com/timbray/status/822470746773409794
[picture]: https://twitter.com/withzombies/status/829716565834752000
[tests-cost]: https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e
+[RSpec]: https://github.com/rspec/rspec-rails#feature-specs
+[Capybara]: https://github.com/teamcapybara/capybara
---