Add a manual job to run QA against a Review App49570-add-review_qa-job

1. Renames review to review-deploy.
2. Renames stop_review to review-stop.
3. Adds a build-qa-image job to the prepare stage and save the QA
  Docker image as artifact (example:
  https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/120967019).
4. Adds a manual review-qa job to the test stage to run QA tests
  against the Review App (example:
  https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/120967238).
5. Make the review-deploy job plays the review-qa job as soon as the
  Review App is deployed (example:
  https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/120988775).
6. Updates the Review Apps documentation accordingly.

Signed-off-by: Rémy Coutable <remy@rymai.me>

1 files changed, 49 insertions, 35 deletions

diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md
index 1830641431e..a6ed9e85a41 100644
--- a/doc/development/testing_guide/review_apps.md
+++ b/doc/development/testing_guide/review_apps.md
@@ -7,31 +7,36 @@ Review Apps are automatically deployed by each pipeline, both in
 ## How does it work?
 
 1. On every [pipeline][gitlab-pipeline] during the `test` stage, the
-  [`review` job][review-job] is automatically started.
-1. The `review` job [triggers a pipeline][cng-pipeline] in the
-  [`CNG-mirror`][cng-mirror] project.
-    - We use the `CNG-mirror` project so that the `CNG`, (**C**loud **N**ative
-      **G**itLab), project's registry is not overloaded with a lot of transient
-      Docker images.
-1. The `CNG-mirror` pipeline creates the Docker images of each component (e.g.
-  `gitlab-rails-ee`, `gitlab-shell`, `gitaly` etc.) based on the commit from the
-  [GitLab pipeline][gitlab-pipeline] and store them in its
-  [registry][cng-mirror-registry].
-1. Once all images are built, the Review App is deployed using
-  [the official GitLab Helm chart][helm-chart] to the
-  [`review-apps-ee` Kubernetes cluster on GCP][review-apps-ee]
-    - The actual scripts used to deploy the Review App can be found at
-      [`scripts/review_apps/review-apps.sh`][review-apps.sh]
-    - These scripts are basically
-      [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the
-      default CNG images are overridden with the images built and stored in the
-      [`CNG-mirror` project's registry][cng-mirror-registry].
-    - Since we're using [the official GitLab Helm chart][helm-chart], this means
-      you get a dedicated environment for your branch that's very close to what it
-      would look in production.
-1. Once the `review` job succeeds, you should be able to use your Review App
-  thanks to the direct link to it from the MR widget. The default username is
-  `root` and its password can be found in the 1Password secure note named
+  [`review-deploy`][review-deploy-job] job is automatically started.
+1. The `review-deploy` job:
+    1. Waits for the `gitlab:assets:compile` job to finish since the
+      [`CNG-mirror`][cng-mirror] pipeline triggerred in the following step
+      depends on it.
+    1. [Triggers a pipeline][cng-pipeline] in the [`CNG-mirror`][cng-mirror]
+      project.
+        - We use the `CNG-mirror` project so that the `CNG`, (**C**loud
+          **N**ative **G**itLab), project's registry is not overloaded with a
+          lot of transient Docker images.
+        - The `CNG-mirror` pipeline creates the Docker images of each component
+          (e.g. `gitlab-rails-ee`, `gitlab-shell`, `gitaly` etc.) based on the
+          commit from the [GitLab pipeline][gitlab-pipeline] and store them in
+          its [registry][cng-mirror-registry].
+    1. Once all images are built by [`CNG-mirror`][cng-mirror], the Review App
+      is deployed using [the official GitLab Helm chart][helm-chart] to the
+      [`review-apps-ce`][review-apps-ce] / [`review-apps-ee`][review-apps-ee]
+      Kubernetes cluster on GCP.
+        - The actual scripts used to deploy the Review App can be found at
+          [`scripts/review_apps/review-apps.sh`][review-apps.sh].
+        - These scripts are basically
+          [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the
+          default CNG images are overridden with the images built and stored in the
+          [`CNG-mirror` project's registry][cng-mirror-registry].
+        - Since we're using [the official GitLab Helm chart][helm-chart], this means
+          you get a dedicated environment for your branch that's very close to what
+          it would look in production.
+1. Once the `review-deploy` job succeeds, you should be able to use your Review
+  App thanks to the direct link to it from the MR widget. The default username
+  is `root` and its password can be found in the 1Password secure note named
   **gitlab-{ce,ee} Review App's root password** (note that there's currently
   [a bug where the default password seems to be overridden][password-bug]).
 
@@ -39,16 +44,23 @@ Review Apps are automatically deployed by each pipeline, both in
 
 - The Kubernetes cluster is connected to the `gitlab-{ce,ee}` projects using
   [GitLab's Kubernetes integration][gitlab-k8s-integration]. This basically
-  allows to have a link to the Review App directly from the merge request widget.
-- The manual `stop_review` in the `test` stage can be used to stop a Review App
-  manually, and is also started by GitLab once a branch is deleted.
-- Review Apps are cleaned up regularly using a pipeline schedule that runs
-  the [`scripts/review_apps/automated_cleanup.rb`][automated_cleanup.rb] script.
+  allows to have a link to the Review App directly from the merge request
+  widget.
 - If the Review App deployment fails, you can simply retry it (there's no need
-  to run the `stop_review` job first).
-- If you're unable to log in using the `root` username and password, you may
-  encounter [this bug][password-bug]. Stop the Review App via the `stop_review`
-  manual job and then retry the `review` job to redeploy the Review App.
+  to run the [`review-stop`][gitlab-ci-yml] job first).
+- The manual [`review-stop`][gitlab-ci-yml] in the `test` stage can be used to
+  stop a Review App manually, and is also started by GitLab once a branch is
+  deleted.
+- Review Apps are cleaned up regularly using a pipeline schedule that runs
+  the [`schedule:review-cleanup`][gitlab-ci-yml] job.
+
+## QA runs
+
+On every [pipeline][gitlab-pipeline] during the `test` stage, the
+`review-qa-smoke` job is automatically started: it runs the smoke QA suite.
+You can also manually start the `review-qa-all`: it runs the full QA suite.
+
+Note that both jobs first wait for the `review-deploy` job to be finished.
 
 ## Frequently Asked Questions
 
@@ -74,15 +86,17 @@ find a way to limit it to only us.**
   > This isn't enabled for forks.
 
 [gitlab-pipeline]: https://gitlab.com/gitlab-org/gitlab-ce/pipelines/35850709
-[review-job]: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/118076368
+[review-deploy-job]: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/118076368
 [cng-mirror]: https://gitlab.com/gitlab-org/build/CNG-mirror
 [cng-pipeline]: https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/35883435
 [cng-mirror-registry]: https://gitlab.com/gitlab-org/build/CNG-mirror/container_registry
 [helm-chart]: https://gitlab.com/charts/gitlab/
+[review-apps-ce]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-a/review-apps-ce?project=gitlab-review-apps
 [review-apps-ee]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-b/review-apps-ee?project=gitlab-review-apps
 [review-apps.sh]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/review-apps.sh
 [automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb
 [Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml
+[gitlab-ci-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml
 [gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html
 [password-bug]: https://gitlab.com/gitlab-org/gitlab-ce/issues/53621