Merge branch 'improve-review-apps-jobs-structure' into 'master'

Organize better Review Apps and QA jobs

See merge request gitlab-org/gitlab-ce!27509

2 files changed, 82 insertions, 80 deletions

diff --git a/doc/development/testing_guide/img/review_apps_cicd_architecture.png b/doc/development/testing_guide/img/review_apps_cicd_architecture.png
index 87e472076f3..1ee28d3db91 100644
--- a/doc/development/testing_guide/img/review_apps_cicd_architecture.png
+++ b/doc/development/testing_guide/img/review_apps_cicd_architecture.png
diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md
index 55ca502f84a..7ad33f05f77 100644
--- a/doc/development/testing_guide/review_apps.md
+++ b/doc/development/testing_guide/review_apps.md
@@ -6,7 +6,7 @@ Review Apps are automatically deployed by each pipeline, both in
 
 ## How does it work?
 
-### CD/CD architecture diagram
+### CI/CD architecture diagram
 
 ![Review Apps CI/CD architecture](img/review_apps_cicd_architecture.png)
 
@@ -14,23 +14,29 @@ Review Apps are automatically deployed by each pipeline, both in
 <summary>Show mermaid source</summary>
 <pre>
 graph TD
-    B1 -.->|2. once gitlab:assets:compile is done,<br />triggers a CNG-mirror pipeline and wait for it to be done| A2
-    C1 -.->|2. once review-build-cng is done,<br />Helm deploys the Review App using the Cloud<br/>Native images built by the CNG-mirror pipeline| A3
-
-subgraph gitlab-ce/ee `test` stage
-    A1[gitlab:assets:compile]
-    B1[review-build-cng] -->|1. wait for| A1
-    C1[review-deploy] -->|1. wait for| B1
-    D1[review-qa-smoke] -->|1. wait for| C1
-    D1[review-qa-smoke] -.->|2. once review-deploy is done| E1>gitlab-qa runs the smoke<br/>suite against the Review App]
+    build-qa-image -.->|once the `prepare` stage is done| gitlab:assets:compile
+    review-build-cng -->|triggers a CNG-mirror pipeline and wait for it to be done| CNG-mirror
+    review-build-cng -.->|once the `test` stage is done| review-deploy
+    review-deploy -.->|once the `review` stage is done| review-qa-smoke
+
+subgraph 1. gitlab-ce/ee `prepare` stage
+    build-qa-image
     end
 
-subgraph CNG-mirror pipeline
-    A2>Cloud Native images are built];
+subgraph 2. gitlab-ce/ee `test` stage
+    gitlab:assets:compile -->|plays dependent job once done| review-build-cng
     end
 
-subgraph GCP `gitlab-review-apps` project
-    A3>"Cloud Native images are deployed to the<br />`review-apps-ce` or `review-apps-ee` Kubernetes (GKE) cluster"];
+subgraph 3. gitlab-ce/ee `review` stage
+    review-deploy["review-deploy<br /><br />Helm deploys the Review App using the Cloud<br/>Native images built by the CNG-mirror pipeline.<br /><br />Cloud Native images are deployed to the `review-apps-ce` or `review-apps-ee`<br />Kubernetes (GKE) cluster, in the GCP `gitlab-review-apps` project."]
+    end
+
+subgraph 4. gitlab-ce/ee `qa` stage
+    review-qa-smoke[review-qa-smoke<br /><br />gitlab-qa runs the smoke suite against the Review App.]
+    end
+
+subgraph CNG-mirror pipeline
+    CNG-mirror>Cloud Native images are built];
     end
 </pre>
 </details>
@@ -38,40 +44,37 @@ subgraph GCP `gitlab-review-apps` project
 ### Detailed explanation
 
 1. On every [pipeline][gitlab-pipeline] during the `test` stage, the
-  [`review-build-cng`][review-build-cng] and
-  [`review-deploy`][review-deploy] jobs are automatically started.
-    - The [`review-deploy`][review-deploy] job waits for the
-      [`review-build-cng`][review-build-cng] job to finish.
-    - The [`review-build-cng`][review-build-cng] job waits for the
-      [`gitlab:assets:compile`][gitlab:assets:compile] job to finish since the
-      [`CNG-mirror`][cng-mirror] pipeline triggered in the following step depends on it.
-1. Once the [`gitlab:assets:compile`][gitlab:assets:compile] job is done,
-  [`review-build-cng`][review-build-cng] [triggers a pipeline][cng-pipeline]
-  in the [`CNG-mirror`][cng-mirror] project.
-    - The [`CNG-mirror`][cng-pipeline] pipeline creates the Docker images of
-      each component (e.g. `gitlab-rails-ee`, `gitlab-shell`, `gitaly` etc.)
-      based on the commit from the [GitLab pipeline][gitlab-pipeline] and store
-      them in its [registry][cng-mirror-registry].
-    - We use the [`CNG-mirror`][cng-mirror] project so that the `CNG`, (**C**loud
-      **N**ative **G**itLab), project's registry is not overloaded with a
-      lot of transient Docker images.
-1. Once the [`review-build-cng`][review-build-cng] job is done, the
-  [`review-deploy`][review-deploy] job deploys the Review App using
-  [the official GitLab Helm chart][helm-chart] to the
-  [`review-apps-ce`][review-apps-ce] / [`review-apps-ee`][review-apps-ee]
-  Kubernetes cluster on GCP.
-    - The actual scripts used to deploy the Review App can be found at
-      [`scripts/review_apps/review-apps.sh`][review-apps.sh].
-    - These scripts are basically
-      [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the
-      default CNG images are overridden with the images built and stored in the
-      [`CNG-mirror` project's registry][cng-mirror-registry].
-    - Since we're using [the official GitLab Helm chart][helm-chart], this means
-      you get a dedicated environment for your branch that's very close to what
-      it would look in production.
+   [`gitlab:assets:compile`][gitlab:assets:compile] job is automatically started.
+   - Once it's done, it starts the [`review-build-cng`][review-build-cng]
+     manual job since the [`CNG-mirror`][cng-mirror] pipeline triggered in the
+     following step depends on it.
+1. The [`review-build-cng`][review-build-cng] job [triggers a pipeline][cng-mirror-pipeline]
+   in the [`CNG-mirror`][cng-mirror] project.
+   - The [`CNG-mirror`][cng-mirror-pipeline] pipeline creates the Docker images of
+     each component (e.g. `gitlab-rails-ee`, `gitlab-shell`, `gitaly` etc.)
+     based on the commit from the [GitLab pipeline][gitlab-pipeline] and stores
+     them in its [registry][cng-mirror-registry].
+   - We use the [`CNG-mirror`][cng-mirror] project so that the `CNG`, (**C**loud
+     **N**ative **G**itLab), project's registry is not overloaded with a
+     lot of transient Docker images.
+   - Note that the official CNG images are built by the `cloud-native-image`
+     job, which runs only for tags, and triggers itself a [`CNG`][cng] pipeline.
+1. Once the `test` stage is done, the [`review-deploy`][review-deploy] job
+   deploys the Review App using [the official GitLab Helm chart][helm-chart] to
+   the [`review-apps-ce`][review-apps-ce] / [`review-apps-ee`][review-apps-ee]
+   Kubernetes cluster on GCP.
+   - The actual scripts used to deploy the Review App can be found at
+     [`scripts/review_apps/review-apps.sh`][review-apps.sh].
+   - These scripts are basically
+     [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the
+     default CNG images are overridden with the images built and stored in the
+     [`CNG-mirror` project's registry][cng-mirror-registry].
+   - Since we're using [the official GitLab Helm chart][helm-chart], this means
+     you get a dedicated environment for your branch that's very close to what
+     it would look in production.
 1. Once the [`review-deploy`][review-deploy] job succeeds, you should be able to
-  use your Review App thanks to the direct link to it from the MR widget. To log
-  into the Review App, see "Log into my Review App?" below.
+   use your Review App thanks to the direct link to it from the MR widget. To log
+   into the Review App, see "Log into my Review App?" below.
 
 **Additional notes:**
 
@@ -82,71 +85,69 @@ subgraph GCP `gitlab-review-apps` project
 - If the Review App deployment fails, you can simply retry it (there's no need
   to run the [`review-stop`][gitlab-ci-yml] job first).
 - The manual [`review-stop`][gitlab-ci-yml] in the `test` stage can be used to
-  stop a Review App manually, and is also started by GitLab once a branch is
-  deleted.
-- Review Apps are cleaned up regularly using a pipeline schedule that runs
+  stop a Review App manually, and is also started by GitLab once a merge
+  request's branch is deleted after being merged.
+- Review Apps are cleaned up regularly via a pipeline schedule that runs
   the [`schedule:review-cleanup`][gitlab-ci-yml] job.
 
 ## QA runs
 
-On every [pipeline][gitlab-pipeline] during the `test` stage, the
-`review-qa-smoke` job is automatically started: it runs the QA smoke suite.
-You can also manually start the `review-qa-all`: it runs the QA full suite.
+On every [pipeline][gitlab-pipeline] in the `qa` stage (which comes after the
+`review` stage), the `review-qa-smoke` job is automatically started and it runs
+the QA smoke suite.
 
-Note that both jobs first wait for the `review-deploy` job to be finished.
+You can also manually start the `review-qa-all`: it runs the full QA suite.
 
 ## Performance Metrics
 
-On every [pipeline][gitlab-pipeline] during the `test` stage, the
+On every [pipeline][gitlab-pipeline] in the `qa` stage, the
 `review-performance` job is automatically started: this job does basic
-browser performance testing using [Sitespeed.io Container](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html) .
+browser performance testing using a
+[Sitespeed.io Container](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html).
 
-This job waits for the `review-deploy` job to be finished.
+## How to:
 
-## How to?
-
-### Log into my Review App?
+### Log into my Review App
 
 The default username is `root` and its password can be found in the 1Password
 secure note named **gitlab-{ce,ee} Review App's root password**.
 
-### Enable a feature flag for my Review App?
+### Enable a feature flag for my Review App
 
 1. Open your Review App and log in as documented above.
 1. Create a personal access token.
 1. Enable the feature flag using the [Feature flag API](../../api/features.md).
 
-### Find my Review App slug?
+### Find my Review App slug
 
 1. Open the `review-deploy` job.
 1. Look for `Checking for previous deployment of review-*`.
 1. For instance for `Checking for previous deployment of review-qa-raise-e-12chm0`,
-  your Review App slug would be `review-qa-raise-e-12chm0` in this case.
+   your Review App slug would be `review-qa-raise-e-12chm0` in this case.
 
-### Run a Rails console?
+### Run a Rails console
 
 1. [Filter Workloads by your Review App slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps)
-  , e.g. `review-29951-issu-id2qax`.
-1. Find and open the `task-runner` Deployment, e.g. `review-29951-issu-id2qax-task-runner`.
-1. Click on the Pod in the "Managed pods" section, e.g. `review-29951-issu-id2qax-task-runner-d5455cc8-2lsvz`.
+  , e.g. `review-qa-raise-e-12chm0`.
+1. Find and open the `task-runner` Deployment, e.g. `review-qa-raise-e-12chm0-task-runner`.
+1. Click on the Pod in the "Managed pods" section, e.g. `review-qa-raise-e-12chm0-task-runner-d5455cc8-2lsvz`.
 1. Click on the `KUBECTL` dropdown, then `Exec` -> `task-runner`.
 1. Replace `-c task-runner -- ls` with `-it -- gitlab-rails console` from the
-  default command or
-  - Run `kubectl exec --namespace review-apps-ce review-29951-issu-id2qax-task-runner-d5455cc8-2lsvz -it -- gitlab-rails console`
-    and
-  - Replace `review-apps-ce` with `review-apps-ee` if the Review App
-    is running EE, and
-  - Replace `review-29951-issu-id2qax-task-runner-d5455cc8-2lsvz`
-    with your Pod's name.
+   default command or
+   - Run `kubectl exec --namespace review-apps-ce review-qa-raise-e-12chm0-task-runner-d5455cc8-2lsvz -it -- gitlab-rails console` and
+     - Replace `review-apps-ce` with `review-apps-ee` if the Review App
+       is running EE, and
+     - Replace `review-qa-raise-e-12chm0-task-runner-d5455cc8-2lsvz`
+       with your Pod's name.
 
-### Dig into a Pod's logs?
+### Dig into a Pod's logs
 
-1. [Filter Workloads by your Review App slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps)
-  , e.g. `review-1979-1-mul-dnvlhv`.
+1. [Filter Workloads by your Review App slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps),
+   e.g. `review-qa-raise-e-12chm0`.
 1. Find and open the `migrations` Deployment, e.g.
-  `review-1979-1-mul-dnvlhv-migrations.1`.
+   `review-qa-raise-e-12chm0-migrations.1`.
 1. Click on the Pod in the "Managed pods" section, e.g.
-  `review-1979-1-mul-dnvlhv-migrations.1-nqwtx`.
+   `review-qa-raise-e-12chm0-migrations.1-nqwtx`.
 1. Click on the `Container logs` link.
 
 ## Frequently Asked Questions
@@ -182,7 +183,8 @@ find a way to limit it to only us.**
 [review-build-cng]: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/149511623
 [review-deploy]: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/149511624
 [cng-mirror]: https://gitlab.com/gitlab-org/build/CNG-mirror
-[cng-pipeline]: https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/44364657
+[cng]: https://gitlab.com/gitlab-org/build/CNG
+[cng-mirror-pipeline]: https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/44364657
 [cng-mirror-registry]: https://gitlab.com/gitlab-org/build/CNG-mirror/container_registry
 [helm-chart]: https://gitlab.com/charts/gitlab/
 [review-apps-ce]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-a/review-apps-ce?project=gitlab-review-apps