Add latest changes from gitlab-org/gitlab@14-0-stable-eev14.0.0-rc42

2 files changed, 391 insertions, 6 deletions

diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md
index a20fa1f8aa9..7a57d8abf0d 100644
--- a/doc/ci/jobs/index.md
+++ b/doc/ci/jobs/index.md
@@ -1,6 +1,6 @@
 ---
 stage: Verify
-group: Continuous Integration
+group: Pipeline Execution
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
@@ -29,7 +29,7 @@ jobs, where each of the jobs executes a different command.
 Of course a command can execute code directly (`./configure;make;make install`)
 or run a script (`test.sh`) in the repository.
 
-Jobs are picked up by [runners](../runners/README.md) and executed within the
+Jobs are picked up by [runners](../runners/README.md) and executed in the
 environment of the runner. What is important is that each job is run
 independently from each other.
 
@@ -136,7 +136,7 @@ In the pipeline, the result is a group named `build ruby` with three jobs:
 The jobs are ordered by comparing the numbers from left to right. You
 usually want the first number to be the index and the second number to be the total.
 
-[This regular expression](https://gitlab.com/gitlab-org/gitlab/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99)
+[This regular expression](https://gitlab.com/gitlab-org/gitlab/-/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99)
 evaluates the job names: `([\b\s:]+((\[.*\])|(\d+[\s:\/\\]+\d+)))+\s*\z`.
 One or more `: [...]`, `X Y`, `X/Y`, or `X\Y` sequences are removed from the **end**
 of job names only. Matching substrings found at the beginning or in the middle of
diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md
index 6e9197c223b..d7e192bbfda 100644
--- a/doc/ci/jobs/job_control.md
+++ b/doc/ci/jobs/job_control.md
@@ -1,6 +1,6 @@
 ---
 stage: Verify
-group: Continuous Integration
+group: Pipeline Execution
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
@@ -19,6 +19,277 @@ To configure a job to be included or excluded from certain pipelines, you can us
 Use [`needs`](../yaml/README.md#needs) to configure a job to run as soon as the
 earlier jobs it depends on finish running.
 
+## Specify when jobs run with `rules`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3.
+
+Use [`rules`](../yaml/README.md#rules) to include or exclude jobs in pipelines.
+
+Rules are evaluated in order until the first match. When a match is found, the job
+is either included or excluded from the pipeline, depending on the configuration.
+See the [`rules`](../yaml/README.md#rules) reference for more details.
+
+Future keyword improvements are being discussed in our [epic for improving `rules`](https://gitlab.com/groups/gitlab-org/-/epics/2783),
+where anyone can add suggestions or requests.
+
+### `rules` examples
+
+The following example uses `if` to define that the job runs in only two specific cases:
+
+```yaml
+job:
+  script: echo "Hello, Rules!"
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+      when: manual
+      allow_failure: true
+    - if: '$CI_PIPELINE_SOURCE == "schedule"'
+```
+
+- If the pipeline is for a merge request, the first rule matches, and the job
+  is added to the [merge request pipeline](../merge_request_pipelines/index.md)
+  with attributes of:
+  - `when: manual` (manual job)
+  - `allow_failure: true` (the pipeline continues running even if the manual job is not run)
+- If the pipeline is **not** for a merge request, the first rule doesn't match, and the
+  second rule is evaluated.
+- If the pipeline is a scheduled pipeline, the second rule matches, and the job
+  is added to the scheduled pipeline. No attributes were defined, so it is added
+  with:
+  - `when: on_success` (default)
+  - `allow_failure: false` (default)
+- In **all other cases**, no rules match, so the job is **not** added to any other pipeline.
+
+Alternatively, you can define a set of rules to exclude jobs in a few cases, but
+run them in all other cases:
+
+```yaml
+job:
+  script: echo "Hello, Rules!"
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+      when: never
+    - if: '$CI_PIPELINE_SOURCE == "schedule"'
+      when: never
+    - when: on_success
+```
+
+- If the pipeline is for a merge request, the job is **not** added to the pipeline.
+- If the pipeline is a scheduled pipeline, the job is **not** added to the pipeline.
+- In **all other cases**, the job is added to the pipeline, with `when: on_success`.
+
+WARNING:
+If you use a `when:` clause as the final rule (not including `when: never`), two
+simultaneous pipelines may start. Both push pipelines and merge request pipelines can
+be triggered by the same event (a push to the source branch for an open merge request).
+See how to [prevent duplicate pipelines](#avoid-duplicate-pipelines)
+for more details.
+
+### Complex rules
+
+You can use all `rules` keywords, like `if`, `changes`, and `exists`, in the same
+rule. The rule evaluates to true only when all included keywords evaluate to true.
+
+For example:
+
+```yaml
+docker build:
+  script: docker build -t my-image:$CI_COMMIT_REF_SLUG .
+  rules:
+    - if: '$VAR == "string value"'
+      changes:  # Include the job and set to when:manual if any of the follow paths match a modified file.
+        - Dockerfile
+        - docker/scripts/*
+      when: manual
+      allow_failure: true
+```
+
+If the `Dockerfile` file or any file in `/docker/scripts` has changed **and** `$VAR` == "string value",
+then the job runs manually and is allowed to fail.
+
+You can use [parentheses](#group-variable-expressions-together-with-parentheses) with `&&` and `||` to build more complicated variable expressions.
+[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230938) in GitLab 13.3:
+
+```yaml
+job1:
+  script:
+    - echo This rule uses parentheses.
+  rules:
+    if: ($CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE
+```
+
+WARNING:
+[Before GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/230938),
+rules that use both `||` and `&&` may evaluate with an unexpected order of operations.
+
+### Avoid duplicate pipelines
+
+If a job uses `rules`, a single action, like pushing a commit to a branch, can trigger
+multiple pipelines. You don't have to explicitly configure rules for multiple types
+of pipeline to trigger them accidentally.
+
+Some configurations that have the potential to cause duplicate pipelines cause a
+[pipeline warning](../troubleshooting.md#pipeline-warnings) to be displayed.
+[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/219431) in GitLab 13.3.
+
+For example:
+
+```yaml
+job:
+  script: echo "This job creates double pipelines!"
+  rules:
+    - if: '$CUSTOM_VARIABLE == "false"'
+      when: never
+    - when: always
+```
+
+This job does not run when `$CUSTOM_VARIABLE` is false, but it *does* run in **all**
+other pipelines, including **both** push (branch) and merge request pipelines. With
+this configuration, every push to an open merge request's source branch
+causes duplicated pipelines.
+
+To avoid duplicate pipelines, you can:
+
+- Use [`workflow`](../yaml/README.md#workflow) to specify which types of pipelines
+  can run.
+- Rewrite the rules to run the job only in very specific cases,
+  and avoid a final `when:` rule:
+
+  ```yaml
+  job:
+    script: echo "This job does NOT create double pipelines!"
+    rules:
+      - if: '$CUSTOM_VARIABLE == "true" && $CI_PIPELINE_SOURCE == "merge_request_event"'
+  ```
+
+You can also avoid duplicate pipelines by changing the job rules to avoid either push (branch)
+pipelines or merge request pipelines. However, if you use a `- when: always` rule without
+`workflow: rules`, GitLab still displays a [pipeline warning](../troubleshooting.md#pipeline-warnings).
+
+For example, the following does not trigger double pipelines, but is not recommended
+without `workflow: rules`:
+
+```yaml
+job:
+  script: echo "This job does NOT create double pipelines!"
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "push"'
+      when: never
+    - when: always
+```
+
+You should not include both push and merge request pipelines in the same job without
+[`workflow:rules` that prevent duplicate pipelines](../yaml/README.md#switch-between-branch-pipelines-and-merge-request-pipelines):
+
+```yaml
+job:
+  script: echo "This job creates double pipelines!"
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "push"'
+    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+```
+
+Also, do not mix `only/except` jobs with `rules` jobs in the same pipeline.
+It may not cause YAML errors, but the different default behaviors of `only/except`
+and `rules` can cause issues that are difficult to troubleshoot:
+
+```yaml
+job-with-no-rules:
+  script: echo "This job runs in branch pipelines."
+
+job-with-rules:
+  script: echo "This job runs in merge request pipelines."
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+```
+
+For every change pushed to the branch, duplicate pipelines run. One
+branch pipeline runs a single job (`job-with-no-rules`), and one merge request pipeline
+runs the other job (`job-with-rules`). Jobs with no rules default
+to [`except: merge_requests`](../yaml/README.md#only--except), so `job-with-no-rules`
+runs in all cases except merge requests.
+
+### Common `if` clauses for `rules`
+
+For behavior similar to the [`only`/`except` keywords](../yaml/README.md#only--except), you can
+check the value of the `$CI_PIPELINE_SOURCE` variable:
+
+| Value                         | Description                                                                                                                                                                                                                      |
+|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `api`                         | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline).                                                                                                                                    |
+| `chat`                        | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command.                                                                                                                                                 |
+| `external`                    | When you use CI services other than GitLab.                                                                                                                                                                                        |
+| `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests).                                            |
+| `merge_request_event`         | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). |
+| `parent_pipeline`             | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline.                |
+| `pipeline`                    | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#triggering-multi-project-pipelines-through-api), or the [`trigger`](../yaml/README.md#trigger) keyword. |
+| `push`                        | For pipelines triggered by a `git push` event, including for branches and tags.                                                                                                                                                  |
+| `schedule`                    | For [scheduled pipelines](../pipelines/schedules.md).                                                                                                                                                                            |
+| `trigger`                     | For pipelines created by using a [trigger token](../triggers/README.md#trigger-token).                                                                                                                                           |
+| `web`                         | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section.                                                                                                       |
+| `webide`                      | For pipelines created by using the [WebIDE](../../user/project/web_ide/index.md).                                                                                                                                                |
+
+The following example runs the job as a manual job in scheduled pipelines or in push
+pipelines (to branches or tags), with `when: on_success` (default). It does not
+add the job to any other pipeline type.
+
+```yaml
+job:
+  script: echo "Hello, Rules!"
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "schedule"'
+      when: manual
+      allow_failure: true
+    - if: '$CI_PIPELINE_SOURCE == "push"'
+```
+
+The following example runs the job as a `when: on_success` job in [merge request pipelines](../merge_request_pipelines/index.md)
+and scheduled pipelines. It does not run in any other pipeline type.
+
+```yaml
+job:
+  script: echo "Hello, Rules!"
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+    - if: '$CI_PIPELINE_SOURCE == "schedule"'
+```
+
+Other commonly used variables for `if` clauses:
+
+- `if: $CI_COMMIT_TAG`: If changes are pushed for a tag.
+- `if: $CI_COMMIT_BRANCH`: If changes are pushed to any branch.
+- `if: '$CI_COMMIT_BRANCH == "main"'`: If changes are pushed to `main`.
+- `if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'`: If changes are pushed to the default
+  branch. Use when you want to have the same configuration in multiple
+  projects with different default branches.
+- `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression.
+- `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/README.md#custom-cicd-variables)
+  `CUSTOM_VARIABLE` does **not** match a regular expression.
+- `if: '$CUSTOM_VARIABLE == "value1"'`: If the custom variable `CUSTOM_VARIABLE` is
+  exactly `value1`.
+
+### Variables in `rules:changes`
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34272) in GitLab 13.6.
+> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.7.
+
+You can use CI/CD variables in `rules:changes` expressions to determine when
+to add jobs to a pipeline:
+
+```yaml
+docker build:
+  variables:
+    DOCKERFILES_DIR: 'path/to/files/'
+  script: docker build -t my-image:$CI_COMMIT_REF_SLUG .
+  rules:
+    - changes:
+        - $DOCKERFILES_DIR/*
+```
+
+You can use the `$` character for both variables and paths. For example, if the
+`$DOCKERFILES_DIR` variable exists, its value is used. If it does not exist, the
+`$` is interpreted as being part of a path.
+
 ## Specify when jobs run with `only` and `except`
 
 You can use [`only`](../yaml/README.md#only--except) and [`except`](../yaml/README.md#only--except)
@@ -73,7 +344,7 @@ end-to-end:
       - $CI_COMMIT_MESSAGE =~ /skip-end-to-end-tests/
 ```
 
-You can use [parentheses](../variables/README.md#parentheses) with `&&` and `||`
+You can use [parentheses](#group-variable-expressions-together-with-parentheses) with `&&` and `||`
 to build more complicated variable expressions:
 
 ```yaml
@@ -82,9 +353,12 @@ job1:
     - echo This rule uses parentheses.
   only:
     variables:
-      - ($CI_COMMIT_BRANCH == "master" || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE
+      - ($CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE
 ```
 
+When multiple entries are specified in `only:variables`, the job runs when at least one of them evaluates to `true`.
+You can use `&&` in a single entry when multiple conditions must be satisfied at the same time.
+
 ### `only:changes` / `except:changes` examples
 
 You can skip a job if a change is detected in any file with a
@@ -315,3 +589,114 @@ this feature flag again:
 ```ruby
 Feature.enable(:allow_unsafe_ruby_regexp)
 ```
+
+## CI/CD variable expressions
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37397) in GitLab 10.7 for [the `only` and `except` CI keywords](../yaml/README.md#onlyvariables--exceptvariables)
+> - [Expanded](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3 with [the `rules` keyword](../yaml/README.md#rules)
+
+Use variable expressions to control which jobs are created in a pipeline after changes
+are pushed to GitLab. You can use variable expressions with:
+
+- [`rules:if`](../yaml/README.md#rules).
+- [`only:variables` and `except:variables`](../yaml/README.md#onlyvariables--exceptvariables).
+
+For example, with `rules:if`:
+
+```yaml
+job1:
+  variables:
+    VAR1: "variable1"
+  script:
+    - echo "Test variable comparison
+  rules:
+    - if: $VAR1 == "variable1"
+```
+
+### Compare a variable to a string
+
+You can use the equality operators `==` and `!=` to compare a variable with a
+string. Both single quotes and double quotes are valid. The order doesn't matter,
+so the variable can be first, or the string can be first. For example:
+
+- `if: $VARIABLE == "some value"`
+- `if: $VARIABLE != "some value"`
+- `if: "some value" == $VARIABLE`
+
+### Compare two variables
+
+You can compare the values of two variables. For example:
+
+- `if: $VARIABLE_1 == $VARIABLE_2`
+- `if: $VARIABLE_1 != $VARIABLE_2`
+
+### Check if a variable is undefined
+
+You can compare a variable to the `null` keyword to see if it is defined. For example:
+
+- `if: $VARIABLE == null`
+- `if: $VARIABLE != null`
+
+### Check if a variable is empty
+
+You can check if a variable is defined but empty. For example:
+
+- `if: $VARIABLE == ""`
+- `if: $VARIABLE != ""`
+
+### Check if a variable exists
+
+You can check for the existence of a variable by using just the variable name in
+the expression. The variable must not be empty. For example:
+
+- `if: $VARIABLE`
+
+### Compare a variable to a regex pattern
+
+You can do regex pattern matching on variable values with the `=~` and `!~` operators.
+Variable pattern matching with regular expressions uses the
+[RE2 regular expression syntax](https://github.com/google/re2/wiki/Syntax).
+
+Expressions evaluate as `true` if:
+
+- Matches are found when using `=~`.
+- Matches are *not* found when using `!~`.
+
+For example:
+
+- `$VARIABLE =~ /^content.*/`
+- `$VARIABLE_1 !~ /^content.*/`
+
+Pattern matching is case-sensitive by default. Use the `i` flag modifier to make a
+pattern case-insensitive. For example: `/pattern/i`.
+
+### Join variable expressions together with `&&` or `||`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/62867) in GitLab 12.0
+
+You can join multiple expressions using `&&` (and) or `||` (or), for example:
+
+- `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"`
+- `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 =~ /thing$/ && $VARIABLE3`
+- `$VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/ && $VARIABLE3`
+
+The precedence of operators follows the [Ruby 2.5 standard](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html),
+so `&&` is evaluated before `||`.
+
+#### Group variable expressions together with parentheses
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230938) in GitLab 13.3.
+> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/238174) in GitLab 13.5.
+
+You can use parentheses to group expressions together. Parentheses take precedence over
+`&&` and `||`, so expressions enclosed in parentheses are evaluated first, and the
+result is used for the rest of the expression.
+
+You can nest parentheses to create complex conditions, and the inner-most expressions
+in parentheses are evaluated first.
+
+For example:
+
+- `($VARIABLE1 =~ /^content.*/ || $VARIABLE2) && ($VARIABLE3 =~ /thing$/ || $VARIABLE4)`
+- `($VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/) && $VARIABLE3`
+- `$CI_COMMIT_BRANCH == "my-branch" || (($VARIABLE1 == "thing" || $VARIABLE2 == "thing") && $VARIABLE3)`