diff options
Diffstat (limited to 'doc/ci/yaml/README.md')
| -rw-r--r-- | doc/ci/yaml/README.md | 1055 |
1 files changed, 333 insertions, 722 deletions
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 2e4ab68a0e8..1057a1389de 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -7,105 +7,29 @@ type: reference # GitLab CI/CD pipeline configuration reference -GitLab CI/CD [pipelines](../pipelines/index.md) are configured using a YAML file called `.gitlab-ci.yml` within each project. +This document lists the configuration options for your GitLab `.gitlab-ci.yml` file. -The `.gitlab-ci.yml` file defines the structure and order of the pipelines and determines: - -- What to execute using [GitLab Runner](https://docs.gitlab.com/runner/). -- What decisions to make when specific conditions are encountered. For example, when a process succeeds or fails. - -This topic covers CI/CD pipeline configuration. For other CI/CD configuration information, see: - -- [GitLab CI/CD Variables](../variables/README.md), for configuring the environment the pipelines run in. -- [GitLab Runner advanced configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html), for configuring GitLab Runner. - -We have complete examples of configuring pipelines: - -- For a quick introduction to GitLab CI/CD, follow our [quick start guide](../quick_start/README.md). +- For a quick introduction to GitLab CI/CD, follow the [quick start guide](../quick_start/README.md). - For a collection of examples, see [GitLab CI/CD Examples](../examples/README.md). -- To see a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml). - -> For some additional information about GitLab CI/CD: -> -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch the [CI/CD Ease of configuration](https://www.youtube.com/embed/opdLqwz6tcE) video. -> - Watch the [Making the case for CI/CD in your organization](https://about.gitlab.com/compare/github-actions-alternative/) -> webcast to learn the benefits of CI/CD and how to measure the results of CI/CD automation. -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Learn how [Verizon reduced rebuilds](https://about.gitlab.com/blog/2019/02/14/verizon-customer-story/) -> from 30 days to under 8 hours with GitLab. - -If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository), -you may need to enable pipeline triggering. Go to your project's **Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. +- To view a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml). -## Introduction - -Pipeline configuration begins with jobs. Jobs are the most fundamental element of a `.gitlab-ci.yml` file. - -Jobs are: - -- Defined with constraints stating under what conditions they should be executed. -- Top-level elements with an arbitrary name and must contain at least the [`script`](#script) clause. -- Not limited in how many can be defined. - -For example: - -```yaml -job1: - script: "execute-script-for-job1" - -job2: - script: "execute-script-for-job2" -``` - -The above example is the simplest possible CI/CD configuration with two separate -jobs, where each of the jobs executes a different command. -Of course a command can execute code directly (`./configure;make;make install`) -or run a script (`test.sh`) in the repository. - -Jobs are picked up by [runners](../runners/README.md) and executed within the -environment of the runner. What is important is that each job is run -independently from each other. - -### Validate the `.gitlab-ci.yml` - -Each instance of GitLab CI/CD has an embedded debug tool called Lint, which validates the -content of your `.gitlab-ci.yml` files. You can find the Lint under the page `ci/lint` of your +While you are authoring your `.gitlab-ci.yml` file, you can validate it +by using the [CI Lint](../lint.md) tool. project namespace. For example, `https://gitlab.example.com/gitlab-org/project-123/-/ci/lint`. -### Unavailable names for jobs - -Each job must have a unique name, but there are a few **reserved `keywords` that -can't be used as job names**: - -- `image` -- `services` -- `stages` -- `types` -- `before_script` -- `after_script` -- `variables` -- `cache` -- `include` +## Job keywords -### Using reserved keywords +A job is defined as a list of keywords that define the job's behavior. -If you get validation error when using specific values (for example, `true` or `false`), try to: - -- Quote them. -- Change them to a different form. For example, `/bin/true`. - -## Configuration parameters - -A job is defined as a list of parameters that define the job's behavior. - -The following table lists available parameters for jobs: +The following table lists available keywords for jobs: | Keyword | Description | |:---------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [`script`](#script) | Shell script that is executed by a runner. | -| [`after_script`](#before_script-and-after_script) | Override a set of commands that are executed after job. | +| [`after_script`](#after_script) | Override a set of commands that are executed after job. | | [`allow_failure`](#allow_failure) | Allow job to fail. Failed job does not contribute to commit status. | | [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, and `artifacts:reports`. | -| [`before_script`](#before_script-and-after_script) | Override a set of commands that are executed before job. | +| [`before_script`](#before_script) | Override a set of commands that are executed before job. | | [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, `cache:when`, and `cache:policy`. | | [`coverage`](#coverage) | Code coverage settings for a given job. | | [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. | @@ -130,22 +54,44 @@ The following table lists available parameters for jobs: | [`variables`](#variables) | Define job variables on a job level. | | [`when`](#when) | When to run job. Also available: `when:manual` and `when:delayed`. | -## Global parameters +### Unavailable names for jobs -Some parameters must be defined at a global level, affecting all jobs in the pipeline. +Each job must have a unique name, but there are a few **reserved `keywords` that +can't be used as job names**: + +- `image` +- `services` +- `stages` +- `types` +- `before_script` +- `after_script` +- `variables` +- `cache` +- `include` + +## Global keywords + +Some keywords must be defined at a global level, affecting all jobs in the pipeline. + +### Using reserved keywords + +If you get validation error when using specific values (for example, `true` or `false`), try to: + +- Quote them. +- Change them to a different form. For example, `/bin/true`. ### Global defaults -Some parameters can be set globally as the default for all jobs using the -`default:` keyword. Default parameters can then be overridden by job-specific +Some keywords can be set globally as the default for all jobs using the +`default:` keyword. Default keywords can then be overridden by job-specific configuration. -The following job parameters can be defined inside a `default:` block: +The following job keywords can be defined inside a `default:` block: - [`image`](#image) - [`services`](#services) -- [`before_script`](#before_script-and-after_script) -- [`after_script`](#before_script-and-after_script) +- [`before_script`](#before_script) +- [`after_script`](#after_script) - [`tags`](#tags) - [`cache`](#cache) - [`artifacts`](#artifacts) @@ -173,20 +119,20 @@ rspec 2.6: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207484) in GitLab 12.9. You can disable inheritance of globally defined defaults -and variables with the `inherit:` parameter. +and variables with the `inherit:` keyword. -To enable or disable the inheritance of all `variables:` or `default:` parameters, use the following format: +To enable or disable the inheritance of all `variables:` or `default:` keywords, use the following format: - `default: true` or `default: false` - `variables: true` or `variables: false` -To inherit only a subset of `default:` parameters or `variables:`, specify what +To inherit only a subset of `default:` keywords or `variables:`, specify what you wish to inherit. Anything not listed is **not** inherited. Use one of the following formats: ```yaml inherit: - default: [parameter1, parameter2] + default: [keyword1, keyword2] variables: [VARIABLE1, VARIABLE2] ``` @@ -195,8 +141,8 @@ Or: ```yaml inherit: default: - - parameter1 - - parameter2 + - keyword1 + - keyword2 variables: - VARIABLE1 - VARIABLE2 @@ -288,23 +234,23 @@ There are also two edge cases worth mentioning: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29654) in GitLab 12.5 -The top-level `workflow:` key applies to the entirety of a pipeline, and -determines whether or not a pipeline is created. It accepts a single -`rules:` key that operates similarly to [`rules:` defined within jobs](#rules), -enabling dynamic configuration of the pipeline. +The top-level `workflow:` keyword determines whether or not a pipeline is created. +It accepts a single `rules:` keyword that is similar to [`rules:` defined within jobs](#rules). +Use it to define what can trigger a new pipeline. -If you are new to GitLab CI/CD and `workflow: rules`, you may find the [`workflow:rules` templates](#workflowrules-templates) useful. +You can use the [`workflow:rules` templates](#workflowrules-templates) to import +a preconfigured `workflow: rules` entry. -To define your own `workflow: rules`, the available configuration options are: +`workflow: rules` accepts these keywords: -- [`if`](#rulesif): Define a rule. -- [`when`](#when): May be set to `always` or `never` only. If not provided, the default value is `always`​. +- [`if`](#rulesif): Check this rule to determine when to run a pipeline. +- [`when`](#when): Specify what to do when the `if` rule evaluates to true. + - To run a pipeline, set to `always`. + - To prevent pipelines from running, set to `never`. -If a pipeline attempts to run but matches no rule, it's dropped and doesn't run. +When no rules evaluate to true, the pipeline does not run. -Use the example rules below exactly as written to allow pipelines that match the rule -to run. Add `when: never` to prevent pipelines that match the rule from running. See -the [common `if` clauses for `rules`](#common-if-clauses-for-rules) for more examples. +Some example `if` clauses for `workflow: rules`: | Example rules | Details | |------------------------------------------------------|-----------------------------------------------------------| @@ -313,9 +259,12 @@ the [common `if` clauses for `rules`](#common-if-clauses-for-rules) for more exa | `if: $CI_COMMIT_TAG` | Control when tag pipelines run. | | `if: $CI_COMMIT_BRANCH` | Control when branch pipelines run. | +See the [common `if` clauses for `rules`](#common-if-clauses-for-rules) for more examples. + For example, in the following configuration, pipelines run for all `push` events (changes to -branches and new tags). Only push events with `-wip` in the commit message are excluded. Scheduled -pipelines and merge request pipelines don't run, as there's no rule allowing them. +branches and new tags). Pipelines for push events with `-wip` in the commit message +don't run, because they are set to `when: never`. Pipelines for schedules or merge requests +don't run either, because no rules evaluate to true for them: ```yaml workflow: @@ -325,11 +274,11 @@ workflow: - if: '$CI_PIPELINE_SOURCE == "push"' ``` -This example has strict rules, and no other pipelines can run. +This example has strict rules, and pipelines do **not** run in any other case. -Alternatively, you can have loose rules by using only `when: never` rules, followed -by a final `when: always` rule. This allows all types of pipelines, except for any -that match the `when: never` rules: +Alternatively, all of the rules can be `when: never`, with a final +`when: always` rule. Pipelines that match the `when: never` rules do not run. +All other pipeline types run: ```yaml workflow: @@ -341,12 +290,13 @@ workflow: - when: always ``` -This example never allows pipelines for schedules or `push` (branches and tags) pipelines, -but does allow pipelines in **all** other cases, *including* merge request pipelines. +This example prevents pipelines for schedules or `push` (branches and tags) pipelines. +The final `when: always` rule lets all other pipeline types run, **including** merge +request pipelines. -Be careful not to use a configuration that might run -merge request pipelines and branch pipelines at the same time. As with `rules` defined in jobs, -it can cause [duplicate pipelines](#prevent-duplicate-pipelines). +Be careful not to have rules that match both branch pipelines +and merge request pipelines. Similar to `rules` defined in jobs, this can cause +[duplicate pipelines](#prevent-duplicate-pipelines). #### `workflow:rules` templates @@ -405,7 +355,7 @@ of using YAML anchors, you can use the [`extends` keyword](#extends). `include` supports the following inclusion methods: -| Method | Description | +| Keyword | Method | |:--------------------------------|:------------------------------------------------------------------| | [`local`](#includelocal) | Include a file from the local project repository. | | [`file`](#includefile) | Include a file from a different project repository. | @@ -471,7 +421,7 @@ include: file: '/templates/.gitlab-ci-template.yml' ``` -You can also specify `ref`, with the default being the `HEAD` of the project: +You can also specify a `ref`. If not specified, it defaults to the `HEAD` of the project: ```yaml include: @@ -492,11 +442,47 @@ All [nested includes](#nested-includes) are executed in the scope of the target This means you can use local (relative to target project), project, remote, or template includes. +##### Multiple files from a project + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26793) in GitLab 13.6. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to disable it. **(CORE ONLY)** + +You can include multiple files from the same project: + +```yaml +include: + - project: 'my-group/my-project' + ref: master + file: + - '/templates/.builds.yml' + - '/templates/.tests.yml' +``` + +Including multiple files from the same project is under development but ready for production use. It is +deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:ci_include_multiple_files_from_project) +``` + +To disable it: + +```ruby +Feature.disable(:ci_include_multiple_files_from_project) +``` + #### `include:remote` `include:remote` can be used to include a file from a different location, -using HTTP/HTTPS, referenced by using the full URL. The remote file must be -publicly accessible through a simple GET request as authentication schemas +using HTTP/HTTPS, referenced by the full URL. The remote file must be +publicly accessible by a GET request, because authentication schemas in the remote URL are not supported. For example: ```yaml @@ -542,15 +528,15 @@ Nested includes allow you to compose a set of includes. A total of 100 includes is allowed, but duplicate includes are considered a configuration error. In [GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/28212) and later, the time limit -for resolving all files is 30 seconds. +to resolve all files is 30 seconds. #### Additional `includes` examples There is a list of [additional `includes` examples](includes.md) available. -## Parameter details +## Keyword details -The following are detailed explanations for parameters used to configure CI/CD pipelines. +The following are detailed explanations for keywords used to configure CI/CD pipelines. ### `image` @@ -558,7 +544,7 @@ Used to specify [a Docker image](../docker/using_docker_images.md#what-is-an-ima For: -- Simple definition examples, see [Define `image` and `services` from `.gitlab-ci.yml`](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml). +- Usage examples, see [Define `image` and `services` from `.gitlab-ci.yml`](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml). - Detailed usage information, refer to [Docker integration](../docker/README.md) documentation. #### `image:name` @@ -579,7 +565,7 @@ Used to specify a [service Docker image](../docker/using_docker_images.md#what-i For: -- Simple definition examples, see [Define `image` and `services` from `.gitlab-ci.yml`](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml). +- Usage examples, see [Define `image` and `services` from `.gitlab-ci.yml`](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml). - Detailed usage information, refer to [Docker integration](../docker/README.md) documentation. - For example services, see [GitLab CI/CD Services](../services/README.md). @@ -617,9 +603,9 @@ job: script: "bundle exec rspec" ``` -[YAML anchors for scripts](#yaml-anchors-for-script) are available. +You can use [YAML anchors with `script`](#yaml-anchors-for-scripts). -This parameter can also contain several commands using an array: +This keyword can also contain several commands in an array: ```yaml job: @@ -635,8 +621,8 @@ a "key: value" pair. Be careful when using special characters: `:`, `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. If any of the script commands return an exit code other than zero, the job -fails and further commands are not executed. You can avoid this behavior by -storing the exit code in a variable: +fails and further commands are not executed. Store the exit code in a variable to +avoid this behavior: ```yaml job: @@ -645,191 +631,86 @@ job: - if [ $exit_code -ne 0 ]; then echo "Previous command failed"; fi; ``` -#### `before_script` and `after_script` +#### `before_script` > Introduced in GitLab 8.7 and requires GitLab Runner v1.2. -`before_script` is used to define commands that should be run before each -job, including deploy jobs, but after the restoration of any [artifacts](#artifacts). -This must be an array. +`before_script` is used to define commands that should run before each job, including +deploy jobs, but after the restoration of any [artifacts](#artifacts). This must be an array. Scripts specified in `before_script` are concatenated with any scripts specified in the main [`script`](#script), and executed together in a single shell. -`after_script` is used to define commands that run after each -job, including failed jobs. This must be an array. If a job times out or is cancelled, -the `after_script` commands are not executed. Support for executing `after_script` -commands for timed-out or cancelled jobs -[is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/15603). - -Scripts specified in `after_script` are executed in a new shell, separate from any -`before_script` or `script` scripts. As a result, they: - -- Have a current working directory set back to the default. -- Have no access to changes done by scripts defined in `before_script` or `script`, including: - - Command aliases and variables exported in `script` scripts. - - Changes outside of the working tree (depending on the runner executor), like - software installed by a `before_script` or `script` script. -- Have a separate timeout, which is hard coded to 5 minutes. See - [related issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2716) for details. -- Don't affect the job's exit code. If the `script` section succeeds and the - `after_script` times out or fails, the job exits with code `0` (`Job Succeeded`). - -It's possible to overwrite a globally defined `before_script` or `after_script` -if you set it per-job: +It's possible to overwrite a globally defined `before_script` if you define it in a job: ```yaml default: before_script: - - global before script - -job: - before_script: - - execute this instead of global before script - script: - - my command - after_script: - - execute this after my script -``` - -[YAML anchors for `before_script` and `after_script`](#yaml-anchors-for-before_script-and-after_script) are available. + - echo "Execute this in all jobs that don't already have a before_script section." -#### Coloring script output - -Script output can be colored using [ANSI escape codes](https://en.wikipedia.org/wiki/ANSI_escape_code#Colors), -or by running commands or programs that output ANSI escape codes. - -For example, using [Bash with color codes](https://misc.flogisoft.com/bash/tip_colors_and_formatting): - -```yaml -job: +job1: script: - - echo -e "\e[31mThis text is red,\e[0m but this text isn't\e[31m however this text is red again." -``` + - echo "This executes after the global before_script." -You can define the color codes in Shell variables, or even [custom environment variables](../variables/README.md#custom-environment-variables), -which makes the commands easier to read and reusable. - -For example, using the same example as above and variables defined in a `before_script`: - -```yaml job: before_script: - - TXT_RED="\e[31m" && TXT_CLEAR="\e[0m" + - echo "Execute this instead of the global before_script." script: - - echo -e "${TXT_RED}This text is red,${TXT_CLEAR} but this part isn't${TXT_RED} however this part is again." - - echo "This text is not colored" + - echo "This executes after the job's `before_script`" ``` -Or with [PowerShell color codes](https://superuser.com/a/1259916): +You can use [YAML anchors with `before_script`](#yaml-anchors-for-scripts). -```yaml -job: - before_script: - - $esc="$([char]27)"; $TXT_RED="$esc[31m"; $TXT_CLEAR="$esc[0m" - script: - - Write-Host $TXT_RED"This text is red,"$TXT_CLEAR" but this text isn't"$TXT_RED" however this text is red again." - - Write-Host "This text is not colored" -``` - -#### Multi-line commands +#### `after_script` -You can split long commands into multi-line commands to improve readability -using [`|` (literal) and `>` (folded) YAML multi-line block scalar indicators](https://yaml-multiline.info/). - -CAUTION: **Warning:** -If multiple commands are combined into one command string, only the last command's -failure or success is reported. -[Failures from earlier commands are ignored due to a bug](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25394). -To work around this, -run each command as a separate `script:` item, or add an `exit 1` command -to each command string. +Introduced in GitLab 8.7 and requires GitLab Runner v1.2. -You can use the `|` (literal) YAML multiline block scalar indicator to write -commands over multiple lines in the `script` section of a job description. -Each line is treated as a separate command. -Only the first command is repeated in the job log, but additional -commands are still executed: +`after_script` is used to define commands that run after each job, including failed +jobs. This must be an array. -```yaml -job: - script: - - | - echo "First command line." - echo "Second command line." - echo "Third command line." -``` - -The example above renders in the job log as: +If a job times out or is cancelled, the `after_script` commands are not executed. +Support for executing `after_script` commands for timed-out or cancelled jobs +[is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/15603). -```shell -$ echo First command line # collapsed multi-line command -First command line -Second command line. -Third command line. -``` +Scripts specified in `after_script` are executed in a new shell, separate from any +`before_script` or `script` scripts. As a result, they: -The `>` (folded) YAML multiline block scalar indicator treats empty lines between -sections as the start of a new command: +- Have a current working directory set back to the default. +- Have no access to changes done by scripts defined in `before_script` or `script`, including: + - Command aliases and variables exported in `script` scripts. + - Changes outside of the working tree (depending on the runner executor), like + software installed by a `before_script` or `script` script. +- Have a separate timeout, which is hard coded to 5 minutes. See the + [related issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2716) for details. +- Don't affect the job's exit code. If the `script` section succeeds and the + `after_script` times out or fails, the job exits with code `0` (`Job Succeeded`). ```yaml -job: - script: - - > - echo "First command line - is split over two lines." - - echo "Second command line." -``` - -This behaves similarly to writing multiline commands without the `>` or `|` block -scalar indicators: +default: + after_script: + - echo "Execute this in all jobs that don't already have an after_script section." -```yaml -job: +job1: script: - - echo "First command line - is split over two lines." - - echo "Second command line." -``` - -Both examples above render in the job log as: - -```shell -$ echo First command line is split over two lines. # collapsed multi-line command -First command line is split over two lines. -Second command line. -``` - -When you omit the `>` or `|` block scalar indicators, GitLab forms the command -by concatenating non-empty lines. Make sure the lines can run when concatenated. + - echo "This executes first. When it completes, the global after_script executes." -Shell [here documents](https://en.wikipedia.org/wiki/Here_document) work with the -`|` and `>` operators as well. The example below transliterates the lower case letters -to upper case: - -```yaml job: script: - - | - tr a-z A-Z << END_TEXT - one two three - four five six - END_TEXT + - echo "This executes first. When it completes, the job's `after_script` executes." + after_script: + - echo "Execute this instead of the global after_script." ``` -Results in: +You can use [YAML anchors with `after_script`](#yaml-anchors-for-scripts). -```shell -$ tr a-z A-Z << END_TEXT # collapsed multi-line command - ONE TWO THREE - FOUR FIVE SIX -``` +#### Script syntax -#### Custom collapsible sections +You can use special syntax in [`script`](README.md#script) sections to: -See [custom collapsible sections](../pipelines/index.md#custom-collapsible-sections). +- [Split long commands](script.md#split-long-commands) into multiline commands. +- [Use color codes](script.md#add-color-codes-to-script-output) to make job logs easier to review. +- [Create custom collapsible sections](../jobs/index.md#custom-collapsible-sections) + to simplify job log output. ### `stage` @@ -968,7 +849,7 @@ rspec: - $RSPEC ``` -If you do want to include the `rake test`, see [`before_script` and `after_script`](#before_script-and-after_script). +If you do want to include the `rake test`, see [`before_script`](#before_script) or [`after_script`](#after_script). `.tests` in this example is a [hidden job](#hide-jobs), but it's possible to inherit from regular jobs as well. @@ -1422,29 +1303,20 @@ Other commonly used variables for `if` clauses: #### `rules:changes` -To determine if jobs should be added to a pipeline, `rules: changes` clauses check -the files changed by Git push events. +`rules:changes` determines whether or not to add jobs to a pipeline by checking for +changes to specific files. `rules: changes` works exactly the same way as [`only: changes` and `except: changes`](#onlychangesexceptchanges), -accepting an array of paths. Similarly, it always returns true if there is no -Git push event, for example, when a new tag is created. It's recommended to use it -only with branch pipelines or merge request pipelines. For example, it's common to -use `rules: changes` with one of the following `if` clauses: - -- `if: $CI_COMMIT_BRANCH` -- `if: '$CI_PIPELINE_SOURCE == "merge_request_event"'` - -For example: +accepting an array of paths. It's recommended to only use `rules: changes` with branch +pipelines or merge request pipelines. For example, it's common to use `rules: changes` +with merge request pipelines: ```yaml -workflow: - rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' - docker build: script: docker build -t my-image:$CI_COMMIT_REF_SLUG . rules: - - changes: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + changes: - Dockerfile when: manual allow_failure: true @@ -1452,14 +1324,77 @@ docker build: In this example: -- [`workflow: rules`](#workflowrules) allows only pipelines for merge requests for all jobs. +- If the pipeline is a merge request pipeline, check `Dockerfile` for changes. - If `Dockerfile` has changed, add the job to the pipeline as a manual job, and allow the pipeline to continue running even if the job is not triggered (`allow_failure: true`). - If `Dockerfile` has not changed, do not add job to any pipeline (same as `when: never`). -To implement a rule similar to [`except: changes`](#onlychangesexceptchanges), +To use `rules: changes` with branch pipelines instead of merge request pipelines, +change the `if:` clause in the example above to: + +```yaml +rules: + - if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH +``` + +To implement a rule similar to [`except:changes`](#onlychangesexceptchanges), use `when: never`. +CAUTION: **Caution:** +You can use `rules: changes` with other pipeline types, but it is not recommended +because `rules: changes` always evaluates to true when there is no Git `push` event. +Tag pipelines, scheduled pipelines, and so on do **not** have a Git `push` event +associated with them. A `rules: changes` job is **always** added to those pipeline +if there is no `if:` statement that limits the job to branch or merge request pipelines. + +##### Variables in `rules:changes` + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34272) in GitLab 13.6. +> - It was [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.6. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-variables-support-in-ruleschanges). **(CORE ONLY)** + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +Environment variables can be used in `rules:changes` expressions to determine when +to add jobs to a pipeline: + +```yaml +docker build: + variables: + DOCKERFILES_DIR: 'path/to/files/' + script: docker build -t my-image:$CI_COMMIT_REF_SLUG . + rules: + - changes: + - $DOCKERFILES_DIR/* +``` + +The `$` character can be used for both variables and paths. For example, if the +`$DOCKERFILES_DIR` variable exists, its value is used. If it does not exist, the +`$` is interpreted as being part of a path. + +###### Enable or disable variables support in `rules:changes` **(CORE ONLY)** + +Variables support in `rules:changes` is under development, but is ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:ci_variable_expansion_in_rules_changes) +``` + +To disable it: + +```ruby +Feature.disable(:ci_variable_expansion_in_rules_changes) +``` + #### `rules:exists` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4. @@ -1567,7 +1502,7 @@ The [`rules`](#rules) syntax is an improved, more powerful solution for defining when jobs should run or not. Consider using `rules` instead of `only/except` to get the most out of your pipelines. -`only` and `except` are two parameters that set a job policy to limit when +`only` and `except` are two keywords that set a job policy to limit when jobs are created: 1. `only` defines the names of branches and tags the job runs for. @@ -1695,7 +1630,7 @@ while just `/issue/` would also match a branch called `severe-issues`. #### Supported `only`/`except` regexp syntax In GitLab 11.9.4, GitLab began internally converting the regexp used -in `only` and `except` parameters to [RE2](https://github.com/google/re2/wiki/Syntax). +in `only` and `except` keywords to [RE2](https://github.com/google/re2/wiki/Syntax). [RE2](https://github.com/google/re2/wiki/Syntax) limits the set of available features due to computational complexity, and some features, like negative lookaheads, became unavailable. @@ -2489,7 +2424,7 @@ deployment to the `production` environment. > - Before GitLab 8.11, the name of an environment could be defined as a string like > `environment: production`. The recommended way now is to define it under the > `name` keyword. -> - The `name` parameter can use any of the defined CI variables, +> - The `name` keyword can use any of the defined CI variables, > including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). > You however can't use variables defined under `script`. @@ -2525,7 +2460,7 @@ deploy to production: > - Introduced in GitLab 8.11. > - Before GitLab 8.11, the URL could be added only in GitLab's UI. The > recommended way now is to define it in `.gitlab-ci.yml`. -> - The `url` parameter can use any of the defined CI variables, +> - The `url` keyword can use any of the defined CI variables, > including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). > You however can't use variables defined under `script`. @@ -2624,7 +2559,7 @@ In the example above, if the configuration is not identical: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20956) in GitLab 12.8. -The `auto_stop_in` keyword is for specifying life period of the environment, +The `auto_stop_in` keyword is for specifying the lifetime of the environment, that when expired, GitLab automatically stops them. For example, @@ -2637,8 +2572,8 @@ review_app: auto_stop_in: 1 day ``` -When `review_app` job is executed and a review app is created, a life period of -the environment is set to `1 day`. +When the environment for `review_app` is created, the environment's lifetime is set to `1 day`. +Every time the review app is deployed, that lifetime is also reset to `1 day`. For more information, see [the environments auto-stop documentation](../environments/index.md#environments-auto-stop) @@ -2679,7 +2614,7 @@ To follow progress on support for GitLab-managed clusters, see the > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21971) in GitLab 8.12 and GitLab Runner 1.6. > - The `$CI_ENVIRONMENT_SLUG` was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/22864) in GitLab 8.15. -> - The `name` and `url` parameters can use any of the defined CI variables, +> - The `name` and `url` keywords can use any of the defined CI variables, > including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). > You however can't use variables defined under `script`. @@ -2774,11 +2709,7 @@ Otherwise cache content can be overwritten. > Introduced in GitLab Runner v1.0.0. -The cache is shared between jobs, so if you're using different -paths for different jobs, you should also set a different `cache:key`. -Otherwise cache content can be overwritten. - -The `key` parameter defines the affinity of caching between jobs. +The `key` keyword defines the affinity of caching between jobs. You can have a single cache for all jobs, cache per-job, cache per-branch, or any other way that fits your workflow. This way, you can fine tune caching, including caching data between different jobs or even different branches. @@ -2812,6 +2743,32 @@ URI-encoded `%2F`. A value made only of dots (`.`, `%2E`) is also forbidden. You can specify a [fallback cache key](#fallback-cache-key) to use if the specified `cache:key` is not found. +#### Fallback cache key + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4. + +You can use the `$CI_COMMIT_REF_SLUG` [variable](#variables) to specify your [`cache:key`](#cachekey). +For example, if your `$CI_COMMIT_REF_SLUG` is `test` you can set a job +to download cache that's tagged with `test`. + +If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to +specify a cache to use when none exists. + +For example: + +```yaml +variables: + CACHE_FALLBACK_KEY: fallback-key + +cache: + key: "$CI_COMMIT_REF_SLUG" + paths: + - binaries/ +``` + +In this example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined +by the `CACHE_FALLBACK_KEY` variable. + ##### `cache:key:files` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18986) in GitLab v12.5. @@ -2847,7 +2804,7 @@ use the new cache, instead of rebuilding the dependencies. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18986) in GitLab v12.5. When you want to combine a prefix with the SHA computed for `cache:key:files`, -use the `prefix` parameter with `key:files`. +use the `prefix` keyword with `key:files`. For example, if you add a `prefix` of `test`, the resulting key is: `test-feef9576d21ee9b6a32e30c5c79d0a0ceb68d1e5`. If neither file was changed in any commits, the prefix is added to `default`, so the key in the example would be `test-default`. @@ -3307,7 +3264,7 @@ It also exposes these reports in GitLab's UI (merge requests, pipeline views, an These are the available report types: -| Parameter | Description | +| Keyword | Description | |--------------------------------------------------------------------------------------------------------------------------------------|-------------| | [`artifacts:reports:cobertura`](../pipelines/job_artifacts.md#artifactsreportscobertura) | The `cobertura` report collects Cobertura coverage XML files. | | [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects CodeQuality issues. | @@ -3329,7 +3286,7 @@ These are the available report types: > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. By default, all [`artifacts`](#artifacts) from previous [stages](#stages) -are passed to each job. However, you can use the `dependencies` parameter to +are passed to each job. However, you can use the `dependencies` keyword to define a limited list of jobs to fetch artifacts from. You can also set a job to download no artifacts at all. To use this feature, define `dependencies` in context of the job and pass @@ -3495,7 +3452,7 @@ Possible values for `when` are: - `scheduler_failure`: Retry if the scheduler failed to assign the job to a runner. - `data_integrity_failure`: Retry if there was a structural integrity problem detected. -You can specify the number of [retry attempts for certain stages of job execution](#job-stages-attempts) using variables. +You can specify the number of [retry attempts for certain stages of job execution](../runners/README.md#job-stages-attempts) using variables. ### `timeout` @@ -3522,7 +3479,7 @@ exceed the runner-specific timeout. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21480) in GitLab 11.5. Use `parallel` to configure how many instances of a job to run in -parallel. This value has to be greater than or equal to two (2) and less than or equal to 50. +parallel. This value can be from 2 to 50. This creates N instances of the same job that run in parallel. They are named sequentially from `job_name 1/N` to `job_name N/N`. @@ -3574,9 +3531,6 @@ There can be from 2 to 50 jobs. [In GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/26362) and later, you can have one-dimensional matrices with a single job. -The ability to have one-dimensional matrices is [deployed behind a feature flag](../../user/feature_flags.md), -enabled by default. It's enabled on GitLab.com. For self-managed GitLab instances, -administrators can opt to disable it by [disabling the `one_dimensional_matrix:` feature flag](../../administration/feature_flags.md). **(CORE ONLY)** Every job gets the same `CI_NODE_TOTAL` [environment variable](../variables/README.md#predefined-environment-variables) value, and a unique `CI_NODE_INDEX` value. @@ -3624,8 +3578,8 @@ Use `trigger` to define a downstream pipeline trigger. When GitLab starts a job with a `trigger` definition, a downstream pipeline is created. Jobs with `trigger` can only use a [limited set of keywords](../multi_project_pipelines.md#limitations). -For example, you can't run commands with [`script`](#script), [`before_script`](#before_script-and-after_script), -or [`after_script`](#before_script-and-after_script). +For example, you can't run commands with [`script`](#script), [`before_script`](#before_script), +or [`after_script`](#after_script). You can use this keyword to create two different types of downstream pipelines: @@ -3784,7 +3738,7 @@ starting, which reduces parallelization. To force a rebuild of a specific branch, tag, or commit, you can use an API call with a trigger token. -The trigger token is different than the [`trigger`](#trigger) parameter. +The trigger token is different than the [`trigger`](#trigger) keyword. [Read more in the triggers documentation.](../triggers/README.md) @@ -3884,8 +3838,8 @@ For more information, see [Deployments Safety](../environments/deployment_safety These methods are supported: - [`tag_name`](#releasetag_name) +- [`description`](#releasedescription) - [`name`](#releasename) (optional) -- [`description`](#releasedescription) (optional) - [`ref`](#releaseref) (optional) - [`milestones`](#releasemilestones) (optional) - [`released_at`](#releasereleased_at) (optional) @@ -3943,7 +3897,7 @@ For example, when creating a Release from a Git tag: job: release: tag_name: $CI_COMMIT_TAG - description: changelog.txt + description: 'Release description' ``` It is also possible to create any unique tag, in which case `only: tags` is not mandatory. @@ -3953,7 +3907,7 @@ A semantic versioning example: job: release: tag_name: ${MAJOR}_${MINOR}_${REVISION} - description: changelog.txt + description: 'Release description' ``` - The Release is created only if the job's main script succeeds. @@ -4160,8 +4114,8 @@ Read more on [GitLab Pages user documentation](../../user/project/pages/index.md > Introduced in GitLab Runner v0.5.0. -Variables are configurable values that are passed to jobs. They can be set -globally and per-job. +[CI/CD variables](../variables/README.md) are configurable values that are passed to jobs. +They can be set globally and per-job. There are two types of variables. @@ -4178,372 +4132,49 @@ Variables are meant for non-sensitive project configuration, for example: ```yaml variables: - DATABASE_URL: "postgres://postgres@postgres/my_database" + DEPLOY_SITE: "https://example.com/" + +deploy_job: + stage: deploy + script: + - deploy-script --url $DEPLOY_SITE --path "/" + +deploy_review_job: + stage: deploy + variables: + REVIEW_PATH: "/review" + script: + - deploy-review-script --url $DEPLOY_SITE --path $REVIEW_PATH ``` -You can use integers and strings for the variable's name and value. -You cannot use floats. +You can use only integers and strings for the variable's name and value. If you define a variable at the top level of the `gitlab-ci.yml` file, it is global, -meaning it applies to all jobs. - -If you define a variable within a job, it's available to that job only. +meaning it applies to all jobs. If you define a variable within a job, it's available +to that job only. If a variable of the same name is defined globally and for a specific job, the [job-specific variable is used](../variables/README.md#priority-of-environment-variables). All YAML-defined variables are also set to any linked -[service containers](../docker/using_docker_images.md#what-is-a-service). - -[YAML anchors for variables](#yaml-anchors-for-variables) are available. - -Learn more about [variables and their priority](../variables/README.md). - -### Git strategy - -> - Introduced in GitLab 8.9 as an experimental feature. -> - `GIT_STRATEGY=none` requires GitLab Runner v1.7+. - -You can set the `GIT_STRATEGY` used for getting recent application code, either -globally or per-job in the [`variables`](#variables) section. If left -unspecified, the default from the project settings is used. - -There are three possible values: `clone`, `fetch`, and `none`. - -`clone` is the slowest option. It clones the repository from scratch for every -job, ensuring that the local working copy is always pristine. - -```yaml -variables: - GIT_STRATEGY: clone -``` - -`fetch` is faster as it re-uses the local working copy (falling back to `clone` -if it does not exist). `git clean` is used to undo any changes made by the last -job, and `git fetch` is used to retrieve commits made since the last job ran. - -```yaml -variables: - GIT_STRATEGY: fetch -``` - -`none` also re-uses the local working copy. However, it skips all Git operations, -including GitLab Runner's pre-clone script, if present. - -It's useful for jobs that operate exclusively on artifacts, like a deployment job. -Git repository data may be present, but it's likely out-of-date. You should only -rely on files brought into the local working copy from cache or artifacts. - -```yaml -variables: - GIT_STRATEGY: none -``` - -NOTE: **Note:** -`GIT_STRATEGY` is not supported for -[Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html), -but may be in the future. See the [support Git strategy with Kubernetes executor feature proposal](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3847) -for updates. - -### Git submodule strategy - -> Requires GitLab Runner v1.10+. - -The `GIT_SUBMODULE_STRATEGY` variable is used to control if / how Git -submodules are included when fetching the code before a build. You can set them -globally or per-job in the [`variables`](#variables) section. - -There are three possible values: `none`, `normal`, and `recursive`: - -- `none` means that submodules are not included when fetching the project - code. This is the default, which matches the pre-v1.10 behavior. - -- `normal` means that only the top-level submodules are included. It's - equivalent to: - - ```shell - git submodule sync - git submodule update --init - ``` - -- `recursive` means that all submodules (including submodules of submodules) - are included. This feature needs Git v1.8.1 and later. When using a - GitLab Runner with an executor not based on Docker, make sure the Git version - meets that requirement. It's equivalent to: - - ```shell - git submodule sync --recursive - git submodule update --init --recursive - ``` - -For this feature to work correctly, the submodules must be configured -(in `.gitmodules`) with either: - -- the HTTP(S) URL of a publicly-accessible repository, or -- a relative path to another repository on the same GitLab server. See the - [Git submodules](../git_submodules.md) documentation. - -### Git checkout - -> Introduced in GitLab Runner 9.3. - -The `GIT_CHECKOUT` variable can be used when the `GIT_STRATEGY` is set to either -`clone` or `fetch` to specify whether a `git checkout` should be run. If not -specified, it defaults to true. You can set them globally or per-job in the -[`variables`](#variables) section. - -If set to `false`, the runner: - -- when doing `fetch` - updates the repository and leaves the working copy on - the current revision, -- when doing `clone` - clones the repository and leaves the working copy on the - default branch. - -If `GIT_CHECKOUT` is set to `true`, both `clone` and `fetch` work the same way. -The runner checks out the working copy of a revision related -to the CI pipeline: - -```yaml -variables: - GIT_STRATEGY: clone - GIT_CHECKOUT: "false" -script: - - git checkout -B master origin/master - - git merge $CI_COMMIT_SHA -``` - -### Git clean flags - -> Introduced in GitLab Runner 11.10 - -The `GIT_CLEAN_FLAGS` variable is used to control the default behavior of -`git clean` after checking out the sources. You can set it globally or per-job in the -[`variables`](#variables) section. - -`GIT_CLEAN_FLAGS` accepts all possible options of the [`git clean`](https://git-scm.com/docs/git-clean) -command. - -`git clean` is disabled if `GIT_CHECKOUT: "false"` is specified. - -If `GIT_CLEAN_FLAGS` is: - -- Not specified, `git clean` flags default to `-ffdx`. -- Given the value `none`, `git clean` is not executed. - -For example: - -```yaml -variables: - GIT_CLEAN_FLAGS: -ffdx -e cache/ -script: - - ls -al cache/ -``` - -### Git fetch extra flags - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4142) in GitLab Runner 13.1. - -The `GIT_FETCH_EXTRA_FLAGS` variable is used to control the behavior of -`git fetch`. You can set it globally or per-job in the [`variables`](#variables) section. - -`GIT_FETCH_EXTRA_FLAGS` accepts all options of the [`git fetch`](https://git-scm.com/docs/git-fetch) command. However, `GIT_FETCH_EXTRA_FLAGS` flags are appended after the default flags that can't be modified. - -The default flags are: - -- [GIT_DEPTH](#shallow-cloning). -- The list of [refspecs](https://git-scm.com/book/en/v2/Git-Internals-The-Refspec). -- A remote called `origin`. - -If `GIT_FETCH_EXTRA_FLAGS` is: - -- Not specified, `git fetch` flags default to `--prune --quiet` along with the default flags. -- Given the value `none`, `git fetch` is executed only with the default flags. - -For example, the default flags are `--prune --quiet`, so you can make `git fetch` more verbose by overriding this with just `--prune`: - -```yaml -variables: - GIT_FETCH_EXTRA_FLAGS: --prune -script: - - ls -al cache/ -``` - -The configuration above results in `git fetch` being called this way: +[Docker service containers](../docker/using_docker_images.md#what-is-a-service). -```shell -git fetch origin $REFSPECS --depth 50 --prune -``` - -Where `$REFSPECS` is a value provided to the runner internally by GitLab. - -### Job stages attempts - -> Introduced in GitLab, it requires GitLab Runner v1.9+. - -You can set the number of attempts that the running job tries to execute -the following stages: - -| Variable | Description | -|-----------------------------------|--------------------------------------------------------| -| **ARTIFACT_DOWNLOAD_ATTEMPTS** | Number of attempts to download artifacts running a job | -| **EXECUTOR_JOB_SECTION_ATTEMPTS** | [In GitLab 12.10](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450) and later, the number of attempts to run a section in a job after a [`No Such Container`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450) error ([Docker executor](https://docs.gitlab.com/runner/executors/docker.html) only). | -| **GET_SOURCES_ATTEMPTS** | Number of attempts to fetch sources running a job | -| **RESTORE_CACHE_ATTEMPTS** | Number of attempts to restore the cache running a job | - -The default is one single attempt. - -Example: - -```yaml -variables: - GET_SOURCES_ATTEMPTS: 3 -``` - -You can set them globally or per-job in the [`variables`](#variables) section. - -### Fallback cache key - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4. - -You can use the `$CI_COMMIT_REF_SLUG` variable to specify your [`cache:key`](#cachekey). -For example, if your `$CI_COMMIT_REF_SLUG` is `test` you can set a job -to download cache that's tagged with `test`. - -If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to -specify a cache to use when none exists. - -For example: - -```yaml -variables: - CACHE_FALLBACK_KEY: fallback-key - -cache: - key: "$CI_COMMIT_REF_SLUG" - paths: - - binaries/ -``` - -In this example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined -by the `CACHE_FALLBACK_KEY` variable. - -### Shallow cloning - -> Introduced in GitLab 8.9 as an experimental feature. - -You can specify the depth of fetching and cloning using `GIT_DEPTH`. -`GIT_DEPTH` does a shallow clone of the repository and can significantly speed up cloning. -It can be helpful for repositories with a large number of commits or old, large binaries. The value is -passed to `git fetch` and `git clone`. - -In GitLab 12.0 and later, newly-created projects automatically have a -[default `git depth` value of `50`](../pipelines/settings.md#git-shallow-clone). - -If you use a depth of `1` and have a queue of jobs or retry -jobs, jobs may fail. - -Git fetching and cloning is based on a ref, such as a branch name, so runners -can't clone a specific commit SHA. If multiple jobs are in the queue, or -you're retrying an old job, the commit to be tested must be within the -Git history that is cloned. Setting too small a value for `GIT_DEPTH` can make -it impossible to run these old commits and `unresolved reference` is displayed in -job logs. You should then reconsider changing `GIT_DEPTH` to a higher value. - -Jobs that rely on `git describe` may not work correctly when `GIT_DEPTH` is -set since only part of the Git history is present. - -To fetch or clone only the last 3 commits: - -```yaml -variables: - GIT_DEPTH: "3" -``` - -You can set it globally or per-job in the [`variables`](#variables) section. - -### Custom build directories +You can use [YAML anchors for variables](#yaml-anchors-for-variables). -> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2211) in GitLab Runner 11.10. +### Configure runner behavior with variables -By default, GitLab Runner clones the repository in a unique subpath of the -`$CI_BUILDS_DIR` directory. However, your project might require the code in a -specific directory (Go projects, for example). In that case, you can specify -the `GIT_CLONE_PATH` variable to tell the runner the directory to clone the -repository in: +You can use [CI/CD variables](../variables/README.md) to configure runner Git behavior: -```yaml -variables: - GIT_CLONE_PATH: $CI_BUILDS_DIR/project-name - -test: - script: - - pwd -``` - -The `GIT_CLONE_PATH` has to always be within `$CI_BUILDS_DIR`. The directory set in `$CI_BUILDS_DIR` -is dependent on executor and configuration of [runners.builds_dir](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) -setting. +- [`GIT_STRATEGY`](../runners/README.md#git-strategy) +- [`GIT_SUBMODULE_STRATEGY`](../runners/README.md#git-submodule-strategy) +- [`GIT_CHECKOUT`](../runners/README.md#git-checkout) +- [`GIT_CLEAN_FLAGS`](../runners/README.md#git-clean-flags) +- [`GIT_FETCH_EXTRA_FLAGS`](../runners/README.md#git-fetch-extra-flags) +- [`GIT_DEPTH`](../runners/README.md#shallow-cloning) (shallow cloning) +- [`GIT_CLONE_PATH`](../runners/README.md#custom-build-directories) (custom build directories) -This can only be used when `custom_build_dir` is enabled in the -[runner's configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnerscustom_build_dir-section). -This is the default configuration for the `docker` and `kubernetes` executors. - -#### Handling concurrency - -An executor that uses a concurrency greater than `1` might lead -to failures. Multiple jobs might be working on the same directory if the `builds_dir` -is shared between jobs. - -The runner does not try to prevent this situation. It's up to the administrator -and developers to comply with the requirements of runner configuration. - -To avoid this scenario, you can use a unique path within `$CI_BUILDS_DIR`, because runner -exposes two additional variables that provide a unique `ID` of concurrency: - -- `$CI_CONCURRENT_ID`: Unique ID for all jobs running within the given executor. -- `$CI_CONCURRENT_PROJECT_ID`: Unique ID for all jobs running within the given executor and project. - -The most stable configuration that should work well in any scenario and on any executor -is to use `$CI_CONCURRENT_ID` in the `GIT_CLONE_PATH`. For example: - -```yaml -variables: - GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_CONCURRENT_ID/project-name - -test: - script: - - pwd -``` - -The `$CI_CONCURRENT_PROJECT_ID` should be used in conjunction with `$CI_PROJECT_PATH` -as the `$CI_PROJECT_PATH` provides a path of a repository. That is, `group/subgroup/project`. For example: - -```yaml -variables: - GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_CONCURRENT_ID/$CI_PROJECT_PATH - -test: - script: - - pwd -``` - -#### Nested paths - -The value of `GIT_CLONE_PATH` is expanded once and nesting variables -within is not supported. - -For example, you define both the variables below in your -`.gitlab-ci.yml` file: - -```yaml -variables: - GOPATH: $CI_BUILDS_DIR/go - GIT_CLONE_PATH: $GOPATH/src/namespace/project -``` - -The value of `GIT_CLONE_PATH` is expanded once into -`$CI_BUILDS_DIR/go/src/namespace/project`, and results in failure -because `$CI_BUILDS_DIR` is not expanded. +You can also use variables to configure how many times a runner +[attempts certain stages of job execution](../runners/README.md#job-stages-attempts). ## Special YAML features @@ -4573,7 +4204,7 @@ feature. Anchors are only valid within the file they were defined in. Instead of using YAML anchors, you can use the [`extends` keyword](#extends). The following example uses anchors and map merging. It creates two jobs, -`test1` and `test2`, that inherit the parameters of `.job_template`, each +`test1` and `test2`, that inherit the `.job_template` configuration, each with their own custom `script` defined: ```yaml @@ -4696,50 +4327,30 @@ test:mysql: You can see that the hidden jobs are conveniently used as templates, and `tags: [dev]` has been overwritten by `tags: [postgres]`. -#### YAML anchors for `before_script` and `after_script` +#### YAML anchors for scripts > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23005) in GitLab 12.5. -You can use [YAML anchors](#anchors) with `before_script` and `after_script`, -which makes it possible to include a predefined list of commands in multiple -jobs. - -Example: +You can use [YAML anchors](#anchors) with [script](#script), [`before_script`](#before_script), +and [`after_script`](#after_script) to use predefined commands in multiple jobs: ```yaml -.something_before: &something_before - - echo 'something before' - -.something_after: &something_after - - echo 'something after' - - echo 'another thing after' - -job_name: - before_script: - - *something_before - script: - - echo 'this is the script' - after_script: - - *something_after -``` - -#### YAML anchors for `script` +.some-script: &some-script + - echo "Execute this in `before_script` sections" -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23005) in GitLab 12.5. +.some-script-before: &some-script-before + - echo "Execute this in `script` sections" -You can use [YAML anchors](#anchors) with scripts, which makes it possible to -include a predefined list of commands in multiple jobs. - -For example: - -```yaml -.something: &something - - echo 'something' +.some-script-after: &some-script-after + - echo "Execute this in `after_script` sections" job_name: + before_script: + - *some-script-before script: - - *something - - echo 'this is the script' + - *some-script + before_script: + - *some-script-after ``` #### YAML anchors for variables @@ -4809,19 +4420,19 @@ This limitation does not affect any of the updated merge request pipelines. All updated merge requests have a pipeline created when using [pipelines for merge requests](../merge_request_pipelines/index.md). -## Deprecated parameters +## Deprecated keywords -The following parameters are deprecated. +The following keywords are deprecated. ### Globally-defined `types` -CAUTION: **Deprecated:** +DANGER: **Deprecated:** `types` is deprecated, and could be removed in a future release. Use [`stages`](#stages) instead. ### Job-defined `type` -CAUTION: **Deprecated:** +DANGER: **Deprecated:** `type` is deprecated, and could be removed in one of the future releases. Use [`stage`](#stage) instead. |