diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-01-20 09:16:11 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-01-20 09:16:11 +0000 |
| commit | edaa33dee2ff2f7ea3fac488d41558eb5f86d68c (patch) | |
| tree | 11f143effbfeba52329fb7afbd05e6e2a3790241 /doc/ci | |
| parent | d8a5691316400a0f7ec4f83832698f1988eb27c1 (diff) | |
| download | gitlab-ce-edaa33dee2ff2f7ea3fac488d41558eb5f86d68c.tar.gz | |
Add latest changes from gitlab-org/gitlab@14-7-stable-eev14.7.0-rc42
Diffstat (limited to 'doc/ci')
44 files changed, 1262 insertions, 505 deletions
diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 491454aed28..c634491662d 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -235,6 +235,39 @@ test_async: - node ./specs/start.js ./specs/async.spec.js ``` +#### Compute the cache key from the lock file + +You can use [`cache:key:files`](../yaml/index.md#cachekeyfiles) to compute the cache +key from a lock file like `package-lock.json` or `yarn.lock`, and reuse it in many jobs. + +```yaml +# Cache modules using lock file +cache: + key: + files: + - package-lock.json + paths: + - .npm/ +``` + +If you're using [Yarn](https://yarnpkg.com/), you can use [`yarn-offline-mirror`](https://classic.yarnpkg.com/blog/2016/11/24/offline-mirror/) +to cache the zipped `node_modules` tarballs. The cache generates more quickly, because +fewer files have to be compressed: + +```yaml +job: + script: + - echo 'yarn-offline-mirror ".yarn-cache/"' >> .yarnrc + - echo 'yarn-offline-mirror-pruning true' >> .yarnrc + - yarn install --frozen-lockfile --no-progress + cache: + key: + files: + - yarn.lock + paths: + - .yarn-cache/ +``` + ### Cache PHP dependencies If your project uses [Composer](https://getcomposer.org/) to install diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index cf4e846ebb4..0aa7f157602 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -65,8 +65,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In Bitbucket, add a script to push the pipeline status to Bitbucket. NOTE: - Changes made in GitLab are overwritten by any changes made - upstream in Bitbucket. + The changes must be made in Bitbucket as any changes in the GitLab repository are overwritten by Bitbucket when GitLab next mirrors the repository. Create a file `build_status` and insert the script below and run `chmod +x build_status` in your terminal to make the script executable. diff --git a/doc/ci/ci_cd_for_external_repos/img/ci_cd_for_external_repo.png b/doc/ci/ci_cd_for_external_repos/img/ci_cd_for_external_repo.png Binary files differdeleted file mode 100644 index f068688146b..00000000000 --- a/doc/ci/ci_cd_for_external_repos/img/ci_cd_for_external_repo.png +++ /dev/null diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 7bc138d083d..4e0cd7609cb 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -9,10 +9,6 @@ type: index, howto >[Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4642) in GitLab 10.6. -INFO: -Get external repo access and more by upgrading to GitLab Ultimate. -[Try a free 30-day trial now](https://about.gitlab.com/free-trial/index.html?glm_source=docs.gitlab.com&glm_content=p-ci-cd-external-docs). - GitLab CI/CD can be used with [GitHub](github_integration.md), [Bitbucket Cloud](bitbucket_integration.md), or any other Git server. diff --git a/doc/ci/cloud_services/aws/index.md b/doc/ci/cloud_services/aws/index.md new file mode 100644 index 00000000000..f09b23db2c5 --- /dev/null +++ b/doc/ci/cloud_services/aws/index.md @@ -0,0 +1,92 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Configure OpenID Connect in AWS to retrieve temporary credentials + +In this tutorial, we'll show you how to use a GitLab CI/CD job with a JSON web token (JWT) to retrieve temporary credentials from AWS without needing to store secrets. +To do this, you must configure OpenID Connect (OIDC) for ID federation between GitLab and AWS. For background and requirements for integrating GitLab using OIDC, see [Connect to cloud services](../index.md). + +To complete this tutorial: + +1. [Add the identity provider](#add-the-identity-provider) +1. [Configure the role and trust](#configure-a-role-and-trust) +1. [Retrieve a temporary credential](#retrieve-temporary-credentials) + +## Add the identity provider + +Create GitLab as a IAM OIDC provider in AWS following these [instructions](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html). + +Include the following information: + +- **Provider URL**: The address of your GitLab instance, such as `https://gitlab.com` or `http://gitlab.example.com`. +- **Audience**: The address of your GitLab instance, such as `https://gitlab.com` or `http://gitlab.example.com`. + - The address must include `https://`. + - Do not include a trailing slash. + +## Configure a role and trust + +After you create the identity provider, configure a [web identity role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp_oidc.html) with conditions for limiting access to GitLab resources. Temporary credentials are obtained using [AWS Security Token Service](https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html), so set the `Action` to [sts:AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html). + +For the full list of supported filtering types, see [Connect to cloud services](../index.md). + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Federated": "arn:aws:iam::AWS_ACCOUNT:oidc-provider/gitlab.example.com" + }, + "Action": "sts:AssumeRoleWithWebIdentity", + "Condition": { + "StringEquals": { + "gitlab.example.com:sub": "project_path:mygroup/myproject:ref_type:branch:ref:main" + } + } + } + ] +} +``` + +After the role is created, attach a policy defining permissions to an AWS service (S3, EC2, Secrets Manager). + +## Retrieve temporary credentials + +After you configure the OIDC and role, the GitLab CI/CD job can retrieve a temporary credential from [AWS Security Token Service (STS)](https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html). + +```yaml +assume role: + script: + - > + STS=($(aws sts assume-role-with-web-identity + --role-arn ${ROLE_ARN} + --role-session-name "GitLabRunner-${CI_PROJECT_ID}-${CI_PIPELINE_ID}" + --web-identity-token $CI_JOB_JWT_V2 + --duration-seconds 3600 + --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' + --output text)) + - export AWS_ACCESS_KEY_ID="${STS[0]}" + - export AWS_SECRET_ACCESS_KEY="${STS[1]}" + - export AWS_SESSION_TOKEN="${STS[2]}" + - aws sts get-caller-identity +``` + +- `CI_JOB_JWT_V2`: Predefined variable. +- `ROLE_ARN`: The role ARN defined in this [step](#configure-a-role-and-trust). + +## Working example + +See this [reference project](https://gitlab.com/guided-explorations/aws/configure-openid-connect-in-aws) for provisioning OIDC in AWS using Terraform and a sample script to retrieve temporary credentials. + +## Troubleshooting + +### `An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation: Not authorized to perform sts:AssumeRoleWithWebIdentity` + +This error can occur for multiple reasons: + +- The cloud administrator has not configured the project to use OIDC with GitLab. +- The role is restricted from being run on the branch or tag. See [configure a conditional role](../index.md). diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md new file mode 100644 index 00000000000..73e726ea8a9 --- /dev/null +++ b/doc/ci/cloud_services/index.md @@ -0,0 +1,133 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Connect to cloud services + +> - `CI_JOB_JWT` variable for reading secrets from Vault [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207125) in GitLab 12.10. +> - `CI_JOB_JWT_V2` variable to support additional OIDC providers [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346737) in GitLab 14.7. + +GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) that allows your build and deployment job access to cloud credentials and services. Historically, teams stored secrets in projects or applied permissions on the GitLab Runner instance to build and deploy. To support this, a predefined variable named `CI_JOB_JWT_V2` is included in the CI/CD job allowing you to follow a scalable and least-privilege security approach. + +## Requirements + +- Account on GitLab. +- Access to a cloud provider that supports OIDC to configure authorization and create roles. + +The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, GCP, and Vault. + +WARNING: +The `CI_JOB_JWT_V2` variable is under development [(alpha)](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha) and is not yet suitable for production use. + +## Use cases + +- Removes the need to store secrets in your GitLab group or project. Temporary credentials can be retrieved from your cloud provider through OIDC. +- Provides temporary access to cloud resources with granular GitLab conditionals including a group, project, branch, or tag. +- Enables you to define separation of duties in the CI/CD job with conditional access to environments. Historically, apps may have been deployed with a designated GitLab Runner that had only access to staging or production environments. This led to Runner sprawl as each machine had dedicated permissions. +- Allows shared runners to securely access multiple cloud accounts. The access is determined by the JWT token, which is specific to the user running the pipeline. +- Removes the need to create logic to rotate secrets by retrieving temporary credentials by default. + +## How it works + +Each job has a JSON web token (JWT) provided as a CI/CD [predefined variable](../variables/predefined_variables.md) named `CI_JOB_JWT` or `CI_JOB_JWT_V2`. This JWT can be used to authenticate with the OIDC-supported cloud provider such as AWS, GCP, or Vault. + +The following fields are included in the JWT: + +| Field | When | Description | +| ----------------------- | ------ | ----------- | +| `jti` | Always | Unique identifier for this token | +| `iss` | Always | Issuer, the domain of your GitLab instance | +| `iat` | Always | Issued at | +| `nbf` | Always | Not valid before | +| `exp` | Always | Expires at | +| `aud` | Always | Issuer, the domain of your GitLab instance | +| `sub` | Always |`project_path:{group}/{project}:ref_type:{type}:ref:{branch_name}` | +| `namespace_id` | Always | Use this to scope to group or user level namespace by ID | +| `namespace_path` | Always | Use this to scope to group or user level namespace by path | +| `project_id` | Always | Use this to scope to project by ID | +| `project_path` | Always | Use this to scope to project by path | +| `user_id` | Always | ID of the user executing the job | +| `user_login` | Always | Username of the user executing the job | +| `user_email` | Always | Email of the user executing the job | +| `pipeline_id` | Always | ID of this pipeline | +| `pipeline_source` | Always | [Pipeline source](../jobs/job_control.md#common-if-clauses-for-rules) | +| `job_id` | Always | ID of this job | +| `ref` | Always | Git ref for this job | +| `ref_type` | Always | Git ref type, either `branch` or `tag` | +| `ref_protected` | Always | `true` if this Git ref is protected, `false` otherwise | +| `environment` | Job is creating a deployment | Environment this job deploys to ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | +| `environment_protected` | Job is creating a deployment |`true` if deployed environment is protected, `false` otherwise ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | + +```json +{ + "jti": "c82eeb0c-5c6f-4a33-abf5-4c474b92b558", + "iss": "https://gitlab.example.com", + "aud": "https://gitlab.example.com", + "iat": 1585710286, + "nbf": 1585798372, + "exp": 1585713886, + "sub": "project_path:mygroup/myproject:ref_type:branch:ref:main", + "namespace_id": "1", + "namespace_path": "mygroup", + "project_id": "22", + "project_path": "mygroup/myproject", + "user_id": "42", + "user_login": "myuser", + "user_email": "myuser@example.com", + "pipeline_id": "1212", + "pipeline_source": "web", + "job_id": "1212", + "ref": "auto-deploy-2020-04-01", + "ref_type": "branch", + "ref_protected": "true", + "environment": "production", + "environment_protected": "true" +} +``` + +### Authorization workflow + +```mermaid +sequenceDiagram + participant GitLab + Note right of Cloud: Create OIDC identity provider + Note right of Cloud: Create role with conditionals + Note left of GitLab: CI/CD job with CI_JOB_JWT_V2 + GitLab->>+Cloud: Call cloud API with CI_JOB_JWT_V2 + Note right of Cloud: Decode & verify JWT with public key (https://gitlab/-/jwks) + Note right of Cloud: Validate audience defined in OIDC + Note right of Cloud: Validate conditional (sub, aud) role + Note right of Cloud: Generate credential or fetch secret + Cloud->>GitLab: Return temporary credential + Note left of GitLab: Perform operation + +``` + +1. Create an OIDC identity provider in the cloud (for example, AWS, GCP, Vault). +1. Create a conditional role in the cloud service that filters to a group, project, branch, or tag. +1. The CI/CD job includes a predefined variable `CI_JOB_JWT_V2` that is a JWT token. You can use this token for authorization with your cloud API. +1. The cloud verifies the token, validates the conditional role from the payload, and returns a temporary credential. + +## Configure a conditional role with OIDC claims + +To configure the trust between GitLab and OIDC, you must create a conditional role in the cloud provider that checks against the JWT token. The condition is validated against the JWT to create a trust specifically against two claims, the audience and subject. + +- Audience or `aud`: The URL of the GitLab instance. This is defined when the identity provider is first configured in your cloud provider. +- Subject or `sub`: A concatenation of metadata describing the GitLab CI/CD workflow including the group, project, branch, and tag. The `sub` field is in the following format: + - `project_path:{group}/{project}:ref_type:{type}:ref:{branch_name}` + +| Filter type | Example | +| ------------------------------------ | ------------------------------------------------------------ | +| Filter to main branch | `project_path:mygroup/myproject:ref_type:branch:ref:main` | +| Filter to any branch | Wildcard supported. `project_path:mygroup/myproject:ref_type:branch:ref:*` | +| Filter to specific project | `project_path:mygroup/myproject:ref_type:branch:ref:main` | +| Filter to all projects under a group | Wildcard supported. `project_path:mygroup/*:ref_type:branch:ref:main` | +| Filter to a Git tag | Wildcard supported. `project_path:mygroup/*:ref_type:tag:ref:1.0` | + +## OIDC authorization with your cloud provider + +To connect with your cloud provider, see the following tutorials: + +- [Configure OpenID Connect in AWS](aws/index.md) diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 3a05e9aa7d9..9b91cd40338 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -19,7 +19,7 @@ GitLab Runner to support `docker` commands. To enable Docker commands for your CI/CD jobs, you can use: - [The shell executor](#use-the-shell-executor) -- [The Docker executor with the Docker image (Docker-in-Docker)](#use-the-docker-executor-with-the-docker-image-docker-in-docker) +- [Docker-in-Docker](#use-docker-in-docker) - [Docker socket binding](#use-docker-socket-binding) If you don't want to execute a runner in privileged mode, @@ -78,54 +78,29 @@ You can now use `docker` commands (and install `docker-compose` if needed). When you add `gitlab-runner` to the `docker` group, you are effectively granting `gitlab-runner` full root permissions. Learn more about the [security of the `docker` group](https://blog.zopyx.com/on-docker-security-docker-group-considered-harmful/). -### Use the Docker executor with the Docker image (Docker-in-Docker) +### Use Docker-in-Docker "Docker-in-Docker" (`dind`) means: -- Your registered runner uses the [Docker executor](https://docs.gitlab.com/runner/executors/docker.html). +- Your registered runner uses the [Docker executor](https://docs.gitlab.com/runner/executors/docker.html) or the [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html). - The executor uses a [container image of Docker](https://hub.docker.com/_/docker/), provided by Docker, to run your CI/CD jobs. The Docker image has all of the `docker` tools installed and can run the job script in context of the image in privileged mode. -We recommend you use [Docker-in-Docker with TLS enabled](#docker-in-docker-with-tls-enabled), +We recommend you use Docker-in-Docker with TLS enabled, which is supported by [GitLab.com shared runners](../runners/index.md). You should always specify a specific version of the image, like `docker:19.03.12`. If you use a tag like `docker:stable`, you have no control over which version is used. Unpredictable behavior can result, especially when new versions are released. -#### Limitations of Docker-in-Docker +#### Use the Docker executor with Docker-in-Docker -Docker-in-Docker is the recommended configuration, but is -not without its own challenges: +You can use the Docker executor to run jobs in a Docker container. -- **The `docker-compose` command**: This command is not available in this configuration by default. - To use `docker-compose` in your job scripts, follow the `docker-compose` - [installation instructions](https://docs.docker.com/compose/install/). -- **Cache**: Each job runs in a new environment. Concurrent jobs work fine, - because every build gets its own instance of Docker engine and they don't conflict with each other. - However, jobs can be slower because there's no caching of layers. -- **Storage drivers**: By default, earlier versions of Docker use the `vfs` storage driver, - which copies the file system for each job. Docker 17.09 and later use `--storage-driver overlay2`, which is - the recommended storage driver. See [Using the OverlayFS driver](#use-the-overlayfs-driver) for details. -- **Root file system**: Because the `docker:19.03.12-dind` container and the runner container don't share their - root file system, you can use the job's working directory as a mount point for - child containers. For example, if you have files you want to share with a - child container, you might create a subdirectory under `/builds/$CI_PROJECT_PATH` - and use it as your mount point. For a more detailed explanation, view [issue - #41227](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41227). - - ```yaml - variables: - MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt - script: - - mkdir -p "$MOUNT_POINT" - - docker run -v "$MOUNT_POINT:/mnt" my-docker-image - ``` - -#### Docker-in-Docker with TLS enabled +##### Docker-in-Docker with TLS enabled in the Docker executor > Introduced in GitLab Runner 11.11. @@ -198,7 +173,7 @@ To use Docker-in-Docker with TLS enabled: # https://github.com/docker-library/docker/blob/d45051476babc297257df490d22cbd806f1b11e4/19.03/docker-entrypoint.sh#L23-L29 # # The 'docker' hostname is the alias of the service container as described at - # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. + # https://docs.gitlab.com/ee/ci/services/#accessing-the-services. # # Specify to Docker where to create the certificates. Docker # creates them automatically on boot, and creates @@ -219,7 +194,72 @@ To use Docker-in-Docker with TLS enabled: - docker run my-docker-image /script/to/run/tests ``` -#### Docker-in-Docker with TLS enabled in Kubernetes +##### Docker-in-Docker with TLS disabled in the Docker executor + +Sometimes you might have legitimate reasons to disable TLS. +For example, you have no control over the GitLab Runner configuration +that you are using. + +Assuming that the runner's `config.toml` is similar to: + +```toml +[[runners]] + url = "https://gitlab.com/" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.12" + privileged = true + disable_cache = false + volumes = ["/cache"] + [runners.cache] + [runners.cache.s3] + [runners.cache.gcs] +``` + +You can now use `docker` in the job script. Note the inclusion of the +`docker:19.03.12-dind` service: + +```yaml +image: docker:19.03.12 + +variables: + # When using dind service, you must instruct docker to talk with the + # daemon started inside of the service. The daemon is available with + # a network connection instead of the default /var/run/docker.sock socket. + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services + # + # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier, + # the variable must be set to tcp://localhost:2375 because of how the + # Kubernetes executor connects services to the job container + # DOCKER_HOST: tcp://localhost:2375 + # + DOCKER_HOST: tcp://docker:2375 + # + # This instructs Docker not to start over TLS. + DOCKER_TLS_CERTDIR: "" + +services: + - docker:19.03.12-dind + +before_script: + - docker info + +build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` + +#### Use the Kubernetes executor with Docker-in-Docker + +You can use the Kubernetes executor to run jobs in a Docker container. + +##### Docker-in-Docker with TLS enabled in Kubernetes > [Introduced](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/106) in GitLab Runner Helm Chart 0.23.0. @@ -257,7 +297,7 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: DOCKER_HOST: tcp://docker:2376 # # The 'docker' hostname is the alias of the service container as described at - # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. + # https://docs.gitlab.com/ee/ci/services/#accessing-the-services. # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier, # the variable must be set to tcp://localhost:2376 because of how the # Kubernetes executor connects services to the job container @@ -287,66 +327,34 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: - docker run my-docker-image /script/to/run/tests ``` -#### Docker-in-Docker with TLS disabled - -Sometimes you might have legitimate reasons to disable TLS. -For example, you have no control over the GitLab Runner configuration -that you are using. - -Assuming that the runner's `config.toml` is similar to: - -```toml -[[runners]] - url = "https://gitlab.com/" - token = TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:19.03.12" - privileged = true - disable_cache = false - volumes = ["/cache"] - [runners.cache] - [runners.cache.s3] - [runners.cache.gcs] -``` - -You can now use `docker` in the job script. Note the inclusion of the -`docker:19.03.12-dind` service: - -```yaml -image: docker:19.03.12 - -variables: - # When using dind service, you must instruct docker to talk with the - # daemon started inside of the service. The daemon is available with - # a network connection instead of the default /var/run/docker.sock socket. - # - # The 'docker' hostname is the alias of the service container as described at - # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services - # - # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier, - # the variable must be set to tcp://localhost:2375 because of how the - # Kubernetes executor connects services to the job container - # DOCKER_HOST: tcp://localhost:2375 - # - DOCKER_HOST: tcp://docker:2375 - # - # This instructs Docker not to start over TLS. - DOCKER_TLS_CERTDIR: "" +#### Limitations of Docker-in-Docker -services: - - docker:19.03.12-dind +Docker-in-Docker is the recommended configuration, but is +not without its own challenges: -before_script: - - docker info +- **The `docker-compose` command**: This command is not available in this configuration by default. + To use `docker-compose` in your job scripts, follow the `docker-compose` + [installation instructions](https://docs.docker.com/compose/install/). +- **Cache**: Each job runs in a new environment. Concurrent jobs work fine, + because every build gets its own instance of Docker engine and they don't conflict with each other. + However, jobs can be slower because there's no caching of layers. +- **Storage drivers**: By default, earlier versions of Docker use the `vfs` storage driver, + which copies the file system for each job. Docker 17.09 and later use `--storage-driver overlay2`, which is + the recommended storage driver. See [Using the OverlayFS driver](#use-the-overlayfs-driver) for details. +- **Root file system**: Because the `docker:19.03.12-dind` container and the runner container don't share their + root file system, you can use the job's working directory as a mount point for + child containers. For example, if you have files you want to share with a + child container, you might create a subdirectory under `/builds/$CI_PROJECT_PATH` + and use it as your mount point. For a more detailed explanation, view [issue + #41227](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41227). -build: - stage: build + ```yaml + variables: + MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests -``` + - mkdir -p "$MOUNT_POINT" + - docker run -v "$MOUNT_POINT:/mnt" my-docker-image + ``` ### Use Docker socket binding @@ -359,87 +367,50 @@ If you bind the Docker socket and you are you can no longer use `docker:19.03.12-dind` as a service. Volume bindings are done to the services as well, making these incompatible. -To make Docker available in the context of the image: +#### Use the Docker executor with Docker socket binding -1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). -1. From the command line, register a runner with the `docker` executor and share `/var/run/docker.sock`: +To make Docker available in the context of the image, you will need to mount +`/var/run/docker.sock` into the launched containers. To do this with the Docker +executor, you need to add `"/var/run/docker.sock:/var/run/docker.sock"` to the +[Volumes in the `[runners.docker]` section](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section). - ```shell - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:19.03.12" \ - --docker-volumes /var/run/docker.sock:/var/run/docker.sock - ``` - - This command registers a new runner to use the - `docker:19.03.12` image provided by Docker. The command uses - the Docker daemon of the runner itself. Any containers spawned by Docker - commands are siblings of the runner rather than children of the runner. - This may have complications and limitations that are unsuitable for your workflow. - - Your `config.toml` file should now have an entry like this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = REGISTRATION_TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:19.03.12" - privileged = false - disable_cache = false - volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] - [runners.cache] - Insecure = false - ``` - -1. Use `docker` in the job script. You don't need to - include the `docker:19.03.12-dind` service, like you do when you're using - the Docker-in-Docker executor: - - ```yaml - image: docker:19.03.12 - - before_script: - - docker info +Your configuration should look something like this: - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` - -This method avoids using Docker in privileged mode. However, -the implications of this method are: +```toml +[[runners]] + url = "https://gitlab.com/" + token = RUNNER_TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.12" + privileged = false + disable_cache = false + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] + [runners.cache] + Insecure = false +``` -- By sharing the Docker daemon, you are effectively disabling all - the security mechanisms of containers and exposing your host to privilege - escalation, which can lead to container breakout. For example, if a project - ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner - containers. -- Concurrent jobs may not work; if your tests - create containers with specific names, they may conflict with each other. -- Sharing files and directories from the source repository into containers may not - work as expected. Volume mounting is done in the context of the host - machine, not the build container. For example: +You can also do this while registering your runner by providing the following options: - ```shell - docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests - ``` +```shell +sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:19.03.12" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock +``` -#### Enable registry mirror for `docker:dind` service +##### Enable registry mirror for `docker:dind` service When the Docker daemon starts inside of the service container, it uses the default configuration. You may want to configure a [registry mirror](https://docs.docker.com/registry/recipes/mirror/) for performance improvements and to ensure you don't reach Docker Hub rate limits. -##### The service in the `.gitlab-ci.yml` file +###### The service in the `.gitlab-ci.yml` file You can append extra CLI flags to the `dind` service to set the registry mirror: @@ -450,7 +421,7 @@ services: command: ["--registry-mirror", "https://registry-mirror.example.com"] # Specify the registry mirror to use ``` -##### The service in the GitLab Runner configuration file +###### The service in the GitLab Runner configuration file > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27173) in GitLab Runner 13.6. @@ -487,7 +458,7 @@ Kubernetes: command = ["--registry-mirror", "https://registry-mirror.example.com"] ``` -##### The Docker executor in the GitLab Runner configuration file +###### The Docker executor in the GitLab Runner configuration file If you are a GitLab Runner administrator, you can use the mirror for every `dind` service. Update the @@ -520,7 +491,7 @@ picked up by the `dind` service. volumes = ["/opt/docker/daemon.json:/etc/docker/daemon.json:ro"] ``` -##### The Kubernetes executor in the GitLab Runner configuration file +###### The Kubernetes executor in the GitLab Runner configuration file > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3223) in GitLab Runner 13.6. @@ -569,6 +540,45 @@ The configuration is picked up by the `dind` service. sub_path = "daemon.json" ``` +##### Limitations of Docker socket binding + +When you use Docker socket binding, you avoid running Docker in privileged mode. However, +the implications of this method are: + +- By sharing the Docker daemon, you are effectively disabling all + the security mechanisms of containers and exposing your host to privilege + escalation, which can lead to container breakout. For example, if a project + ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner + containers. +- Concurrent jobs may not work; if your tests + create containers with specific names, they may conflict with each other. +- Any containers spawned by Docker commands are siblings of the runner rather + than children of the runner. This may have complications and limitations that + are unsuitable for your workflow. +- Sharing files and directories from the source repository into containers may not + work as expected. Volume mounting is done in the context of the host + machine, not the build container. For example: + + ```shell + docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests + ``` + +You don't need to include the `docker:19.03.12-dind` service, like you do when +you're using the Docker-in-Docker executor: + +```yaml +image: docker:19.03.12 + +before_script: + - docker info + +build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` + ## Authenticate with registry in Docker-in-Docker When you use Docker-in-Docker, the @@ -831,13 +841,13 @@ After you've built a Docker image, you can push it up to the built-in ### `docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?` This is a common error when you are using -[Docker-in-Docker](#use-the-docker-executor-with-the-docker-image-docker-in-docker) +[Docker-in-Docker](#use-docker-in-docker) v19.03 or later. This issue occurs because Docker starts on TLS automatically. - If this is your first time setting it up, read - [use the Docker executor with the Docker image](#use-the-docker-executor-with-the-docker-image-docker-in-docker). + [use the Docker executor with the Docker image](#use-docker-in-docker). - If you are upgrading from v18.09 or earlier, read our [upgrade guide](https://about.gitlab.com/blog/2019/07/31/docker-in-docker-with-docker-19-dot-03/). diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index def7703231c..5bd9293924d 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -401,10 +401,10 @@ pulling from Docker Hub fails. Docker daemon tries to use the same credentials f > Introduced in GitLab Runner 12.0. -As an example, let's assume that you want to use the `aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest` +As an example, let's assume that you want to use the `<aws_account_id>.dkr.ecr.<region>.amazonaws.com/private/image:latest` image. This image is private and requires you to log in into a private container registry. -To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow these steps: +To configure access for `<aws_account_id>.dkr.ecr.<region>.amazonaws.com`, follow these steps: 1. Make sure `docker-credential-ecr-login` is available in the GitLab Runner `$PATH`. 1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials). @@ -418,7 +418,7 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th ```json { "credHelpers": { - "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" + "<aws_account_id>.dkr.ecr.<region>.amazonaws.com": "ecr-login" } } ``` @@ -438,14 +438,14 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th GitLab Runner reads this configuration file and uses the needed helper for this specific repository. -1. You can now use any private image from `aws_account_id.dkr.ecr.region.amazonaws.com` defined in +1. You can now use any private image from `<aws_account_id>.dkr.ecr.<region>.amazonaws.com` defined in `image` and/or `services` in your `.gitlab-ci.yml` file: ```yaml - image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest + image: <aws_account_id>.dkr.ecr.<region>.amazonaws.com/private/image:latest ``` - In the example, GitLab Runner looks at `aws_account_id.dkr.ecr.region.amazonaws.com` for the + In the example, GitLab Runner looks at `<aws_account_id>.dkr.ecr.<region>.amazonaws.com` for the image `private/image:latest`. You can add configuration for as many registries as you want, adding more diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index ea3e81329d3..098ec9bdd02 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -13,7 +13,7 @@ type: howto container images from a Dockerfile, inside a container or Kubernetes cluster. kaniko solves two problems with using the -[Docker-in-Docker build](using_docker_build.md#use-the-docker-executor-with-the-docker-image-docker-in-docker) +[Docker-in-Docker build](using_docker_build.md#use-docker-in-docker) method: - Docker-in-Docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md new file mode 100644 index 00000000000..d60e5704877 --- /dev/null +++ b/doc/ci/environments/deployment_approvals.md @@ -0,0 +1,114 @@ +--- +stage: Release +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +description: Require approvals prior to deploying to a Protected Environment +--- + +# Deployment approvals **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343864) in GitLab 14.7 with a flag named `deployment_approvals`. Disabled by default. + +WARNING: +This feature is in an alpha stage and subject to change without prior notice. + +It may be useful to require additional approvals before deploying to certain protected environments (for example, production). This pre-deployment approval requirement is useful to accommodate testing, security, or compliance processes that must happen before each deployment. + +When a protected environment requires one or more approvals, all deployments to that environment become blocked and wait for the required approvals before running. + +NOTE: +See the [epic](https://gitlab.com/groups/gitlab-org/-/epics/6832) for planned features. + +## Requirements + +- Basic knowledge of [GitLab Environments and Deployments](index.md). +- Basic knowledge of [Protected Environments](protected_environments.md). + +## Configure deployment approvals for a project + +To configure deployment approvals for a project: + +1. [Create a deployment job](#create-a-deployment-job). +1. [Require approvals for a protected environment](#require-approvals-for-a-protected-environment). + +### Create a deployment job + +Create a deployment job in the `.gitlab-ci.yaml` file of the desired project. The job does **not** need to be manual (`when: manual`). + +Example: + + ```yaml + stages: + - deploy + + production: + stage: deploy + script: + - 'echo "Deploying to ${CI_ENVIRONMENT_NAME}"' + environment: + name: ${CI_JOB_NAME} + ``` + +### Require approvals for a protected environment + +NOTE: +At this time, only API-based configuration is available. UI-based configuration is planned for the near future. See [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/344675). + +Use the [Protected Environments API](../../api/protected_environments.md#protect-repository-environments) to create an environment with `required_approval_count` > 0. After this is set, all jobs deploying to this environment automatically go into a blocked state and wait for approvals before running. + +Example: + +```shell +curl --header 'Content-Type: application/json' --request POST \ + --data '{"name": "production", "deploy_access_levels": [{"group_id": 9899826}], "required_approval_count": 1}' \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/22034114/protected_environments" +``` + +To protect, update, or unprotect an environment, you must have at least the +[Maintainer role](../../user/permissions.md). + +## Approve or reject a deployment + +NOTE: +This functionality is currently only available through the API. UI is planned for the near future. See [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342180/). + +A blocked deployment is enqueued as soon as it receives the required number of approvals. A single rejection causes the deployment to fail. The creator of a deployment cannot approve it, even if they have permission to deploy. + +Using the [Deployments API](../../api/deployments.md#approve-or-reject-a-blocked-deployment), users who are allowed to deploy to the protected environment can approve or reject a blocked deployment. + +Example: + +```shell +curl --data "status=approved" \ + --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/deployments/1/approval" +``` + +### How to see blocked deployments + +#### Using the UI + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. +1. Select the environment being deployed to. +1. Look for the `blocked` label. + +#### Using the API + +Use the [Deployments API](../../api/deployments.md) to see deployments. The `status` field indicates if a deployment is blocked. + +## Related features + +For details about other GitLab features aimed at protecting deployments, see [safe deployments](deployment_safety.md). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index ca7b01edf39..55c3c83338d 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -144,6 +144,10 @@ appropriate permissions in the other project. For more information, see [Custom CI/CD configuration path](../pipelines/settings.md#specify-a-custom-cicd-configuration-file). +## Require an approval before deploying + +Before promoting a deployment to a production environment, cross-verifying it with a dedicated testing group is an effective way to ensure safety. For more information, see [Deployment Approvals](deployment_approvals.md). + ## Troubleshooting ### Pipelines jobs fail with `The deployment job is older than the previously succeeded deployment job...` diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 561507cab97..794e4320fc6 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -166,7 +166,7 @@ deploy_prod: The `when: manual` action: -- Exposes a play button for the job in the GitLab UI. +- Exposes a play button for the job in the GitLab UI, with the text **Can be manually deployed to <environment>**. - Means the `deploy_prod` job is only triggered when the play button is clicked. You can find the play button in the pipelines, environments, deployments, and jobs views. @@ -774,10 +774,8 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/* ### Archive Old Deployments > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73628) in GitLab 14.5. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.6. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `deployments_archive`. On GitLab.com, this feature is available. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/345027) in GitLab 14.6. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73628) in GitLab 14.0. [Feature flag `deployments_archive`](https://gitlab.com/gitlab-org/gitlab/-/issues/345027) removed. When a new deployment happens in your project, GitLab creates [a special Git-ref to the deployment](#check-out-deployments-locally). diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 57fd72863c1..78db2345de4 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -248,6 +248,10 @@ NOTE: Configuration [with the UI](https://gitlab.com/gitlab-org/gitlab/-/issues/325249) is scheduled for a later release. +## Deployment approvals + +Protected environments can also be used to require manual approvals before deployments. See [Deployment approvals](deployment_approvals.md) for more information. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index 1141583df3f..aed45951239 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -189,6 +189,28 @@ Combined with [protected branches](../../../user/project/protected_branches.md), [`bound_claims_type`](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values are interpreted as globs, with `*` matching any number of characters. +The claim fields listed in [the table above](#how-it-works) can also be accessed for [Vault's policy path templating](https://learn.hashicorp.com/tutorials/vault/policy-templating?in=vault/policies) purposes by using the accessor name of the JWT auth within Vault. The [mount accessor name](https://learn.hashicorp.com/tutorials/vault/identity#step-1-create-an-entity-with-alias) (`ACCESSOR_NAME` in the example below) can be retrieved by running `vault auth list`. + +Policy template example making use of a named metadata field named `project_path`: + +```plaintext +path "secret/data/{{identity.entity.aliases.ACCESSOR_NAME.metadata.project_path}}/staging/*" { + capabilities = [ "read" ] +} +``` + +Role example to support the templated policy above, mapping the claim field `project_path` as a metadata field through use of [`claim_mappings`](https://www.vaultproject.io/api-docs/auth/jwt#claim_mappings) configuration: + +```plaintext +{ + "role_type": "jwt", + ... + "claim_mappings": { + "project_path": "project_path" + } +} +``` + For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). WARNING: diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index a9794afaf10..9881c9657bc 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -4,8 +4,6 @@ group: Testing info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments author: Vincent Tunru author_gitlab: Vinnl -type: tutorial -date: 2019-02-18 description: 'Confidence checking your entire app every time a new feature is added can quickly become repetitive. Learn how to automate it with GitLab CI/CD.' --- diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index e2e12235eba..be33e62b75c 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -5,8 +5,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w disqus_identifier: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' author: Mehran Rasulian author_gitlab: mehranrasulian -type: tutorial -date: 2017-08-31 --- <!-- vale off --> diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 2a002b8fb9f..cdc75fd2bec 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -64,3 +64,14 @@ If you use the [`CI_JOB_TOKEN`](jobs/ci_job_token.md) to clone a submodule in a pipeline job, the user executing the job must be assigned to a role that has [permission](../user/permissions.md#gitlab-cicd-permissions) to trigger a pipeline in the upstream submodule project. + +## Troubleshooting + +### Can't find the `.gitmodules` file + +The `.gitmodules` file might be hard to find because it is usually a hidden file. +You can check documentation for your specific OS to learn how to find and display +hidden files. + +If there is no `.gitmodules` file, it's possible the submodule settings are in a +[gitconfig](https://www.atlassian.com/git/tutorials/setting-up-a-repository/git-config) file. diff --git a/doc/ci/index.md b/doc/ci/index.md index 5dcb0bcb242..c557e9e6f57 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -81,6 +81,7 @@ GitLab CI/CD features, grouped by DevOps stage, include: | **Configure** | | | [Auto DevOps](../topics/autodevops/index.md) | Set up your app's entire lifecycle. | | [ChatOps](chatops/index.md) | Trigger CI jobs from chat, with results sent back to the channel. | +| [Connect to cloud services](cloud_services/index.md) | Connect to cloud providers using OpenID Connect (OIDC) to retrieve temporary credentials to access services or secrets. | |-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| | **Verify** | | | [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index 5724c56b096..f49fdd6c39f 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -32,10 +32,18 @@ Two things need to be configured for the interactive web terminal to work: - If you are using a reverse proxy with your GitLab instance, web terminals need to be [enabled](../../administration/integration/terminal.md#enabling-and-disabling-terminal-support) -NOTE: -Interactive web terminals are not yet supported by -[`gitlab-runner` Helm chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-runner/index.html). -Support is tracked [in this issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/79). +### Partial support for Helm chart + +Interactive web terminals are partially supported in `gitlab-runner` Helm chart. +They are enabled when: + +- The number of replica is one +- You use the `loadBalancer` service + +Support for fixing these limitations is tracked in the following issues: + +- [Support of more than one replica](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/323) +- [Support of more service types](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/324) ## Debugging a running job diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index 532a0dffbce..1906d9cdb6c 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -24,7 +24,10 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Terraform plan](../../user/infrastructure/index.md). The token has the same permissions to access the API as the user that executes the -job. Therefore, this user must be assigned to [a role that has the required privileges](../../user/permissions.md#gitlab-cicd-permissions). +The token has the same permissions to access the API as the user that caused the +job to run. A user can cause a job to run by pushing a commit, triggering a manual job, +being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to +[a role that has the required privileges](../../user/permissions.md#gitlab-cicd-permissions). The token is valid only while the pipeline job runs. After the job finishes, you can't use the token anymore. diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 596df34b5c2..a2406a68bb2 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -404,10 +404,7 @@ build: If you change multiple files, but only one file ends in `.md`, the `build` job is still skipped. The job does not run for any of the files. -Read more about how to use `only:changes` and `except:changes`: - -- [New branches or tags *without* pipelines for merge requests](#use-onlychanges-without-pipelines-for-merge-requests). -- [Scheduled pipelines](#use-onlychanges-with-scheduled-pipelines). +With some configurations that use `changes`, [jobs or pipelines might run unexpectedly](#jobs-or-pipelines-run-unexpectedly-when-using-changes) #### Use `only:changes` with pipelines for merge requests @@ -459,22 +456,6 @@ it doesn't matter that an earlier pipeline failed because of a change that has n When you use this configuration, ensure that the most recent pipeline properly corrects any failures from previous pipelines. -#### Use `only:changes` without pipelines for merge requests - -Without [pipelines for merge requests](../pipelines/merge_request_pipelines.md), pipelines -run on branches or tags that don't have an explicit association with a merge request. -In this case, a previous SHA is used to calculate the diff, which is equivalent to `git diff HEAD~`. -This can result in some unexpected behavior, including: - -- When pushing a new branch or a new tag to GitLab, the policy always evaluates to true. -- When pushing a new commit, the changed files are calculated by using the previous commit - as the base SHA. - -#### Use `only:changes` with scheduled pipelines - -`only:changes` always evaluates as true in [Scheduled pipelines](../pipelines/schedules.md). -All files are considered to have changed when a scheduled pipeline runs. - ### Combine multiple keywords with `only` or `except` If you use multiple keywords with `only` or `except`, the keywords are evaluated @@ -777,7 +758,7 @@ job1: - echo rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" - - if: $CI_PIPELINE_SOURCE == "scheduled" + - if: $CI_PIPELINE_SOURCE == "schedule" - if: $CI_PIPELINE_SOURCE == "push" when: never ``` @@ -943,3 +924,23 @@ For example: - `($VARIABLE1 =~ /^content.*/ || $VARIABLE2) && ($VARIABLE3 =~ /thing$/ || $VARIABLE4)` - `($VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/) && $VARIABLE3` - `$CI_COMMIT_BRANCH == "my-branch" || (($VARIABLE1 == "thing" || $VARIABLE2 == "thing") && $VARIABLE3)` + +## Troubleshooting + +### Jobs or pipelines run unexpectedly when using `changes:` + +You might have jobs or pipelines that run unexpectedly when using [`rules: changes`](../yaml/index.md#ruleschanges) +or [`only: changes`](../yaml/index.md#onlychanges--exceptchanges) without +[pipelines for merge requests](../pipelines/merge_request_pipelines.md). + +Pipelines on branches or tags that don't have an explicit association with a merge request +use a previous SHA to calculate the diff. This calculation is equivalent to `git diff HEAD~` +and can cause unexpected behavior, including: + +- The `changes` rule always evaluates to true when pushing a new branch or a new tag to GitLab. +- When pushing a new commit, the changed files are calculated by using the previous commit + as the base SHA. + +Additionally, rules with `changes` always evaluate as true in [scheduled pipelines](../pipelines/schedules.md). +All files are considered to have changed when a scheduled pipeline runs, so jobs +might always be added to scheduled pipelines that use `changes`. diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index 5be016aff40..d72cb14681d 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -83,7 +83,41 @@ where: - Configuration imported with [`include`](../yaml/index.md#include) is copied into the view. - Jobs that use [`extends`](../yaml/index.md#extends) display with the [extended configuration merged into the job](../yaml/yaml_optimization.md#merge-details). -- YAML anchors are [replaced with the linked configuration](../yaml/yaml_optimization.md#anchors). +- [YAML anchors](../yaml/yaml_optimization.md#anchors) are replaced with the linked configuration. +- [YAML `!reference` tags](../yaml/yaml_optimization.md#reference-tags) are also replaced + with the linked configuration. + +Using `!refence` tags can cause nested configuration that display with +multiple hyphens (`-`) in the expanded view. This behavior is expected, and the extra +hyphens do not affect the job's execution. For example, this configuration and +fully expanded version are both valid: + +- `.gitlab-ci.yml` file: + + ```yaml + .python-req: + script: + - pip install pyflakes + + lint-python: + image: python:latest + script: + - !reference [.python-req, script] + - pyflakes python/ + ``` + +- Expanded configuration in **View merged YAML** tab: + + ```yaml + ".python-req": + script: + - pip install pyflakes + lint-python: + script: + - - pip install pyflakes # <- The extra hyphens do not affect the job's execution. + - pyflakes python/ + image: python:latest + ``` ## Commit changes to CI configuration @@ -97,3 +131,20 @@ If you enter a new branch name, the **Start a new merge request with these chang checkbox appears. Select it to start a new merge request after you commit the changes.  + +## Troubleshooting + +### `Configuration validation currently not available` message + +This message is due to a problem with the syntax validation in the pipeline editor. +If GitLab is unable to communicate with the service that validates the syntax, the +information in these sections may not display properly: + +- The syntax status on the **Edit** tab (valid or invalid). +- The **Visualize** tab. +- The **Lint** tab. +- The **View merged YAML** tab. + +You can still work on your CI/CD configuration and commit the changes you made without +any issues. As soon as the service becomes available again, the syntax validation +should display immediately. diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md new file mode 100644 index 00000000000..e0fb5b45986 --- /dev/null +++ b/doc/ci/pipelines/cicd_minutes.md @@ -0,0 +1,221 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# CI/CD minutes quota **(PREMIUM)** + +[Shared runners](../runners/runners_scope.md#shared-runners) are shared with every project and group in a GitLab instance. +When jobs run on shared runners, CI/CD minutes are used. + +You can set limits on the number of CI/CD minutes that are used each month. + +- On GitLab.com, the quota of CI/CD minutes is set for each [namespace](../../user/group/index.md#namespaces), + and is determined by [your license tier](https://about.gitlab.com/pricing/). +- On self-managed GitLab instances, the quota of CI/CD minutes for each namespace is set by administrators. + +In addition to the monthly quota, you can add more CI/CD minutes when needed. + +- On GitLab.com, you can [purchase additional CI/CD minutes](#purchase-additional-cicd-minutes). +- On self-managed GitLab instances, administrators can [assign more CI/CD minutes](#set-the-quota-of-cicd-minutes-for-a-specific-namespace). + +[Specific runners](../runners/runners_scope.md#specific-runners) +are not subject to a quota of CI/CD minutes. + +## Set the quota of CI/CD minutes for all namespaces + +> [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. + +By default, GitLab instances do not have a quota of CI/CD minutes. +The default value for the quota is `0`, which grants unlimited CI/CD minutes. +However, you can change this default value. + +Prerequisite: + +- You must be a GitLab administrator. + +To change the default quota that applies to all namespaces: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment**. +1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. +1. Select **Save changes**. + +If a quota is already defined for a specific namespace, this value does not change that quota. + +## Set the quota of CI/CD minutes for a specific namespace + +> [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. + +You can override the global value and set a quota of CI/CD minutes +for a specific namespace. + +Prerequisite: + +- You must be a GitLab administrator. + +To set a quota of CI/CD minutes for a namespace: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Overview > Groups**. +1. For the group you want to update, select **Edit**. +1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. +1. Select **Save changes**. + +You can also use the [update group API](../../api/groups.md#update-group) or the +[update user API](../../api/users.md#user-modification) instead. + +NOTE: +You can set a quota of CI/CD minutes for only top-level groups or user namespaces. +If you set a quota for a subgroup, it is not used. + +## View CI/CD minutes used by a group + +You can view the number of CI/CD minutes being used by a group. + +Prerequisite: + +- You must have the Owner role for the group. + +To view CI/CD minutes being used for your group: + +1. On the top bar, select **Menu > Groups** and find your group. The group must not be a subgroup. +1. On the left sidebar, select **Settings > Usage Quotas**. +1. Select the **Pipelines** tab. + + + +## View CI/CD minutes used by a personal namespace + +You can view the number of CI/CD minutes being used by a personal namespace: + +1. On the top bar, in the top right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Usage Quotas**. + +## Purchase additional CI/CD minutes **(FREE SAAS)** + +If you're using GitLab SaaS, you can purchase additional packs of CI/CD minutes. +These additional CI/CD minutes: + +- Are used only after the monthly quota included in your subscription runs out. +- Are carried over to the next month, if any remain at the end of the month. +- Don't expire. + +If you use more CI/CD minutes than your monthly quota, when you purchase more, +those CI/CD minutes are deducted from your quota. For example, with a GitLab SaaS +Premium license: + +- You have `10,000` monthly minutes. +- You purchase an additional `5,000` minutes. +- Your total limit is `15,000` minutes. + +If you use `13,000` minutes during the month, the next month your additional minutes become +`2,000`. If you use `9,000` minutes during the month, your additional minutes remain the same. + +You can find pricing for additional CI/CD minutes on the +[GitLab Pricing page](https://about.gitlab.com/pricing/). + +### Purchase CI/CD minutes for a group **(FREE SAAS)** + +You can purchase additional CI/CD minutes for your group. +You cannot transfer purchased CI/CD minutes from one group to another, +so be sure to select the correct group. + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Usage Quotas**. +1. Select **Buy additional minutes**. +1. Complete the details of the transaction. + +After your payment is processed, the additional CI/CD minutes are added to your group +namespace. + +### Purchase CI/CD minutes for a personal namespace **(FREE SAAS)** + +To purchase additional minutes for your personal namespace: + +1. On the top bar, in the top right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Usage Quotas**. +1. Select **Buy additional minutes**. GitLab redirects you to the Customers Portal. +1. Locate the subscription card that's linked to your personal namespace on GitLab SaaS, select **Buy more CI minutes**, + and complete the details of the transaction. + +After your payment is processed, the additional CI/CD minutes are added to your personal +namespace. + +## How CI/CD minutes are calculated + +CI/CD minutes are calculated based on: + +- The duration the job runs. +- The visibility of the projects where the job runs. + +GitLab uses this formula to calculate CI/CD minutes consumed by a job: + +```plaintext +Job duration * Cost factor +``` + +- **Job duration**: The time, in seconds, that a job took to run on a shared runner. + It does not include time spent in `created` or `pending` status. +- **Cost factor**: A number based on project visibility. + +The number is transformed into minutes and added to the overall quota in the job's top-level namespace. + +For example: + +- A user, `alice`, runs a pipeline under the `gitlab-org` namespace. +- The CI/CD minutes consumed by each job in the pipeline are added to the + overall consumption for the `gitlab-org` namespace, not the `alice` namespace. +- If a pipeline runs for one of the personal projects for `alice`, the CI/CD minutes + are added to the overall consumption for the `alice` namespace. + +### Cost factor + +The cost factor for a job running on a shared runner is: + +- `0.008` for public projects on GitLab SaaS, if [created 2021-07-17 or later](https://gitlab.com/gitlab-org/gitlab/-/issues/332708). + (For every 125 minutes of job time, you accrue 1 CD/CD minute.) +- `0.008` for projects members of GitLab [Open Source program](../../subscriptions/index.md#gitlab-for-open-source). + (For every 125 minutes of job time, you accrue 1 CD/CD minute.) +- `0` for public projects on GitLab self-managed instances, and for GitLab SaaS public projects created before 2021-07-17. +- `1` for internal and private projects. + +### Additional costs on GitLab SaaS + +On GitLab SaaS, shared runners can have different cost factors depending on the cost involved +in executing the runner. For example, a high spec shared runner could be set to have a cost factor of `2`. +Conversely, a shared runner that executes jobs for public projects could have a low cost factor, like `0.008`. + +### Monthly reset of CI/CD minutes + +On the first day of each calendar month, the accumulated usage of CI/CD minutes is reset to `0` +for all namespaces that use shared runners. + +Usage data for the previous month is kept to show historical view of the consumption over time. + +## What happens when you exceed the quota + +When the quota of CI/CD minutes is used for the current month, GitLab stops +processing new jobs. + +- Any non-running job that should be picked by shared runners is automatically dropped. +- Any job being retried is automatically dropped. +- Any running job can be dropped at any point if the overall namespace usage goes over-quota + by a grace period. + +The grace period for running jobs is `1,000` CI/CD minutes. + +Jobs on specific runners are not affected by the quota of CI/CD minutes. + +### GitLab SaaS usage notifications + +On GitLab SaaS an email notification is sent to the namespace owners when: + +- The available CI/CD minutes are below 30% of the quota. +- The available CI/CD minutes are below 5% of the quota. +- All CI/CD minutes have been used. diff --git a/doc/ci/pipelines/img/group_cicd_minutes_quota.png b/doc/ci/pipelines/img/group_cicd_minutes_quota.png Binary files differnew file mode 100644 index 00000000000..318527426bd --- /dev/null +++ b/doc/ci/pipelines/img/group_cicd_minutes_quota.png diff --git a/doc/ci/pipelines/img/pipeline-fork_v13_7.png b/doc/ci/pipelines/img/pipeline_fork_v13_7.png Binary files differindex eb44290aa66..eb44290aa66 100644 --- a/doc/ci/pipelines/img/pipeline-fork_v13_7.png +++ b/doc/ci/pipelines/img/pipeline_fork_v13_7.png diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 24e518b1f69..b873b2ae30f 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -50,10 +50,6 @@ Pipelines can be configured in many different ways: followed by the next stage. - [Directed Acyclic Graph Pipeline (DAG) pipelines](../directed_acyclic_graph/index.md) are based on relationships between jobs and can run more quickly than basic pipelines. -- [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. -- [Parent-Child pipelines](parent_child_pipelines.md) break down complex pipelines - into one parent pipeline that can trigger multiple child sub-pipelines, which all - run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. - [Pipelines for Merge Requests](../pipelines/merge_request_pipelines.md) run for merge requests only (rather than for every commit). - [Pipelines for Merged Results](../pipelines/pipelines_for_merged_results.md) @@ -61,6 +57,44 @@ Pipelines can be configured in many different ways: already been merged into the target branch. - [Merge Trains](../pipelines/merge_trains.md) use pipelines for merged results to queue merges one after the other. +- [Parent-Child pipelines](parent_child_pipelines.md) break down complex pipelines + into one parent pipeline that can trigger multiple child sub-pipelines, which all + run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. +- [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. + +### How parent-child pipelines compare to multi-project pipelines + +Parent-child pipelines and multi-project pipelines can sometimes be used for similar +purposes, but there are some key differences: + +Parent-child pipelines: + +- Run under the same project, ref, and commit SHA as the parent pipeline. +- Affect the overall status of the ref the pipeline runs against. For example, + if a pipeline fails for the main branch, it's common to say that "main is broken". + The status of child pipelines don't directly affect the status of the ref, unless the child + pipeline is triggered with [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are automatically canceled if the pipeline is configured with [`interruptible`](../yaml/index.md#interruptible) + when a new pipeline is created for the same ref. +- Display only the parent pipelines in the pipeline index page. Child pipelines are + visible when visiting their parent pipeline's page. +- Are limited to 2 levels of nesting. A parent pipeline can trigger multiple child pipelines, + and those child pipeline can trigger multiple child pipelines (`A -> B -> C`). + +Multi-project pipelines: + +- Are triggered from another pipeline, but the upstream (triggering) pipeline does + not have much control over the downstream (triggered) pipeline. However, it can + choose the ref of the downstream pipeline, and pass CI/CD variables to it. +- Affect the overall status of the ref of the project it runs in, but does not + affect the status of the triggering pipeline's ref, unless it was triggered with + [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are not automatically canceled in the downstream project when using [`interruptible`](../yaml/index.md#interruptible) + if a new pipeline runs for the same ref in the upstream pipeline. They can be + automatically canceled if a new pipeline is triggered for the same ref on the downstream project. +- Multi-project pipelines are standalone pipelines because they are normal pipelines + that happened to be triggered by an external project. They are all visible on the pipeline index page. +- Are independent, so there are no nesting limits. ## Configure a pipeline @@ -257,14 +291,33 @@ WARNING: Deleting a pipeline expires all pipeline caches, and deletes all related objects, such as builds, logs, artifacts, and triggers. **This action cannot be undone.** -### Pipeline quotas +### Pipeline security on protected branches + +A strict security model is enforced when pipelines are executed on +[protected branches](../../user/project/protected_branches.md). + +The following actions are allowed on protected branches only if the user is +[allowed to merge or push](../../user/project/protected_branches.md) +on that specific branch: + +- Run manual pipelines (using the [Web UI](#run-a-pipeline-manually) or [pipelines API](#pipelines-api)). +- Run scheduled pipelines. +- Run pipelines using triggers. +- Run on-demand DAST scan. +- Trigger manual actions on existing pipelines. +- Retry or cancel existing jobs (using the Web UI or pipelines API). -Each user has a personal pipeline quota that tracks the usage of shared runners in all personal projects. -Each group has a [usage quota](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes) that tracks the usage of shared runners for all projects created within the group. +**Variables** marked as **protected** are accessible only to jobs that +run on protected branches, preventing untrusted users getting unintended access to +sensitive information like deployment credentials and tokens. -When a pipeline is triggered, regardless of who triggered it, the pipeline quota for the project owner's [namespace](../../user/group/index.md#namespaces) is used. In this case, the namespace can be the user or group that owns the project. +**Runners** marked as **protected** can run jobs only on protected +branches, preventing untrusted code from executing on the protected runner and +preserving deployment keys and other credentials from being unintentionally +accessed. In order to ensure that jobs intended to be executed on protected +runners do not use regular runners, they must be tagged accordingly. -#### How pipeline duration is calculated +### How pipeline duration is calculated Total running time for a given pipeline excludes retries and pending (queued) time. @@ -301,44 +354,6 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time (4 - 1) + (7 - 6) => 4 ``` -#### How pipeline quota usage is calculated - -Pipeline quota usage is calculated as the sum of the duration of each individual job. This is slightly different to how pipeline _duration_ is [calculated](#how-pipeline-duration-is-calculated). Pipeline quota usage doesn't consider any overlap of jobs running in parallel. - -For example, a pipeline consists of the following jobs: - -- Job A takes 3 minutes. -- Job B takes 3 minutes. -- Job C takes 2 minutes. - -The pipeline quota usage is the sum of each job's duration. In this example, 8 runner minutes would be used, calculated as: 3 + 3 + 2. - -### Pipeline security on protected branches - -A strict security model is enforced when pipelines are executed on -[protected branches](../../user/project/protected_branches.md). - -The following actions are allowed on protected branches only if the user is -[allowed to merge or push](../../user/project/protected_branches.md) -on that specific branch: - -- Run manual pipelines (using the [Web UI](#run-a-pipeline-manually) or [pipelines API](#pipelines-api)). -- Run scheduled pipelines. -- Run pipelines using triggers. -- Run on-demand DAST scan. -- Trigger manual actions on existing pipelines. -- Retry or cancel existing jobs (using the Web UI or pipelines API). - -**Variables** marked as **protected** are accessible only to jobs that -run on protected branches, preventing untrusted users getting unintended access to -sensitive information like deployment credentials and tokens. - -**Runners** marked as **protected** can run jobs only on protected -branches, preventing untrusted code from executing on the protected runner and -preserving deployment keys and other credentials from being unintentionally -accessed. In order to ensure that jobs intended to be executed on protected -runners do not use regular runners, they must be tagged accordingly. - ## Visualize pipelines Pipelines can be complex structures with many sequential and parallel jobs. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index e47b6dddc5f..1710c57b36b 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -405,3 +405,27 @@ generated. Check the job log for these messages. If you find no helpful messages, retry the failed job after activating [CI/CD debug logging](../variables/index.md#debug-logging). This logging should provide information to help you investigate further. + +### Error message `Missing /usr/bin/gitlab-runner-helper. Uploading artifacts is disabled.` + +There is a [known issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3068) where setting a CI/CD variable named `DEBUG` can cause artifact uploads to fail. + +To work around this, either use a different variable name or set it inline with `script`: + +```yaml +# This job might fail due to issue gitlab-org/gitlab-runner#3068 +failing_test_job: + variables: + DEBUG: true + script: bin/mycommand + artifacts: + paths: + - bin/results + +# This job does not define a CI/CD variable named `DEBUG` and is not affected by the issue +successful_test_job: + script: DEBUG=true bin/mycommand + artifacts: + paths: + - bin/results +``` diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 85e5b62b0c4..4d7ebc09e6f 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -2,214 +2,197 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, index -last_update: 2019-07-03 --- -# Pipelines for merge requests **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/15310) in GitLab 11.6. -In a [basic configuration](pipeline_architectures.md#basic-pipelines), GitLab runs a pipeline each time -changes are pushed to a branch. +# Pipelines for merge requests **(FREE)** -If you want the pipeline to run jobs **only** on commits associated with a merge request, -you can use *pipelines for merge requests*. +You can configure your [pipeline](index.md) to run every time you commit changes to a branch. +This type of pipeline is called a *branch pipeline*. -These pipelines are labeled as `detached` in the UI, and they do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -Otherwise, these pipelines are the same as other pipelines. +Alternatively, you can configure your pipeline to run every time you make changes to the +source branch for a merge request. This type of pipeline is called a *pipeline for merge requests*. -Pipelines for merge requests can run when you: +Branch pipelines: -- Create a new merge request. -- Commit changes to the source branch for the merge request. -- Select the **Run pipeline** button from the **Pipelines** tab in the merge request. +- Run when you push a new commit to a branch. +- Are the default type of pipeline. +- Have access to [some predefined variables](../variables/predefined_variables.md). +- Have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -If you use this feature with [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md), -pipelines for merge requests take precedence over other pipelines. +Pipelines for merge requests: -## Prerequisites +- Run when you: + - Create a new merge request. + - Push a new commit to the source branch for a merge request. + - Select **Run pipeline** from the **Pipelines** tab in a merge request. This option + is only available when pipelines for merge requests are configured for the pipeline. +- Do not run by default. The jobs in the CI/CD configuration file [must be configured](#prerequisites) + to run in pipelines for merge request. +- Have access to [more predefined variables](#available-predefined-variables). +- Do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -To enable pipelines for merge requests: +Both of these types of pipelines can appear on the **Pipelines** tab of a merge request. -- Your repository must be a GitLab repository, not an - [external repository](../ci_cd_for_external_repos/index.md). -- You must have the Developer [role](../../user/permissions.md) - to run a pipeline for merge requests. +## Types of pipelines for merge requests -## Configure pipelines for merge requests +The three types of pipelines for merge requests are: -To configure pipelines for merge requests, you must configure your [CI/CD configuration file](../yaml/index.md). -To do this, you can use [`rules`](#use-rules-to-run-pipelines-for-merge-requests) or [`only/except`](#use-only-or-except-to-run-pipelines-for-merge-requests). +- Pipelines for merge requests, which run on the changes in the merge request's + source branch. These pipelines display a `detached` label to indicate that the + pipeline ran only on the contents of the source branch, ignoring the target branch. +- [Pipelines for merged results](pipelines_for_merged_results.md), which run on + the result of combining the source branch's changes with the target branch. +- [Merge trains](merge_trains.md), which run when merging multiple merge requests + at the same time. The changes from each merge request are combined into the + target branch with the changes in the earlier enqueued merge requests, to ensure + they all work together. -### Use `rules` to run pipelines for merge requests +## Prerequisites -GitLab recommends that you use the `rules` keyword, which is available in -[`workflow:rules` templates](../yaml/workflow.md#workflowrules-templates). +To use pipelines for merge requests: -### Use `only` or `except` to run pipelines for merge requests +- Your project's [CI/CD configuration file](../yaml/index.md) must be configured with + jobs that run in pipelines for merge requests. To do this, you can use: + - [`rules`](#use-rules-to-add-jobs). + - [`only/except`](#use-only-to-add-jobs). +- You must have at least the Developer [role](../../user/permissions.md) in the + source project to run a pipeline for merge requests. +- Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md). -You can use the `only/except` keywords. However, with this method, you must specify `only: - merge_requests` for each job. +## Use `rules` to add jobs -In the following example, the pipeline contains a `test` job that is configured to run on merge requests. -The `build` and `deploy` jobs don't have the `only: - merge_requests` keyword, -so they don't run on merge requests. +You can use the [`rules`](../yaml/index.md#rules) keyword to configure jobs to run in +pipelines for merge requests. For example: ```yaml -build: - stage: build - script: ./build - only: - - main - -test: - stage: test - script: ./test - only: - - merge_requests - -deploy: - stage: deploy - script: ./deploy - only: - - main +job1: + script: + - echo "This job runs in pipelines for merge requests" + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' ``` -#### Exclude specific jobs - -When you use `only: [merge_requests]`, only jobs with -that keyword are run in the context of a merge request. No other jobs run. - -However, you can invert this behavior and have all of your jobs run except -for one or two. For example, you might have a pipeline with jobs `A`, `B`, and `C`, and you want: - -- All pipelines to always run `A` and `B`. -- `C` to run only for merge requests. - -To achieve this outcome, configure your `.gitlab-ci.yml` file as follows: +You can also use the [`workflow: rules`](../yaml/index.md#workflowrules) keyword +to configure the entire pipeline to run in pipelines for merge requests. For example: ```yaml -.only-default: &only-default - only: - - main - - merge_requests - - tags +workflow: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' -A: - <<: *only-default +job1: script: - - ... + - echo "This job runs in pipelines for merge requests" -B: - <<: *only-default +job2: script: - - ... - -C: - script: - - ... - only: - - merge_requests + - echo "This job also runs in pipelines for merge requests" ``` -- `A` and `B` always run, because they get the `only` rule to execute in all cases. -- `C` only runs for merge requests. It doesn't run for any pipeline - except a merge request pipeline. - -In this example, you don't have to add the `only` rule to all of your jobs to make -them always run. You can use this format to set up a Review App, which helps to -save resources. +## Use `only` to add jobs -#### Exclude specific branches - -Branch refs use this format: `refs/heads/my-feature-branch`. -Merge request refs use this format: `refs/merge-requests/:iid/head`. - -Because of this difference, the following configuration does not work as expected: - -```yaml -# Does not exclude a branch named "docs-my-fix"! -test: - only: [merge_requests] - except: [/^docs-/] -``` - -Instead, use the -[`$CI_COMMIT_REF_NAME` predefined environment -variable](../variables/predefined_variables.md) in -combination with -[`only:variables`](../yaml/index.md#onlyvariables--exceptvariables) to -accomplish this behavior: +You can use the [`only`](../yaml/index.md#onlyrefs--exceptrefs) keyword with `merge_requests` +to configure jobs to run in pipelines for merge requests. ```yaml -test: - only: [merge_requests] - except: - variables: - - $CI_COMMIT_REF_NAME =~ /^docs-/ +job1: + script: + - echo "This job runs in pipelines for merge requests" + only: + - merge_requests ``` -## Run pipelines in the parent project for merge requests from a forked project **(PREMIUM)** +## Use with forked projects > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217451) in GitLab 13.3. > - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. -By default, external contributors who work in forks can't create pipelines in the -parent project. When a merge request that comes from a fork triggers a pipeline: +External contributors who work in forks can't create pipelines in the parent project. -- The pipeline is created and runs in the fork (source) project, not the parent (target) project. -- The pipeline uses the fork project's CI/CD configuration and resources. +A merge request from a fork that is submitted to the parent project triggers a +pipeline that: -If a pipeline runs in a fork, a **fork** badge appears for the pipeline in the merge request. +- Is created and runs in the fork (source) project, not the parent (target) project. +- Uses the fork project's CI/CD configuration, resources, and project CI/CD variables. - +Pipelines for forks display with the **fork** badge in the parent project: -Sometimes parent project members want the pipeline to run in the parent -project. They may want to ensure that the post-merge pipeline passes in the parent project. -For example, a fork project could try to use a corrupted runner that doesn't execute -test scripts properly, but reports a passed pipeline. Reviewers in the parent project -could mistakenly trust the merge request because it passed a faked pipeline. + -Parent project members with at least the [Developer role](../../user/permissions.md) -can create pipelines in the parent project for merge requests -from a forked project. In the merge request, go to the **Pipelines** tab and select -**Run pipeline**. +### Run pipelines in the parent project **(PREMIUM)** + +Project members in the parent project can choose to run a pipeline for merge requests +for a merge request submitted from a fork project. This pipeline: + +- Is created and runs in the parent (target) project, not the fork (source) project. +- Uses the CI/CD configuration present in the fork project's branch +- Uses the parent project's CI/CD configuration, resources, and project CI/CD variables. +- Uses the permissions of the parent project member that triggers the pipeline. + +Run pipelines in fork project MRs to ensure that the post-merge pipeline passes in +the parent project. Additionally, if you do not trust the fork project's runner, +running the pipeline in the parent project uses the parent project's trusted runners. WARNING: Fork merge requests can contain malicious code that tries to steal secrets in the -parent project when the pipeline runs, even before merge. As a reviewer, you must carefully +parent project when the pipeline runs, even before merge. As a reviewer, carefully check the changes in the merge request before triggering the pipeline. GitLab shows a warning that you must accept before you can trigger the pipeline. -## Predefined variables available for pipelines for merge requests +Parent project members with at least the [Developer role](../../user/permissions.md) +can create pipelines in the parent project for merge requests from a forked project: -When you use pipelines for merge requests, [additional predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) are available to the CI/CD jobs. -These variables contain information from the associated merge request, so that you can -integrate your job with the [GitLab Merge Request API](../../api/merge_requests.md). +1. In the merge request, go to the **Pipelines** tab. +1. Select **Run pipeline**. You must accept the warning, or the pipeline does not run. -## Troubleshooting +## Available predefined variables -### Two pipelines created when pushing to a merge request +When you use pipelines for merge requests, you can use: + +- All the same [predefined variables](../variables/predefined_variables.md) that are + available in branch pipelines. +- [Additional predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) + available only to jobs in pipelines for merge requests. These variables contain + information from the associated merge request, which can be when calling the + [GitLab Merge Request API endpoint](../../api/merge_requests.md) from a job. + +## Troubleshooting -If you are experiencing duplicated pipelines when using `rules`, take a look at -the [important differences between `rules` and `only`/`except`](../jobs/job_control.md#avoid-duplicate-pipelines), -which helps you get your starting configuration correct. +### Two pipelines when pushing to a branch -If you are seeing two pipelines when using `only/except`, please see the caveats -related to using `only/except` above (or, consider moving to `rules`). +If you get duplicate pipelines in merge requests, your pipeline might be configured +to run for both branches and merge requests at the same time. Adjust your pipeline +configuration to [avoid duplicate pipelines](../jobs/job_control.md#avoid-duplicate-pipelines). In [GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201845), -you can add `workflow:rules` to [switch from branch pipelines to merge request pipelines](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). +you can add `workflow:rules` to [switch from branch pipelines to pipelines for merge requests](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). After a merge request is open on the branch, the pipeline switches to a merge request pipeline. -### Two pipelines created when pushing an invalid CI configuration file +### Two pipelines when pushing an invalid CI/CD configuration file + +If you push an invalid CI/CD configuration to a merge request's branch, two failed +pipelines appear in the pipelines tab. One pipeline is a failed branch pipeline, +the other is a failed pipeline for merge requests. + +When the configuration syntax is fixed, no further failed pipelines should appear. +To find and fix the configuration problem, you can use: + +- The [pipeline editor](../pipeline_editor/index.md). +- The [CI lint tool](../lint.md). + +### The merge request's pipeline is marked as failed but the latest pipeline succeeded + +It's possible to have both branch pipelines and pipelines for merge requests in the +**Pipelines** tab of a single merge request. This might be [by configuration](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines), +or [by accident](#two-pipelines-when-pushing-to-a-branch). -Pushing to a branch with an invalid CI configuration file can trigger -the creation of two types of failed pipelines. One pipeline is a failed merge request -pipeline, and the other is a failed branch pipeline, but both are caused by the same -invalid configuration. +If both types of pipelines are in one merge request, the merge request's pipeline +is not considered successful if: -## Related topics +- The branch pipeline succeeds. +- The pipeline for merge request fails. -- [Pipelines for merged results](pipelines_for_merged_results.md). -- [Merge trains](merge_trains.md). +When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) +feature and both pipelines types are present, the pipelines for merge requests are checked, +not the branch pipelines. diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 593cdb68b3f..d47cbf5f47c 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -2,8 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference -last_update: 2019-07-03 --- # Merge trains **(PREMIUM)** @@ -11,10 +9,6 @@ last_update: 2019-07-03 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9186) in GitLab 12.0. > - [Squash and merge](../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13001) in GitLab 12.6. -INFO: -Get merge trains and more in GitLab Ultimate. -[Try a free 30-day trial now](https://about.gitlab.com/free-trial/index.html?glm_source=docs.gitlab.com&glm_content=p-ci-cd-external-docs). - For more information about why you might want to use merge trains, read [How merge trains keep your master green](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). When [pipelines for merged results](pipelines_for_merged_results.md) are @@ -84,7 +78,7 @@ To enable merge trains: To enable merge trains for your project: 1. If you are on a self-managed GitLab instance, ensure the [feature flag](#merge-trains-feature-flag) is set correctly. -1. [Configure your CI/CD configuration file](merge_request_pipelines.md#configure-pipelines-for-merge-requests) +1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) so that the pipeline or individual jobs run for merge requests. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. diff --git a/doc/ci/pipelines/pipelines_for_merged_results.md b/doc/ci/pipelines/pipelines_for_merged_results.md index 718519faf48..91a49a48882 100644 --- a/doc/ci/pipelines/pipelines_for_merged_results.md +++ b/doc/ci/pipelines/pipelines_for_merged_results.md @@ -2,18 +2,12 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference -last_update: 2019-07-03 --- # Pipelines for merged results **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7380) in GitLab 11.10. -INFO: -Get these pipelines and more in GitLab Ultimate. -[Try a free 30-day trial now](https://about.gitlab.com/free-trial/index.html?glm_source=docs.gitlab.com&glm_content=p-ci-cd-external-docs). - When you submit a merge request, you are requesting to merge changes from a source branch into a target branch. By default, the CI pipeline runs jobs against the source branch. @@ -61,7 +55,7 @@ To enable pipelines for merge results: To enable pipelines for merged results for your project: -1. [Configure your CI/CD configuration file](merge_request_pipelines.md#configure-pipelines-for-merge-requests) +1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) so that the pipeline or individual jobs run for merge requests. 1. Visit your project's **Settings > General** and expand **Merge requests**. 1. Check **Enable merged results pipelines**. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index cf470836e32..85824dfb7c7 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -21,11 +21,7 @@ For public and internal projects, you can change who can see your: - Pipelines - Job output logs - Job artifacts -- [Pipeline security dashboard](../../user/application_security/security_dashboard/index.md#pipeline-security) - -However: - -- Job output logs and artifacts are [never visible for Guest users and non-project members](https://gitlab.com/gitlab-org/gitlab/-/issues/25649). +- [Pipeline security dashboard](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline) To change the visibility of your pipelines and related features: @@ -41,8 +37,10 @@ To change the visibility of your pipelines and related features: When it is cleared: - - For **public** projects, pipelines are visible to everyone. Related features are visible - only to project members (Reporter or higher). + - For **public** projects, job logs, job artifacts, the pipeline security dashboard, + and the **CI/CD** menu items are visible only to project members (Reporter or higher). + Other users, including guest users, can only view the status of pipelines and jobs, and only + when viewing merge requests or commits. - For **internal** projects, pipelines are visible to all logged in users except [external users](../../user/permissions.md#external-users). Related features are visible only to project members (Reporter or higher). - For **private** projects, pipelines and related features are visible to project members (Reporter or higher) only. @@ -161,7 +159,8 @@ in the `.gitlab-ci.yml` file. ## Limit the number of changes fetched during clone -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/28919) in GitLab 12.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/28919) in GitLab 12.0. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77576) `git depth` value in GitLab 14.7. You can limit the number of changes that GitLab CI/CD fetches when it clones a repository. @@ -173,8 +172,8 @@ a repository. The maximum value is `1000`. To disable shallow clone and make GitLab CI/CD fetch all branches and tags each time, keep the value empty or set to `0`. -In GitLab 12.0 and later, newly created projects automatically have a default -`git depth` value of `50`. +In GitLab versions 14.7 and later, newly created projects have a default `git depth` +value of `20`. GitLab versions 14.6 and earlier have a default `git depth` value of `50`. This value can be overridden by the [`GIT_DEPTH` variable](../large_repositories/index.md#shallow-cloning) in the `.gitlab-ci.yml` file. diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index c67282643a4..37005939eb7 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -181,7 +181,7 @@ After you have the route mapping set up, it takes effect in the following locati  -- In the diff for a merge request, comparison, or commit. +- In the diff for a comparison or commit.  diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index b2885262e9d..d826b28acce 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -2,7 +2,6 @@ stage: Verify group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- # Configuring runners **(FREE)** @@ -640,7 +639,7 @@ support this feature. A meter can be enabled to provide the rate of transfer for uploads and downloads. -You can set a maximum time for cache upload and download with the `CACHE_REQUEST_TIMEOUT` setting. +You can set a maximum time for cache upload and download with the `CACHE_REQUEST_TIMEOUT` setting. This setting can be useful when slow cache uploads substantially increase the duration of your job. ```yaml diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index b4e9fe818cf..064ad64b79f 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -17,6 +17,6 @@ No configuration is required. Your jobs can run on: - [Windows runners](build_cloud/windows_build_cloud.md) (beta). - [macOS runners](build_cloud/macos_build_cloud.md) (beta). -The number of minutes you can use on these runners depends on your -[quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota), -which depends on your [subscription plan](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes). +The number of minutes you can use on these runners depends on the +[maximum number of CI/CD minutes](../pipelines/cicd_minutes.md) +in your [subscription plan](https://about.gitlab.com/pricing/). diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index b16957ae83c..f76a767abec 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -28,13 +28,13 @@ If you are using a self-managed instance of GitLab: going to your project's **Settings > CI/CD**, expanding the **Runners** section, and clicking **Show runner installation instructions**. These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html). -- The administrator can also configure a maximum number of shared runner [pipeline minutes for - each group](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota). +- The administrator can also configure a maximum number of shared runner [CI/CD minutes for + each group](../pipelines/cicd_minutes.md#set-the-quota-of-cicd-minutes-for-a-specific-namespace). If you are using GitLab.com: - You can select from a list of [shared runners that GitLab maintains](index.md). -- The shared runners consume the [pipelines minutes](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes) +- The shared runners consume the [CI/CD minutes](../pipelines/cicd_minutes.md) included with your account. ### Enable shared runners diff --git a/doc/ci/runners/saas/windows_saas_runner.md b/doc/ci/runners/saas/windows_saas_runner.md index 87ee542fb14..b08be14dbc3 100644 --- a/doc/ci/runners/saas/windows_saas_runner.md +++ b/doc/ci/runners/saas/windows_saas_runner.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w SaaS runners on Windows are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) and shouldn't be used for production workloads. -During this beta period, the [shared runner pipeline quota](../../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota) +During this beta period, the [shared runner quota for CI/CD minutes](../../pipelines/cicd_minutes.md) applies for groups and projects in the same manner as Linux runners. This may change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md index 4c840125d24..384bfc10779 100644 --- a/doc/ci/test_cases/index.md +++ b/doc/ci/test_cases/index.md @@ -11,10 +11,6 @@ type: reference > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233479) in GitLab 13.6. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/241983) in GitLab 13.7. -INFO: -Create test cases in GitLab Ultimate. -[Try it free for 30 days](https://about.gitlab.com/free-trial/index.html?glm_source=docs.gitlab.com&glm_content=u-test-cases-docs). - Test cases in GitLab can help your teams create testing scenarios in their existing development platform. Now your Implementation and Testing teams can collaborate better, as they no longer have to diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index d3ac1de7c3b..1b648a486ef 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -54,7 +54,7 @@ For example: ```shell curl --request POST \ --form token=<token> \ - --formref=<ref_name> \ + --form ref=<ref_name> \ "https://gitlab.example.com/api/v4/projects/<project_id>/trigger/pipeline" ``` @@ -104,20 +104,18 @@ To trigger a pipeline from another project's webhook, use a webhook URL like the for push and tag events: ```plaintext -https://gitlab.example.com/api/v4/projects/9/ref/main/trigger/pipeline?token=TOKEN +https://gitlab.example.com/api/v4/projects/<project_id>/ref/<ref_name>/trigger/pipeline?token=<token> ``` Replace: - The URL with `https://gitlab.com` or the URL of your instance. -- `<token>` with your trigger token. -- `<ref_name>` with a branch or tag name, like `main`. - `<project_id>` with your project ID, like `123456`. The project ID is displayed at the top of the project's landing page. - -The `ref` in the URL takes precedence over the `ref` in the webhook payload. The -payload `ref` is the branch that fired the trigger in the source repository. -You must URL-encode `ref` if it contains slashes. +- `<ref_name>` with a branch or tag name, like `main`. This value takes precedence over the `ref_name` in the webhook payload. + The payload's `ref` is the branch that fired the trigger in the source repository. + You must URL-encode the `ref_name` if it contains slashes. +- `<token>` with your trigger token. #### Use a webhook payload diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index acc3489143a..7ce58b015ca 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -81,7 +81,7 @@ to execute scripts. Each shell has its own set of reserved variable names. Make sure each variable is defined for the [scope you want to use it in](where_variables_can_be_used.md). By default, pipelines from forked projects can't access CI/CD variables in the parent project. -If you [run a merge request pipeline in the parent project for a merge request from a fork](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project-for-merge-requests-from-a-forked-project), +If you [run a merge request pipeline in the parent project for a merge request from a fork](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project), all variables become available to the pipeline. ### Create a custom CI/CD variable in the `.gitlab-ci.yml` file @@ -394,7 +394,7 @@ runs on a [protected branch](../../user/project/protected_branches.md) or Review all merge requests that introduce changes to the `.gitlab-ci.yml` file before you: -- [Run a pipeline in the parent project for a merge request submitted from a forked project](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project-for-merge-requests-from-a-forked-project). +- [Run a pipeline in the parent project for a merge request submitted from a forked project](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project). - Merge the changes. The following example shows malicious code in a `.gitlab-ci.yml` file: @@ -554,37 +554,45 @@ export GITLAB_USER_ID="42" > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22638) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217834) in GitLab 13.1. -You can pass environment variables from one job to another job in a later stage. +You can pass environment variables from one job to another job in a later stage +through variable inheritance. These variables cannot be used as CI/CD variables to configure a pipeline, but they can be used in job scripts. 1. In the job script, save the variable as a `.env` file. + - The format of the file must be one variable definition per line. + - Each defined line must be of the form `VARIABLE_NAME=ANY VALUE HERE`. + - Values can be wrapped in quotes, but cannot contain newline characters. 1. Save the `.env` file as an [`artifacts:reports:dotenv`](../yaml/artifacts_reports.md#artifactsreportsdotenv) artifact. -1. Set a job in a later stage to receive the artifact by using the [`dependencies`](../yaml/index.md#dependencies) - or the [`needs`](../yaml/index.md#needs) keywords. -1. The later job can then [use the variable in scripts](#use-cicd-variables-in-job-scripts). +1. Jobs in later stages can then [use the variable in scripts](#use-cicd-variables-in-job-scripts). -For example, with the [`dependencies`](../yaml/index.md#dependencies) keyword: +Inherited variables [take precedence](#cicd-variable-precedence) over +certain types of new variable definitions such as job defined variables. ```yaml build: stage: build script: - - echo "BUILD_VERSION=hello" >> build.env + - echo "BUILD_VARIABLE=value_from_build_job" >> build.env artifacts: reports: dotenv: build.env deploy: stage: deploy + variables: + BUILD_VARIABLE: value_from_deploy_job script: - - echo "$BUILD_VERSION" # Output is: 'hello' - dependencies: - - build + - echo "$BUILD_VARIABLE" # Output is: 'value_from_build_job' due to precedence ``` -For example, with the [`needs:artifacts`](../yaml/index.md#needsartifacts) keyword: +The [`dependencies`](../yaml/index.md#dependencies) or +[`needs`](../yaml/index.md#needs) keywords can be used to control +which jobs receive inherited values. + +To have no inherited dotenv environment variables, pass an empty `dependencies` or +`needs` list, or pass [`needs:artifacts`](../yaml/index.md#needsartifacts) as `false` ```yaml build: @@ -595,15 +603,46 @@ build: reports: dotenv: build.env -deploy: +deploy_one: + stage: deploy + script: + - echo "$BUILD_VERSION" # Output is: 'hello' + dependencies: + - build + +deploy_two: + stage: deploy + script: + - echo "$BUILD_VERSION" # Output is empty + dependencies: [] + +deploy_three: stage: deploy script: - echo "$BUILD_VERSION" # Output is: 'hello' needs: - - job: build - artifacts: true + - build + +deploy_four: + stage: deploy + script: + - echo "$BUILD_VERSION" # Output is: 'hello' + needs: + job: build + artifacts: true + +deploy_five: + stage: deploy + script: + - echo "$BUILD_VERSION" # Output is empty + needs: + job: build + artifacts: false ``` +[Multi-project pipelines](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance) +can also inherit variables from their upstream pipelines. + ## CI/CD variable precedence You can use CI/CD variables with the same name in different places, but the values diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index c06ca6878c4..60888cd9b97 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -62,6 +62,8 @@ There are also a number of [variables you can use to configure runner behavior]( | `CI_JOB_ID` | 9.0 | all | The internal ID of the job, unique across all jobs in the GitLab instance. | | `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. | | `CI_JOB_JWT` | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | +| `CI_JOB_JWT_V1` | 14.6 | all | The same value as `CI_JOB_JWT`. | +| `CI_JOB_JWT_V2` | 14.6 | all | [**alpha:**](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga) A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). | | `CI_JOB_MANUAL` | 8.12 | all | `true` if a job was started manually. | | `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job. | | `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the job's stage. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index e32f0391d2e..4f304a10256 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -22,20 +22,25 @@ There are two places defined variables can be used. On the: ### `.gitlab-ci.yml` file -| Definition | Can be expanded? | Expansion place | Description | -|:-------------------------------------------|:-----------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `environment:url` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | -| `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | -| `resource_group` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/>- `CI_ENVIRONMENT_URL`<br/>- [Persisted variables](#persisted-variables) | -| `include` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>Predefined project variables are supported: `GITLAB_FEATURES`, `CI_DEFAULT_BRANCH`, and all variables that start with `CI_PROJECT_` (for example `CI_PROJECT_NAME`). | -| `variables` | yes | GitLab/Runner | The variable expansion is first made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab, and then any unrecognized or unavailable variables are expanded by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | -| `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | -| `services:[]` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | -| `services:[]:name` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | -| `cache:key` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | -| `artifacts:name` | yes | Runner | The variable expansion is made by GitLab Runner's shell environment | -| `script`, `before_script`, `after_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) | -| `only:variables:[]`, `except:variables:[]`, `rules:if` | no | n/a | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| Definition | Can be expanded? | Expansion place | Description | +|:----------------------|:-----------------|:-----------------------|:------------| +| `after_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). | +| `artifacts:name` | yes | Runner | The variable expansion is made by GitLab Runner's shell environment. | +| `before_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) | +| `cache:key` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| `environment:url` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | +| `except:variables:[]` | no | n/a | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| `include` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>Predefined project variables are supported: `GITLAB_FEATURES`, `CI_DEFAULT_BRANCH`, and all variables that start with `CI_PROJECT_` (for example `CI_PROJECT_NAME`). | +| `only:variables:[]` | no | n/a | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| `resource_group` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/>- `CI_ENVIRONMENT_URL`<br/>- [Persisted variables](#persisted-variables). | +| `rules:if` | no | n/a | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| `script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). | +| `services:[]:name` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| `services:[]` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| `tags` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35742) in GitLab 14.1. | +| `variables` | yes | GitLab/Runner | The variable expansion is first made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab, and then any unrecognized or unavailable variables are expanded by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | ### `config.toml` file diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index cd38cf58c71..25fb861cfb7 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -52,7 +52,7 @@ GitLab can display the results of one or more reports in: - The merge request [security widget](../../user/application_security/api_fuzzing/index.md#view-details-of-an-api-fuzzing-vulnerability). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). -- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#pipeline-security). +- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline). - The [security dashboard](../../user/application_security/api_fuzzing/index.md#security-dashboard). ## `artifacts:reports:browser_performance` **(PREMIUM)** @@ -117,7 +117,7 @@ The collected Container Scanning report uploads to GitLab as an artifact. GitLab can display the results of one or more reports in: - The merge request [container scanning widget](../../user/application_security/container_scanning/index.md). -- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#pipeline-security). +- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline). - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). @@ -131,7 +131,7 @@ The collected coverage fuzzing report uploads to GitLab as an artifact. GitLab can display the results of one or more reports in: - The merge request [coverage fuzzing widget](../../user/application_security/coverage_fuzzing/index.md#interacting-with-the-vulnerabilities). -- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#pipeline-security). +- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). - The [security dashboard](../../user/application_security/security_dashboard/index.md). @@ -143,7 +143,7 @@ report uploads to GitLab as an artifact. GitLab can display the results of one or more reports in: - The merge request [security widget](../../user/application_security/dast/index.md#view-details-of-a-vulnerability-detected-by-dast). -- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#pipeline-security). +- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). - The [security dashboard](../../user/application_security/security_dashboard/index.md). @@ -155,7 +155,7 @@ The collected Dependency Scanning report uploads to GitLab as an artifact. GitLab can display the results of one or more reports in: - The merge request [dependency scanning widget](../../user/application_security/dependency_scanning/index.md#overview). -- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#pipeline-security). +- The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline). - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). - The [dependency list](../../user/application_security/dependency_list/). diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index ed05ef08d02..4530e2675c4 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -24,12 +24,13 @@ A GitLab CI/CD pipeline configuration includes: - [Global keywords](#global-keywords) that configure pipeline behavior: - | Keyword | Description | - |-------------------------|:------------| - | [`default`](#default) | Custom default values for job keywords. | - | [`include`](#include) | Import configuration from other YAML files. | - | [`stages`](#stages) | The names and order of the pipeline stages. | - | [`workflow`](#workflow) | Control what types of pipeline run. | + | Keyword | Description | + |---------------------------|:------------| + | [`default`](#default) | Custom default values for job keywords. | + | [`include`](#include) | Import configuration from other YAML files. | + | [`stages`](#stages) | The names and order of the pipeline stages. | + | [`variables`](#variables) | Define CI/CD variables for all job in the pipeline. | + | [`workflow`](#workflow) | Control what types of pipeline run. | - [Jobs](../jobs/index.md) configured with [job keywords](#job-keywords): @@ -414,6 +415,7 @@ and the pipeline is for either: - You can use the [`workflow:rules` templates](workflow.md#workflowrules-templates) to import a preconfigured `workflow: rules` entry. - [Common `if` clauses for `workflow:rules`](workflow.md#common-if-clauses-for-workflowrules). +- [Use `rules` to run pipelines for merge requests](../pipelines/merge_request_pipelines.md#use-rules-to-add-jobs). #### `workflow:rules:variables` @@ -1429,13 +1431,13 @@ test osx: stage: test script: make test:osx dependencies: - - build:osx + - build osx test linux: stage: test script: make test:linux dependencies: - - build:linux + - build linux deploy: stage: deploy @@ -2506,8 +2508,7 @@ docker build: - [`only: changes` and `except: changes` examples](../jobs/job_control.md#onlychanges--exceptchanges-examples). - If you use `changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds), you should [also use `only:merge_requests`](../jobs/job_control.md#use-onlychanges-with-pipelines-for-merge-requests). -- Use `changes` with [new branches or tags *without* pipelines for merge requests](../jobs/job_control.md#use-onlychanges-without-pipelines-for-merge-requests). -- Use `changes` with [scheduled pipelines](../jobs/job_control.md#use-onlychanges-with-scheduled-pipelines). +- [Jobs or pipelines can run unexpectedly when using `only: changes`](../jobs/job_control.md#jobs-or-pipelines-run-unexpectedly-when-using-changes). #### `only:kubernetes` / `except:kubernetes` @@ -3066,8 +3067,10 @@ job: - If a rule matches and has no `when` defined, the rule uses the `when` defined for the job, which defaults to `on_success` if not defined. -- You can define `when` once per rule, or once at the job-level, which applies to - all rules. You can't mix `when` at the job-level with `when` in rules. +- In GitLab 14.5 and earlier, you can define `when` once per rule, or once at the job-level, + which applies to all rules. You can't mix `when` at the job-level with `when` in rules. +- In GitLab 14.6 and later, you can [mix `when` at the job-level with `when` in rules](https://gitlab.com/gitlab-org/gitlab/-/issues/219437). + `when` configuration in `rules` takes precedence over `when` at the job-level. - Unlike variables in [`script`](../variables/index.md#use-cicd-variables-in-job-scripts) sections, variables in rules expressions are always formatted as `$VARIABLE`. - You can use `rules:if` with `include` to [conditionally include other configuration files](includes.md#use-rules-with-include). @@ -3076,6 +3079,7 @@ job: - [Common `if` expressions for `rules`](../jobs/job_control.md#common-if-clauses-for-rules). - [Avoid duplicate pipelines](../jobs/job_control.md#avoid-duplicate-pipelines). +- [Use `rules` to run pipelines for merge requests](../pipelines/merge_request_pipelines.md#use-rules-to-add-jobs). #### `rules:changes` @@ -3120,6 +3124,10 @@ docker build: - You can use `when: never` to implement a rule similar to [`except:changes`](#onlychanges--exceptchanges). - `changes` resolves to `true` if any of the matching files are changed (an `OR` operation). +**Related topics**: + +- [Jobs or pipelines can run unexpectedly when using `rules: changes`](../jobs/job_control.md#jobs-or-pipelines-run-unexpectedly-when-using-changes). + #### `rules:exists` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4. @@ -3834,13 +3842,13 @@ The following keywords are deprecated. ### Globally-defined `types` WARNING: -`types` is deprecated, and could be removed in a future release. +`types` is deprecated, and is [scheduled to be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). Use [`stages`](#stages) instead. ### Job-defined `type` WARNING: -`type` is deprecated, and could be removed in one of the future releases. +`type` is deprecated, and is [scheduled to be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). Use [`stage`](#stage) instead. ### Globally-defined `image`, `services`, `cache`, `before_script`, `after_script` diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index fdec0947df5..ca1e79c2395 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -164,10 +164,14 @@ Second command line. When you omit the `>` or `|` block scalar indicators, GitLab concatenates non-empty lines to form the command. Make sure the lines can run when concatenated. -[These documents](https://en.wikipedia.org/wiki/Here_document) work with the +<!-- vale gitlab.MeaningfulLinkWords = NO --> + +[Shell here documents](https://en.wikipedia.org/wiki/Here_document) work with the `|` and `>` operators as well. The example below transliterates lower case letters to upper case: +<!-- vale gitlab.MeaningfulLinkWords = YES --> + ```yaml job: script: |
