diff options
Diffstat (limited to 'doc/ci')
82 files changed, 1666 insertions, 1784 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 9b555c0ee68..b1bcb578daf 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -37,6 +37,9 @@ the development cycle, ensuring that all the code deployed to production complies with the code standards you established for your app. +GitLab can also automatically detect, build, test, deploy, and +monitor your applications by using [Auto DevOps](../topics/autodevops/index.md). + For a complete overview of these methodologies and GitLab CI/CD, read the [Introduction to CI/CD with GitLab](introduction/index.md). diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 2a8b050b224..deadc4458a2 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -22,7 +22,7 @@ cannot be used to authenticate with GitHub as an external CI/CD repository. ## Connect with Personal Access Token Personal access tokens can only be used to connect GitHub.com -repositories to GitLab, and the GitHub user must have the [owner role](https://docs.github.com/en/free-pro-team@latest/github/getting-started-with-github/access-permissions-on-github). +repositories to GitLab, and the GitHub user must have the [owner role](https://docs.github.com/en/github/getting-started-with-github/access-permissions-on-github). To perform a one-off authorization with GitHub to grant GitLab access your repositories: diff --git a/doc/ci/cloud_deployment/ecs/img/container-name.png b/doc/ci/cloud_deployment/ecs/img/container-name.png Binary files differnew file mode 100644 index 00000000000..c2b4f23454b --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/container-name.png diff --git a/doc/ci/cloud_deployment/ecs/img/container-port-mapping.png b/doc/ci/cloud_deployment/ecs/img/container-port-mapping.png Binary files differnew file mode 100644 index 00000000000..ca390bf68a0 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/container-port-mapping.png diff --git a/doc/ci/cloud_deployment/ecs/img/ecs-launch-status.png b/doc/ci/cloud_deployment/ecs/img/ecs-launch-status.png Binary files differnew file mode 100644 index 00000000000..389c05d9026 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/ecs-launch-status.png diff --git a/doc/ci/cloud_deployment/ecs/img/ecs-policy.png b/doc/ci/cloud_deployment/ecs/img/ecs-policy.png Binary files differnew file mode 100644 index 00000000000..088a45bb065 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/ecs-policy.png diff --git a/doc/ci/cloud_deployment/ecs/img/ecs-task-definitions.png b/doc/ci/cloud_deployment/ecs/img/ecs-task-definitions.png Binary files differnew file mode 100644 index 00000000000..e91a8656ae0 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/ecs-task-definitions.png diff --git a/doc/ci/cloud_deployment/ecs/img/initial-pipeline.png b/doc/ci/cloud_deployment/ecs/img/initial-pipeline.png Binary files differnew file mode 100644 index 00000000000..7072f550586 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/initial-pipeline.png diff --git a/doc/ci/cloud_deployment/ecs/img/rails-template.png b/doc/ci/cloud_deployment/ecs/img/rails-template.png Binary files differnew file mode 100644 index 00000000000..02c67f8dd21 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/rails-template.png diff --git a/doc/ci/cloud_deployment/ecs/img/registry.png b/doc/ci/cloud_deployment/ecs/img/registry.png Binary files differnew file mode 100644 index 00000000000..694d83fd0ce --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/registry.png diff --git a/doc/ci/cloud_deployment/ecs/img/service-parameter.png b/doc/ci/cloud_deployment/ecs/img/service-parameter.png Binary files differnew file mode 100644 index 00000000000..7252601571b --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/service-parameter.png diff --git a/doc/ci/cloud_deployment/ecs/img/service-running.png b/doc/ci/cloud_deployment/ecs/img/service-running.png Binary files differnew file mode 100644 index 00000000000..2b78960d5f4 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/service-running.png diff --git a/doc/ci/cloud_deployment/ecs/img/view-running-app-2.png b/doc/ci/cloud_deployment/ecs/img/view-running-app-2.png Binary files differnew file mode 100644 index 00000000000..e64a378e11c --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/view-running-app-2.png diff --git a/doc/ci/cloud_deployment/ecs/img/view-running-app.png b/doc/ci/cloud_deployment/ecs/img/view-running-app.png Binary files differnew file mode 100644 index 00000000000..360dd285182 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/img/view-running-app.png diff --git a/doc/ci/cloud_deployment/ecs/quick_start_guide.md b/doc/ci/cloud_deployment/ecs/quick_start_guide.md new file mode 100644 index 00000000000..f75680ccd8c --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/quick_start_guide.md @@ -0,0 +1,250 @@ +--- +stage: Release +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Getting started with Continuous Deployment to AWS Elastic Container Service **(FREE)** + +This step-by-step guide helps you use [Continuous Deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs) +that deploys a project hosted on GitLab.com to [Elastic Container Service](https://aws.amazon.com/ecs/) +(ECS) on AWS. + +In this guide, you begin by creating an ECS cluster manually using the AWS console. You create and +deploy a simple application that you create from a GitLab template. + +These instructions work for both SaaS and self-managed GitLab instances. +Ensure your own [runners are configured](../../runners/README.md). + +## Prerequisites + +- An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/). + Sign in with an existing AWS account or create a new one. +- In this guide, you create an infrastructure in [`us-east-2` region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). + You can use any region, but do not change it after you begin. + +## Create an infrastructure and initial deployment on AWS + +For deploying an application from GitLab, you must first create an infrastructure and initial +deployment on AWS. +This includes an [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) +and related components, such as +[ECS task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html), +[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html), +and containerized application image. + +For the first step here, you create a demo application from a project template. + +### Create a new project from a template + +Use a GitLab project template to get started. As the name suggests, these projects provide a +bare-bones application built on some well-known frameworks. + +1. In GitLab, click the plus icon (**{plus-square}**) at the top of the navigation bar, and select + **New project**. + +1. Click the **Create from template** button, where you can choose from a Ruby on Rails, Spring, or + NodeJS Express project. For this guide, use the Ruby on Rails template. + +  + +1. Give your project a name. In this example, it's named `ecs-demo`. Make it public so that you can + take advantage of the features available in the + [GitLab Ultimate plan](https://about.gitlab.com/pricing/). + +1. Click **Create project**. + +Now that you created a demo project, you must containerize the application and push it to the +container registry. + +### Push a containerized application image to GitLab Container Registry + +[ECS](https://aws.amazon.com/ecs/) is a container orchestration service, meaning that you must +provide a containerized application image during the infrastructure build. To do so, you can use +GitLab [Auto Build](../../../topics/autodevops/stages.md#auto-build) +and [Container Registry](../../../user/packages/container_registry/index.md). + +1. Go to **ecs-demo** project on GitLab. +1. Click **Setup up CI/CD**. It brings you to a [`.gitlab-ci.yml`](../../README.md#getting-started) + creation form. +1. Copy and paste the following content into the empty `.gitlab-ci.yml`. This defines + [a pipeline for continuous deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs). + + ```yaml + include: + - template: AWS/Deploy-ECS.gitlab-ci.yml + ``` + +1. Click **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` + job containerizes the application and pushes the image to [GitLab Container Registry](../../../user/packages/container_registry/index.md). + +  + +1. Visit **Packages & Registries > Container Registry**. Make sure the application image has been + pushed. + +  + +Now you have a containerized application image that can be pulled from AWS. Next, you define the +spec of how this application image is used in AWS. + +Note that the `production_ecs` job fails because ECS Cluster is not connected yet. You'll fix this +later. + +### Create an ECS task definition + +[ECS Task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) +is a specification about how the application image is started by an [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). + +1. Go to **ECS > Task Definitions** on [AWS console](https://aws.amazon.com/). +1. Click **Create new Task Definition**. + +  + +1. Choose **EC2** as the launch type. Click **Next Step**. +1. Set `ecs_demo` to **Task Definition Name**. +1. Set `512` to **Task Size > Task memory** and **Task CPU**. +1. Click **Container Definitions > Add container**. This opens a container registration form. +1. Set `web` to **Container name**. +1. Set `registry.gitlab.com/<your-namespace>/ecs-demo/master:latest` to **Image**. + Alternatively, you can copy and paste the image path from the [GitLab Container Registry page](#push-a-containerized-application-image-to-gitlab-container-registry). + +  + +1. Add a port mapping. Set `80` to **Host Port** and `5000` to **Container port**. + +  + +1. Click **Create**. + +Now you have the initial task definition. Next, you create an actual infrastructure to run the +application image. + +### Create an ECS cluster + +An [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) +is a virtual group of [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). +It's also associated with EC2 or Fargate as the computation resource. + +1. Go to **ECS > Clusters** on [AWS console](https://aws.amazon.com/). +1. Click **Create Cluster**. +1. Select **EC2 Linux + Networking** as the cluster template. Click **Next Step**. +1. Set `ecs-demo` to **Cluster Name**. +1. Choose the default [VPC](https://aws.amazon.com/vpc/?vpc-blogs.sort-by=item.additionalFields.createdDate&vpc-blogs.sort-order=desc) + in **Networking**. If there are no existing VPCs, you can leave it as-is to create a new one. +1. Set all available subnets of the VPC to **Subnets**. +1. Click **Create**. +1. Make sure that the ECS cluster has been successfully created. + +  + +Now you can register an ECS service to the ECS cluster in the next step. + +Note the following: + +- Optionally, you can set a SSH key pair in the creation form. This allows you to SSH to the EC2 + instance for debugging. +- If you don't choose an existing VPC, it creates a new VPC by default. This could cause an error if + it reaches the maximum allowed number of internet gateways on your account. +- The cluster requires an EC2 instance, meaning it costs you [according to the instance-type](https://aws.amazon.com/ec2/pricing/on-demand/). + +### Create an ECS Service + +[ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) +is a daemon to create an application container based on the [ECS task definition](#create-an-ecs-task-definition). + +1. Go to **ECS > Clusters > ecs-demo > Services** on the [AWS console](https://aws.amazon.com/) +1. Click **Deploy**. This opens a service creation form. +1. Select `EC2` in **Launch Type**. +1. Set `ecs_demo` to **Task definition**. This corresponds to [the task definition you created above](#create-an-ecs-task-definition). +1. Set `ecs_demo` to **Service name**. +1. Set `1` to **Desired tasks**. + +  + +1. Click **Deploy**. +1. Make sure that the created service is active. + +  + +Note that AWS's console UI changes from time to time. If you can't find a relevant component in the +instructions, select the closest one. + +### View the demo application + +Now, the demo application is accessible from the internet. + +1. Go to **EC2 > Instances** on the [AWS console](https://aws.amazon.com/) +1. Search by `ECS Instance` to find the corresponding EC2 instance that [the ECS cluster created](#create-an-ecs-cluster). +1. Click the ID of the EC2 instance. This brings you to the instance detail page. +1. Copy **Public IPv4 address** and paste it in the browser. Now you can see the demo application + running. + +  + +In this guide, HTTPS/SSL is **NOT** configured. You can access to the application through HTTP only +(for example, `http://<ec2-ipv4-address>`). + +## Setup Continuous Deployment from GitLab + +Now that you have an application running on ECS, you can set up continuous deployment from GitLab. + +### Create a new IAM user as a deployer + +For GitLab to access the ECS cluster, service, and task definition that you created above, You must +create a deployer user on AWS: + +1. Go to **IAM > Users** on [AWS console](https://aws.amazon.com/). +1. Click **Add user**. +1. Set `ecs_demo` to **User name**. +1. Enable **Programmatic access** checkbox. Click **Next: Permissions**. +1. Select `Attach existing policies directly` in **Set permissions**. +1. Select `AmazonECS_FullAccess` from the policy list. Click **Next: Tags** and **Next: Review**. + +  + +1. Click **Create user**. +1. Take note of the **Access key ID** and **Secret access key** of the created user. + +NOTE: +Do not share the secret access key in a public place. You must save it in a secure place. + +### Setup credentials in GitLab to let pipeline jobs access to ECS + +You can register the access information in [GitLab Environment Variables](../../variables/README.md#create-a-custom-variable-in-the-ui). +These variables are injected into the pipeline jobs and can access the ECS API. + +1. Go to **ecs-demo** project on GitLab. +1. Go to **Settings > CI/CD > Variables**. +1. Click **Add Variable** and set the following key-value pairs. + + |Key|Value|Note| + |---|---|---| + |`AWS_ACCESS_KEY_ID`|`<Access key ID of the deployer>`| For authenticating `aws` CLI. | + |`AWS_SECRET_ACCESS_KEY`|`<Secret access key of the deployer>`| For authenticating `aws` CLI. | + |`AWS_DEFAULT_REGION`|`us-east-2`| For authenticating `aws` CLI. | + |`CI_AWS_ECS_CLUSTER`|`ecs-demo`| The ECS cluster is accessed by `production_ecs` job. | + |`CI_AWS_ECS_SERVICE`|`ecs_demo`| The ECS service of the cluster is updated by `production_ecs` job. | + |`CI_AWS_ECS_TASK_DEFINITION`|`ecs_demo`| The ECS task definition is updated by `production_ecs` job. | + +### Make a change to the demo application + +Change a file in the project and see if it's reflected in the demo application on ECS: + +1. Go to **ecs-demo** project on GitLab. +1. Open the file at **app > views > welcome > index.html.erb**. +1. Click **Edit**. +1. Change the text to `You're on ECS!`. +1. Click **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. +1. [Access the running application on the ECS cluster](#view-the-demo-application). You should see + this: + +  + +Congratulations! You successfully set up continuous deployment to ECS. + +## Further reading + +- If you're interested in more of the continuous deployments to clouds, see [cloud deployments](../index.md). +- If you want to quickly set up DevSecOps in your project, see [Auto DevOps](../../../topics/autodevops/index.md). +- If you want to quickly set up the production-grade environment, see [the 5 Minute Production App](https://gitlab.com/gitlab-org/5-minute-production-app/deploy-template/-/blob/master/README.md). diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index ccacb3c61d3..1e53414cd1e 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -20,6 +20,11 @@ CI/CD pipeline, you can interact with your chosen cloud provider more easily. GitLab provides Docker images that you can use to [run AWS commands from GitLab CI/CD](#run-aws-commands-from-gitlab-cicd), and a template to make it easier to [deploy to AWS](#deploy-your-application-to-the-aws-elastic-container-service-ecs). +### Quick start + +If you're using GitLab.com, see the [quick start guide](ecs/quick_start_guide.md) +for setting up Continuous Deployment to [AWS Elastic Container Service](https://aws.amazon.com/ecs/) (ECS). + ### Run AWS commands from GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31167) in GitLab 12.6. @@ -37,7 +42,7 @@ Some credentials are required to be able to run `aws` commands: NOTE: A new **Access key ID** and **Secret access key** are generated. Please take a note of them right away. -1. In your GitLab project, go to **Settings > CI / CD**. Set the following as +1. In your GitLab project, go to **Settings > CI/CD**. Set the following as [CI/CD variables](../variables/README.md) (see table below): @@ -319,4 +324,4 @@ For a video walkthrough of this configuration process, see [Auto Deploy to EC2]( ## Deploy to Google Cloud -- [Deploying with GitLab on Google Cloud](https://about.gitlab.com/solutions/google-cloud-platform/) +- [Deploying with GitLab on Google Cloud](https://about.gitlab.com/partners/technology-partners/google-cloud-platform/) diff --git a/doc/ci/docker/index.md b/doc/ci/docker/index.md index 18a9d63b694..0897bb183e5 100644 --- a/doc/ci/docker/index.md +++ b/doc/ci/docker/index.md @@ -8,11 +8,18 @@ type: index # Docker integration -GitLab CI/CD can be combined with [Docker](https://www.docker.com) to enable -integration between the two. +There are two primary ways to incorporate [Docker](https://www.docker.com) in your CI/CD workflow. -The following documentation is available for using GitLab CI/CD with Docker: +- **[Run your CI/CD jobs](using_docker_images.md) in Docker containers.** -- [Building Docker images with GitLab CI/CD](using_docker_build.md). -- [Using Docker images](using_docker_images.md). -- [Building images with kaniko and GitLab CI/CD](using_kaniko.md). + You can create CI/CD jobs to do things like test, build, or publish + an application. These jobs can run in Docker containers. + + For example, you can tell GitLab CI/CD to use a Node image that's hosted on Docker Hub + or in the GitLab Container Registry. Your job then runs in a container that's based on the image. + The container has all the Node dependencies you need to build your app. + +- **Use [Docker](using_docker_build.md) or [kaniko](using_kaniko.md) to build Docker images.** + + You can create CI/CD jobs to build Docker images and publish + them to a container registry. diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 46ced9b4d6d..2091a80bdf2 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -5,34 +5,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Building Docker images with GitLab CI/CD +# Use Docker to build Docker images -You can use GitLab CI/CD with Docker Engine to build and test Docker-based projects. - -For example, you might want to: - -1. Create an application image. -1. Run tests against the created image. -1. Push image to a remote registry. -1. Deploy to a server from the pushed image. - -Or, if your application already has a `Dockerfile`, you can -use it to create and test an image: - -```shell -docker build -t my-image dockerfiles/ -docker run my-image /script/to/run/tests -docker tag my-image my-registry:5000/my-image -docker push my-registry:5000/my-image -``` +You can use GitLab CI/CD with Docker to create Docker images. +For example, you can create a Docker image of your application, +test it, and publish it to a container registry. To run Docker commands in your CI/CD jobs, you must configure -GitLab Runner to enable `docker` support. +GitLab Runner to support `docker` commands. ## Enable Docker commands in your CI/CD jobs -There are three ways to enable the use of `docker build` and `docker run` -during jobs, each with their own tradeoffs. You can use: +To enable Docker commands for your CI/CD jobs, you can use: - [The shell executor](#use-the-shell-executor) - [The Docker executor with the Docker image (Docker-in-Docker)](#use-the-docker-executor-with-the-docker-image-docker-in-docker) @@ -47,12 +31,9 @@ to learn more about how these runners are configured. ### Use the shell executor -One way to configure GitLab Runner for `docker` support is to use the -`shell` executor. - -After you register a runner and select the `shell` executor, -your job scripts are executed as the `gitlab-runner` user. -This user needs permission to run Docker commands. +You can include Docker commands in your CI/CD jobs if your runner is configured to +use the `shell` executor. The `gitlab-runner` user runs the Docker commands, but +needs permission to run them. 1. [Install](https://gitlab.com/gitlab-org/gitlab-runner/#installation) GitLab Runner. 1. [Register](https://docs.gitlab.com/runner/register/) a runner. @@ -100,9 +81,11 @@ Learn more about the [security of the `docker` group](https://blog.zopyx.com/on- ### Use the Docker executor with the Docker image (Docker-in-Docker) -Another way to configure GitLab Runner for `docker` support is to -register a runner with the Docker executor and use the [Docker image](https://hub.docker.com/_/docker/) -to run your job scripts. This configuration is referred to as "Docker-in-Docker." +You can use "Docker-in-Docker" to run commands in your CI/CD jobs: + +- Register a runner that uses the Docker executor. +- Use the [Docker image](https://hub.docker.com/_/docker/) provided by Docker to + run the jobs that need Docker commands. The Docker image has all of the `docker` tools installed and can run the job script in context of the image in privileged mode. @@ -111,14 +94,18 @@ The `docker-compose` command is not available in this configuration by default. To use `docker-compose` in your job scripts, follow the `docker-compose` [installation instructions](https://docs.docker.com/compose/install/). +An example project that uses this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. + WARNING: When you enable `--docker-privileged`, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege -escalation which can lead to container breakout. For more information, check +escalation. Doing this can lead to container breakout. For more information, check out the official Docker documentation on [runtime privilege and Linux capabilities](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). -Docker-in-Docker works well, and is the recommended configuration, but it is +#### Limitations of Docker-in-Docker + +Docker-in-Docker is the recommended configuration, but it is not without its own challenges: - When using Docker-in-Docker, each job is in a clean environment without the past @@ -144,8 +131,6 @@ not without its own challenges: - docker run -v "$MOUNT_POINT:/mnt" my-docker-image ``` -An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. - In the examples below, we are using Docker images tags to specify a specific version, such as `docker:19.03.12`. If tags like `docker:stable` are used, you have no control over what version is used. This can lead to @@ -373,9 +358,8 @@ build: ### Use Docker socket binding -Another way to configure GitLab Runner for `docker` support is to -bind-mount `/var/run/docker.sock` into the -container so that Docker is available in the context of the image. +To use Docker commands in your CI/CD jobs, you can bind-mount `/var/run/docker.sock` into the +container. Docker is then available in the context of the image. NOTE: If you bind the Docker socket and you are @@ -478,13 +462,10 @@ services: > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27173) in GitLab Runner 13.6. -If you are an administrator of GitLab Runner and you have the `dind` -service defined for the [Docker -executor](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdockerservices-section), -or the [Kubernetes -executor](https://docs.gitlab.com/runner/executors/kubernetes.html#using-services) -you can specify the `command` to configure the registry mirror for the -Docker daemon. +If you are a GitLab Runner administrator, you can specify the `command` to configure the registry mirror +for the Docker daemon. The `dind` service must be defined for the +[Docker](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdockerservices-section) +or [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html#using-services). Docker: @@ -516,11 +497,10 @@ Kubernetes: ##### Docker executor inside GitLab Runner configuration -If you are an administrator of GitLab Runner and you want to use -the mirror for every `dind` service, update the +If you are a GitLab Runner administrator, you can use +the mirror for every `dind` service. Update the [configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) -to specify a [volume -mount](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section). +to specify a [volume mount](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section). For example, if you have a `/opt/docker/daemon.json` file with the following content: @@ -552,11 +532,10 @@ picked up by the `dind` service. > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3223) in GitLab Runner 13.6. -If you are an administrator of GitLab Runner and you want to use -the mirror for every `dind` service, update the +If you are a GitLab Runner administrator, you can use +the mirror for every `dind` service. Update the [configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) -to specify a [ConfigMap volume -mount](https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes). +to specify a [ConfigMap volume mount](https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes). For example, if you have a `/tmp/daemon.json` file with the following content: @@ -602,7 +581,7 @@ The configuration is picked up by the `dind` service. When you use Docker-in-Docker, the [normal authentication methods](using_docker_images.html#define-an-image-from-a-private-container-registry) -won't work because a fresh Docker daemon is started with the service. +don't work because a fresh Docker daemon is started with the service. ### Option 1: Run `docker login` @@ -634,14 +613,14 @@ empty or remove it. If you are an administrator for GitLab Runner, you can mount a file with the authentication configuration to `~/.docker/config.json`. -Then every job that the runner picks up will be authenticated already. If you +Then every job that the runner picks up is authenticated already. If you are using the official `docker:19.03.13` image, the home directory is under `/root`. If you mount the configuration file, any `docker` command that modifies the `~/.docker/config.json` (for example, `docker login`) fails, because the file is mounted as read-only. Do not change it from -read-only, because other problems will occur. +read-only, because problems occur. Here is an example of `/opt/.docker/config.json` that follows the [`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data) @@ -743,8 +722,8 @@ build: When using Docker-in-Docker, Docker downloads all layers of your image every time you create a build. Recent versions of Docker (Docker 1.13 and above) can -use a pre-existing image as a cache during the `docker build` step, considerably -speeding up the build process. +use a pre-existing image as a cache during the `docker build` step. This considerably +speeds up the build process. ### How Docker caching works @@ -754,8 +733,8 @@ any changes. Change in one layer causes all subsequent layers to be recreated. You can specify a tagged image to be used as a cache source for the `docker build` command by using the `--cache-from` argument. Multiple images can be specified -as a cache source by using multiple `--cache-from` arguments. Keep in mind that -any image that's used with the `--cache-from` argument must first be pulled +as a cache source by using multiple `--cache-from` arguments. Any image that's used +with the `--cache-from` argument must first be pulled (using `docker pull`) before it can be used as a cache source. ### Using Docker caching @@ -872,4 +851,4 @@ If: - This is the first time setting it up, carefully read [using Docker in Docker workflow](#use-the-docker-executor-with-the-docker-image-docker-in-docker). - You are upgrading from v18.09 or earlier, read our - [upgrade guide](https://about.gitlab.com/releases/2019/07/31/docker-in-docker-with-docker-19-dot-03/). + [upgrade guide](https://about.gitlab.com/blog/2019/07/31/docker-in-docker-with-docker-19-dot-03/). diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 67450d442a9..e8028a862c4 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -5,29 +5,25 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Using Docker images +# Run your CI/CD jobs in Docker containers -GitLab CI/CD in conjunction with [GitLab Runner](../runners/README.md) can use -[Docker Engine](https://www.docker.com/) to test and build any application. +You can run your CI/CD jobs in separate, isolated Docker containers. -Docker is an open-source project that has predefined images you can use to -run applications in independent "containers." These containers run in a single Linux -instance. [Docker Hub](https://hub.docker.com/) is a database of pre-built images you can -use to test and build your applications. +When you run a Docker container on your local machine, it acts as a reproducible build environment. +You can run tests in the container, instead of testing on a dedicated CI/CD server. -When you use Docker with GitLab CI/CD, Docker runs each job in a separate and isolated -container. You specify the container image in the project's -[`.gitlab-ci.yml`](../yaml/README.md) file. +To run CI/CD jobs in a Docker container, you need to: -Docker containers provide a reproducible build environment that -can run on your workstation. When a Docker container is running, you can test -commands from your shell, rather than having to -test them on a dedicated CI server. +- Register a runner that uses the Docker executor. Then all jobs run in a Docker container. +- Specify an image in your `.gitlab-ci.yml` file. The runner creates a container from this image + and runs the jobs in it. +- Optional. Specify other images in your `.gitlab-ci.yml` file. These containers are known as + ["services"](#what-is-a-service) and you can use them to run services like MySQL separately. -## Register Docker Runner +## Register a runner that uses the Docker executor -To use GitLab Runner with Docker you need to [register a new runner](https://docs.gitlab.com/runner/register/) -to use the `docker` executor. +To use GitLab Runner with Docker you need to [register a runner](https://docs.gitlab.com/runner/register/) +that uses the Docker executor. In this example, we first set up a temporary template to supply the services: diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 2122cf2bf16..0344e736dd4 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Building images with kaniko and GitLab CI/CD +# Use kaniko to build Docker images > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45512) in GitLab 11.2. Requires GitLab Runner 11.2 and above. diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index eecc8ffd18f..358117ed796 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -74,15 +74,15 @@ runs by enabling the [Skip outdated deployment jobs](../pipelines/settings.md#sk Example of a problematic pipeline flow **before** enabling Skip outdated deployment jobs: -1. Pipeline-A is created on the `master` branch. -1. Later, Pipeline-B is created on the `master` branch (with a newer commit SHA). +1. Pipeline-A is created on the default branch. +1. Later, Pipeline-B is created on the default branch (with a newer commit SHA). 1. The `deploy` job in Pipeline-B finishes first, and deploys the newer code. 1. The `deploy` job in Pipeline-A finished later, and deploys the older code, **overwriting** the newer (latest) deployment. The improved pipeline flow **after** enabling Skip outdated deployment jobs: -1. Pipeline-A is created on the `master` branch. -1. Later, Pipeline-B is created on the `master` branch (with a newer SHA). +1. Pipeline-A is created on the default branch. +1. Later, Pipeline-B is created on the default branch (with a newer SHA). 1. The `deploy` job in Pipeline-B finishes first, and deploys the newer code. 1. The `deploy` job in Pipeline-A is automatically cancelled, so that it doesn't overwrite the deployment from the newer pipeline. diff --git a/doc/ci/environments/img/deployments_list.png b/doc/ci/environments/img/deployments_list.png Binary files differnew file mode 100644 index 00000000000..95afc092d86 --- /dev/null +++ b/doc/ci/environments/img/deployments_list.png diff --git a/doc/ci/environments/img/environment_auto_stop_v13_10.png b/doc/ci/environments/img/environment_auto_stop_v13_10.png Binary files differnew file mode 100644 index 00000000000..1525f670ff2 --- /dev/null +++ b/doc/ci/environments/img/environment_auto_stop_v13_10.png diff --git a/doc/ci/environments/img/environments_dynamic_groups_v13_10.png b/doc/ci/environments/img/environments_dynamic_groups_v13_10.png Binary files differnew file mode 100644 index 00000000000..cf3f9f7c781 --- /dev/null +++ b/doc/ci/environments/img/environments_dynamic_groups_v13_10.png diff --git a/doc/ci/environments/img/environments_list.png b/doc/ci/environments/img/environments_list.png Binary files differnew file mode 100644 index 00000000000..6ddfd858ab6 --- /dev/null +++ b/doc/ci/environments/img/environments_list.png diff --git a/doc/ci/environments/img/environments_terminal_button_on_index_v13_10.png b/doc/ci/environments/img/environments_terminal_button_on_index_v13_10.png Binary files differnew file mode 100644 index 00000000000..13c8d1cd523 --- /dev/null +++ b/doc/ci/environments/img/environments_terminal_button_on_index_v13_10.png diff --git a/doc/ci/environments/img/environments_terminal_button_on_show_v13_10.png b/doc/ci/environments/img/environments_terminal_button_on_show_v13_10.png Binary files differnew file mode 100644 index 00000000000..fcc3e2b6631 --- /dev/null +++ b/doc/ci/environments/img/environments_terminal_button_on_show_v13_10.png diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index b49fcd72172..abb12852fac 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -10,261 +10,149 @@ disqus_identifier: 'https://docs.gitlab.com/ee/ci/environments.html' > Introduced in GitLab 8.9. -Environments allow control of the continuous deployment of your software, -all within GitLab. +Environments describe where code is deployed. -## Introduction - -There are many stages required in the software development process before the software is ready -for public consumption. - -For example: - -1. Develop your code. -1. Test your code. -1. Deploy your code into a testing or staging environment before you release it to the public. - -This helps find bugs in your software, and also in the deployment process as well. - -GitLab CI/CD is capable of not only testing or building your projects, but also -deploying them in your infrastructure, with the added benefit of giving you a -way to track your deployments. In other words, you always know what is -currently being deployed or has been deployed on your servers. - -It's important to know that: - -- Environments are like tags for your CI jobs, describing where code gets deployed. -- Deployments are created when [GitLab CI/CD](../yaml/README.md) is used to deploy versions of code to environments. +Each time [GitLab CI/CD](../yaml/README.md) deploys a version of code to an environment, +a deployment is created. GitLab: -- Provides a full history of your deployments for each environment. -- Keeps track of your deployments, so you always know what is currently being deployed on your +- Provides a full history of deployments to each environment. +- Tracks your deployments, so you always know what is deployed on your servers. -If you have a deployment service such as [Kubernetes](../../user/project/clusters/index.md) -associated with your project, you can use it to assist with your deployments, and -can even access a [web terminal](#web-terminals) for your environment from within GitLab! - -## Configuring environments - -Configuring environments involves: - -1. Understanding how [pipelines](../pipelines/index.md) work. -1. Defining environments in your project's [`.gitlab-ci.yml`](../yaml/README.md) file. -1. Creating a job configured to deploy your application. For example, a deploy job configured with [`environment`](../yaml/README.md#environment) to deploy your application to a [Kubernetes cluster](../../user/project/clusters/index.md). - -The rest of this section illustrates how to configure environments and deployments using -an example scenario. It assumes you have already: - -- Created a [project](../../user/project/working_with_projects.md#create-a-project) in GitLab. -- Set up [a runner](../runners/README.md). - -In the scenario: - -- We are developing an application. -- We want to run tests and build our app on all branches. -- Our default branch is `master`. -- We deploy the app only when a pipeline on `master` branch is run. - -### Defining environments - -Let's consider the following `.gitlab-ci.yml` example: - -```yaml -stages: - - test - - build - - deploy - -test: - stage: test - script: echo "Running tests" - -build: - stage: build - script: echo "Building the app" +If you have a deployment service like [Kubernetes](../../user/project/clusters/index.md) +associated with your project, you can use it to assist with your deployments. +You can even access a [web terminal](#web-terminals) for your environment from within GitLab. -deploy_staging: - stage: deploy - script: - - echo "Deploy to staging server" - environment: - name: staging - url: https://staging.example.com - only: - - master -``` +## View environments and deployments -We have defined three [stages](../yaml/README.md#stages): +Prerequisites: -- `test` -- `build` -- `deploy` +- You must have a minimum of [Reporter permission](../../user/permissions.md#project-members-permissions). -The jobs assigned to these stages run in this order. If any job fails, then -the pipeline fails and jobs that are assigned to the next stage don't run. +To view a list of environments and deployments: -In our case: +1. Go to the project's **Operations > Environments** page. + The environments are displayed. -- The `test` job runs first. -- Then the `build` job. -- Lastly the `deploy_staging` job. +  -With this configuration, we: +1. To view a list of deployments for an environment, select the environment name, + for example, `staging`. -- Check that the tests pass. -- Ensure that our app is able to be built successfully. -- Lastly we deploy to the staging server. +  -Note that the `environment` keyword defines where the app is deployed. The environment `name` and -`url` is exposed in various places within GitLab. Each time a job that has an environment specified -succeeds, a deployment is recorded along with the Git SHA and environment name. +Deployments show up in this list only after a deployment job has created them. -WARNING: -Some characters are not allowed in environment names. Use only letters, -numbers, spaces, and `-`, `_`, `/`, `{`, `}`, or `.`. Also, it must not start nor end with `/`. +## Types of environments -In summary, with the above `.gitlab-ci.yml` we have achieved the following: +There are two types of environments: -- All branches run the `test` and `build` jobs. -- The `deploy_staging` job runs [only](../yaml/README.md#onlyexcept-basic) on the `master` - branch, which means all merge requests that are created from branches don't - get deployed to the staging server. -- When a merge request is merged, all jobs run and the `deploy_staging` - job deploys our code to a staging server while the deployment - is recorded in an environment named `staging`. +- Static environments have static names, like `staging` or `production`. +- Dynamic environments have dynamic names. Dynamic environments + are a fundamental part of [Review apps](../review_apps/index.md). -#### CI/CD variables and runners +### Create a static environment -Starting with GitLab 8.15, the environment name is exposed to the runner in -two forms: +You can create an environment and deployment in the UI or in your `.gitlab-ci.yml` file. -- `$CI_ENVIRONMENT_NAME`. The name given in `.gitlab-ci.yml` (with any CI/CD variables - expanded). -- `$CI_ENVIRONMENT_SLUG`. A "cleaned-up" version of the name, suitable for use in URLs, - DNS, etc. +In the UI: -If you change the name of an existing environment, the: +1. Go to the project's **Operations > Environments** page. +1. Select **New environment**. +1. Enter a name and external URL. +1. Select **Save**. -- `$CI_ENVIRONMENT_NAME` variable is updated with the new environment name. -- `$CI_ENVIRONMENT_SLUG` variable remains unchanged to prevent unintended side - effects. +In your `.gitlab-ci.yml` file: -Starting with GitLab 9.3, the environment URL is exposed to the runner via -`$CI_ENVIRONMENT_URL`. The URL is expanded from either: +1. Specify a name for the environment and optionally, a URL, which determines the deployment URL. + For example: -- `.gitlab-ci.yml`. -- The external URL from the environment if not defined in `.gitlab-ci.yml`. + ```yaml + deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + ``` -#### Set dynamic environment URLs after a job finishes +1. Trigger a deployment. (For example, by creating and pushing a commit.) -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17066) in GitLab 12.9. +When the job runs, the environment and deployment are created. -In a job script, you can specify a static [environment URL](#using-the-environment-url). -However, there may be times when you want a dynamic URL. For example, -if you deploy a Review App to an external hosting -service that generates a random URL per deployment, like `https://94dd65b.amazonaws.com/qa-lambda-1234567`, -you don't know the URL before the deployment script finishes. -If you want to use the environment URL in GitLab, you would have to update it manually. - -To address this problem, you can configure a deployment job to report back a set of -variables, including the URL that was dynamically-generated by the external service. -GitLab supports the [dotenv (`.env`)](https://github.com/bkeepers/dotenv) file format, -and expands the `environment:url` value with variables defined in the `.env` file. - -To use this feature, specify the -[`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) keyword in `.gitlab-ci.yml`. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Set dynamic URLs after a job finished](https://youtu.be/70jDXtOf4Ig). +NOTE: +Some characters cannot be used in environment names. +For more information about the `environment` keywords, see +[the `.gitlab-ci.yml` keyword reference](../yaml/README.md#environment). -##### Example of setting dynamic environment URLs +### Create a dynamic environment -The following example shows a Review App that creates a new environment -per merge request. The `review` job is triggered by every push, and -creates or updates an environment named `review/your-branch-name`. -The environment URL is set to `$DYNAMIC_ENVIRONMENT_URL`: +To create a dynamic name and URL for an environment, you can use +[predefined CI/CD variables](../variables/predefined_variables.md). For example: ```yaml -review: - script: - - DYNAMIC_ENVIRONMENT_URL=$(deploy-script) # In script, get the environment URL. - - echo "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" >> deploy.env # Add the value to a dotenv file. - artifacts: - reports: - dotenv: deploy.env # Report back dotenv file to rails. - environment: - name: review/$CI_COMMIT_REF_SLUG - url: $DYNAMIC_ENVIRONMENT_URL # and set the variable produced in script to `environment:url` - on_stop: stop_review - -stop_review: +deploy_review: + stage: deploy script: - - ./teardown-environment - when: manual + - echo "Deploy a review app" environment: - name: review/$CI_COMMIT_REF_SLUG - action: stop + name: review/$CI_COMMIT_REF_NAME + url: https://$CI_ENVIRONMENT_SLUG.example.com + only: + - branches + except: + - master ``` -As soon as the `review` job finishes, GitLab updates the `review/your-branch-name` -environment's URL. -It parses the `deploy.env` report artifact, registers a list of variables as runtime-created, -uses it for expanding `environment:url: $DYNAMIC_ENVIRONMENT_URL` and sets it to the environment URL. -You can also specify a static part of the URL at `environment:url:`, such as -`https://$DYNAMIC_ENVIRONMENT_URL`. If the value of `DYNAMIC_ENVIRONMENT_URL` is -`example.com`, the final result is `https://example.com`. +In this example: -The assigned URL for the `review/your-branch-name` environment is [visible in the UI](#using-the-environment-url). +- The `name` is `review/$CI_COMMIT_REF_NAME`. Because the [environment name](../yaml/README.md#environmentname) + can contain slashes (`/`), you can use this pattern to distinguish between dynamic and static environments. +- For the `url`, you could use `$CI_COMMIT_REF_NAME`, but because this value + may contain a `/` or other characters that would not be valid in a domain name or URL, + use `$CI_ENVIRONMENT_SLUG` instead. The `$CI_ENVIRONMENT_SLUG` variable is guaranteed to be unique. -Note the following: +You do not have to use the same prefix or only slashes (`/`) in the dynamic environment name. +However, when you use this format, you can [group similar environments](#group-similar-environments). -- `stop_review` doesn't generate a dotenv report artifact, so it doesn't recognize the - `DYNAMIC_ENVIRONMENT_URL` environment variable. Therefore you shouldn't set `environment:url:` in the - `stop_review` job. -- If the environment URL isn't valid (for example, the URL is malformed), the system doesn't update - the environment URL. -- If the script that runs in `stop_review` exists only in your repository and therefore can't use - `GIT_STRATEGY: none`, configure [pipelines for merge requests](../../ci/merge_request_pipelines/index.md) - for these jobs. This ensures that runners can fetch the repository even after a feature branch is - deleted. For more information, see [Ref Specs for Runners](../pipelines/index.md#ref-specs-for-runners). +NOTE: +Some variables cannot be used as environment names or URLs. +For more information about the `environment` keywords, see +[the `.gitlab-ci.yml` keyword reference](../yaml/README.md#environment). -### Configuring manual deployments +## Deployment tier of environments (**FREE**) -Adding `when: manual` to an automatically executed job's configuration converts it to -a job requiring manual action. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300741) in GitLab 13.10. -To expand on the [previous example](#defining-environments), the following includes -another job that deploys our app to a production server and is -tracked by a `production` environment. +There are cases where you might want to use a code name as an environment name instead of using +an [industry standard](https://en.wikipedia.org/wiki/Deployment_environment). For example, your environment might be called `customer-portal` instead of `production`. +This is perfectly fine, however, it loses information that the specific +environment is used as production. -The `.gitlab-ci.yml` file for this is as follows: +To keep information that a specific environment is for production or +some other use, you can set one of the following tiers to each environment: -```yaml -stages: - - test - - build - - deploy +| Environment tier | Environment names examples | +| ---- | -------- | +| `production` | Production, Live | +| `staging` | Staging, Model, Pre, Demo | +| `testing` | Test, QC | +| `development` | Dev, [Review apps](../review_apps/index.md), Trunk | +| `other` | | -test: - stage: test - script: echo "Running tests" +By default, an approximate tier is automatically guessed and set from [the environment name](../yaml/README.md#environmentname). +Alternatively, you can specify a specific tier with `deployment_tier` keyword, +see the [`.gitlab-ci.yml` syntax reference](../yaml/README.md#environmentdeployment_tier) for more details. -build: - stage: build - script: echo "Building the app" +## Configure manual deployments -deploy_staging: - stage: deploy - script: - - echo "Deploy to staging server" - environment: - name: staging - url: https://staging.example.com - only: - - master +You can create a job that requires someone to manually start the deployment. +For example: +```yaml deploy_prod: stage: deploy script: @@ -279,106 +167,12 @@ deploy_prod: The `when: manual` action: -- Exposes a "play" button in the GitLab UI for that job. -- Means the `deploy_prod` job is only triggered when the "play" button is clicked. - -You can find the "play" button in the pipelines, environments, deployments, and jobs views. - -| View | Screenshot | -|:----------------|:-------------------------------------------------------------------------------| -| Pipelines |  | -| Single pipeline |  | -| Environments |  | -| Deployments |  | -| Jobs |  | - -Clicking the play button in any view triggers the `deploy_prod` job. The deployment is recorded as a -new environment named `production`. - -If your environment's name is `production` (all lowercase), it's recorded in -[Value Stream Analytics](../../user/analytics/value_stream_analytics.md). - -### Configuring dynamic environments - -Regular environments are good when deploying to "stable" environments like staging or production. - -However, for environments for branches other than `master`, dynamic environments -can be used. Dynamic environments make it possible to create environments on the fly by -declaring their names dynamically in `.gitlab-ci.yml`. - -Dynamic environments are a fundamental part of [Review apps](../review_apps/index.md). - -#### Allowed variables - -The `name` and `url` keywords for dynamic environments can use most available CI/CD variables, -including: - -- [Predefined CI/CD variables](../variables/README.md#predefined-cicd-variables) -- [Project and group CI/CD variables](../variables/README.md) -- [`.gitlab-ci.yml` CI/CD variables](../yaml/README.md#variables) - -However, you cannot use variables defined: - -- Under `script`. -- On the runner's side. - -There are also other variables that are unsupported in the context of `environment:name`. -For more information, see [Where variables can be used](../variables/where_variables_can_be_used.md). +- Exposes a play button for the job in the GitLab UI. +- Means the `deploy_prod` job is only triggered when the play button is clicked. -#### Example configuration +You can find the play button in the pipelines, environments, deployments, and jobs views. -Runners expose various [predefined CI/CD variables](../variables/predefined_variables.md) when a job runs, so -you can use them as environment names. - -In the following example, the job deploys to all branches except `master`: - -```yaml -deploy_review: - stage: deploy - script: - - echo "Deploy a review app" - environment: - name: review/$CI_COMMIT_REF_NAME - url: https://$CI_ENVIRONMENT_SLUG.example.com - only: - - branches - except: - - master -``` - -In this example: - -- The job's name is `deploy_review` and it runs on the `deploy` stage. -- We set the `environment` with the `environment:name` as `review/$CI_COMMIT_REF_NAME`. - Since the [environment name](../yaml/README.md#environmentname) can contain slashes (`/`), we can - use this pattern to distinguish between dynamic and regular environments. -- We tell the job to run [`only`](../yaml/README.md#onlyexcept-basic) on branches, - [`except`](../yaml/README.md#onlyexcept-basic) `master`. - -For the value of: - -- `environment:name`, the first part is `review`, followed by a `/` and then `$CI_COMMIT_REF_NAME`, - which receives the value of the branch name. -- `environment:url`, we want a specific and distinct URL for each branch. `$CI_COMMIT_REF_NAME` - may contain a `/` or other characters that would be invalid in a domain name or URL, - so we use `$CI_ENVIRONMENT_SLUG` to guarantee that we get a valid URL. - - For example, given a `$CI_COMMIT_REF_NAME` of `100-Do-The-Thing`, the URL is something - like `https://100-do-the-4f99a2.example.com`. Again, the way you set up - the web server to serve these requests is based on your setup. - - We have used `$CI_ENVIRONMENT_SLUG` here because it is guaranteed to be unique. If - you're using a workflow like [GitLab Flow](../../topics/gitlab_flow.md), collisions - are unlikely and you may prefer environment names to be more closely based on the - branch name. In that case, you could use `$CI_COMMIT_REF_NAME` in `environment:url` in - the example above: `https://$CI_COMMIT_REF_NAME.example.com`, which would give a URL - of `https://100-do-the-thing.example.com`. - -You aren't required to use the same prefix or only slashes (`/`) in the dynamic environments' names. -However, using this format enables the [grouping similar environments](#grouping-similar-environments) -feature. - -### Configuring Kubernetes deployments +## Configure Kubernetes deployments > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. @@ -413,237 +207,147 @@ deploy: - master ``` -When deploying to a Kubernetes cluster using the GitLab Kubernetes integration, -information about the cluster and namespace is displayed above the job +When you use the GitLab Kubernetes integration to deploy to a Kubernetes cluster, +cluster and namespace information is displayed above the job trace on the deployment job page:  -#### Configuring incremental rollouts +### Configure incremental rollouts Learn how to release production changes to only a portion of your Kubernetes pods with [incremental rollouts](../environments/incremental_rollouts.md). -### Deployment safety +## CI/CD variables for environments and deployments -Deployment jobs can be more sensitive than other jobs in a pipeline, -and might need to be treated with an extra care. There are multiple features -in GitLab that helps maintain deployment security and stability. +When you create an environment, you specify the name and URL. -- [Restrict write-access to a critical environment](deployment_safety.md#restrict-write-access-to-a-critical-environment) -- [Limit the job-concurrency for deployment jobs](deployment_safety.md#ensure-only-one-deployment-job-runs-at-a-time) -- [Skip outdated deployment jobs](deployment_safety.md#skip-outdated-deployment-jobs) -- [Prevent deployments during deploy freeze windows](deployment_safety.md#prevent-deployments-during-deploy-freeze-windows) +If you want to use the name or URL in another job, you can use: -### Complete example +- `$CI_ENVIRONMENT_NAME`. The name defined in the `.gitlab-ci.yml` file. +- `$CI_ENVIRONMENT_SLUG`. A "cleaned-up" version of the name, suitable for use in URL and DNS, for example. + This variable is guaranteed to be unique. +- `$CI_ENVIRONMENT_URL`. The environment's URL, which was specified in the + `.gitlab-ci.yml` file or automatically assigned. -The configuration in this section provides a full development workflow where your app is: +If you change the name of an existing environment, the: -- Tested. -- Built. -- Deployed as a Review App. -- Deployed to a staging server after the merge request is merged. -- Finally, able to be manually deployed to the production server. +- `$CI_ENVIRONMENT_NAME` variable is updated with the new environment name. +- `$CI_ENVIRONMENT_SLUG` variable remains unchanged to prevent unintended side + effects. -The following combines the previous configuration examples, including: +## Set dynamic environment URLs after a job finishes -- Defining [simple environments](#defining-environments) for testing, building, and deployment to staging. -- Adding [manual actions](#configuring-manual-deployments) for deployment to production. -- Creating [dynamic environments](#configuring-dynamic-environments) for deployments for reviewing. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17066) in GitLab 12.9. -```yaml -stages: - - test - - build - - deploy +In a job script, you can specify a static environment URL. +However, there may be times when you want a dynamic URL. For example, +if you deploy a Review App to an external hosting +service that generates a random URL per deployment, like `https://94dd65b.amazonaws.com/qa-lambda-1234567`. +In this case, you don't know the URL before the deployment script finishes. +If you want to use the environment URL in GitLab, you would have to update it manually. -test: - stage: test - script: echo "Running tests" +To address this problem, you can configure a deployment job to report back a set of +variables. These variables include the URL that was dynamically-generated by the external service. +GitLab supports the [dotenv (`.env`)](https://github.com/bkeepers/dotenv) file format, +and expands the `environment:url` value with variables defined in the `.env` file. -build: - stage: build - script: echo "Building the app" +To use this feature, specify the +[`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) keyword in `.gitlab-ci.yml`. -deploy_review: - stage: deploy - script: - - echo "Deploy a review app" - environment: - name: review/$CI_COMMIT_REF_NAME - url: https://$CI_ENVIRONMENT_SLUG.example.com - only: - - branches - except: - - master +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Set dynamic URLs after a job finished](https://youtu.be/70jDXtOf4Ig). -deploy_staging: - stage: deploy - script: - - echo "Deploy to staging server" - environment: - name: staging - url: https://staging.example.com - only: - - master +### Example of setting dynamic environment URLs -deploy_prod: - stage: deploy +The following example shows a Review App that creates a new environment +per merge request. The `review` job is triggered by every push, and +creates or updates an environment named `review/your-branch-name`. +The environment URL is set to `$DYNAMIC_ENVIRONMENT_URL`: + +```yaml +review: script: - - echo "Deploy to production server" + - DYNAMIC_ENVIRONMENT_URL=$(deploy-script) # In script, get the environment URL. + - echo "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" >> deploy.env # Add the value to a dotenv file. + artifacts: + reports: + dotenv: deploy.env # Report back dotenv file to rails. environment: - name: production - url: https://example.com - when: manual - only: - - master -``` - -A more realistic example would also include copying files to a location where a -webserver (for example, NGINX) could then access and serve them. - -The example below copies the `public` directory to `/srv/nginx/$CI_COMMIT_REF_SLUG/public`: + name: review/$CI_COMMIT_REF_SLUG + url: $DYNAMIC_ENVIRONMENT_URL # and set the variable produced in script to `environment:url` + on_stop: stop_review -```yaml -review_app: - stage: deploy +stop_review: script: - - rsync -av --delete public /srv/nginx/$CI_COMMIT_REF_SLUG + - ./teardown-environment + when: manual environment: - name: review/$CI_COMMIT_REF_NAME - url: https://$CI_COMMIT_REF_SLUG.example.com + name: review/$CI_COMMIT_REF_SLUG + action: stop ``` -This example requires that NGINX and GitLab Runner are set up on the server this job runs on. - -See the [limitations](#limitations) section for some edge cases regarding the naming of your -branches and Review Apps. - -The complete example provides the following workflow to developers: - -- Create a branch locally. -- Make changes and commit them. -- Push the branch to GitLab. -- Create a merge request. - -Behind the scenes, the runner: - -- Picks up the changes and starts running the jobs. -- Runs the jobs sequentially as defined in `stages`: - - First, run the tests. - - If the tests succeed, build the app. - - If the build succeeds, the app is deployed to an environment with a name specific to the - branch. - -So now, every branch: - -- Gets its own environment. -- Is deployed to its own unique location, with the added benefit of: - - Having a [history of deployments](#viewing-deployment-history). - - Being able to [rollback changes](#retrying-and-rolling-back) if needed. - -For more information, see [Using the environment URL](#using-the-environment-url). +As soon as the `review` job finishes, GitLab updates the `review/your-branch-name` +environment's URL. +It parses the `deploy.env` report artifact, registers a list of variables as runtime-created, +uses it for expanding `environment:url: $DYNAMIC_ENVIRONMENT_URL` and sets it to the environment URL. +You can also specify a static part of the URL at `environment:url:`, such as +`https://$DYNAMIC_ENVIRONMENT_URL`. If the value of `DYNAMIC_ENVIRONMENT_URL` is +`example.com`, the final result is `https://example.com`. -### Protected environments +The assigned URL for the `review/your-branch-name` environment is visible in the UI. -Environments can be "protected", restricting access to them. +Note the following: -For more information, see [Protected environments](protected_environments.md). +- `stop_review` doesn't generate a dotenv report artifact, so it doesn't recognize the + `DYNAMIC_ENVIRONMENT_URL` environment variable. Therefore you shouldn't set `environment:url:` in the + `stop_review` job. +- If the environment URL isn't valid (for example, the URL is malformed), the system doesn't update + the environment URL. +- If the script that runs in `stop_review` exists only in your repository and therefore can't use + `GIT_STRATEGY: none`, configure [pipelines for merge requests](../../ci/merge_request_pipelines/index.md) + for these jobs. This ensures that runners can fetch the repository even after a feature branch is + deleted. For more information, see [Ref Specs for Runners](../pipelines/index.md#ref-specs-for-runners). ## Working with environments Once environments are configured, GitLab provides many features for working with them, as documented below. -### Viewing environments and deployments - -A list of environments and deployment statuses is available on each project's **Operations > Environments** page. - -For example: - - - -This example shows: - -- The environment's name with a link to its deployments. -- The last deployment ID number and who performed it. -- The job ID of the last deployment with its respective job name. -- The commit information of the last deployment, such as who committed it, to what - branch, and the Git SHA of the commit. -- The exact time the last deployment was performed. -- The upcoming deployment, if a deployment for the environment is in progress. -- When the environment stops automatically. -- A button that takes you to the URL that you defined under the `environment` keyword - in `.gitlab-ci.yml`. -- A number of deployment actions, including: - - Prevent the environment from [stopping automatically](#automatically-stopping-an-environment). - - [Open the live environment](#using-the-environment-url). - - Trigger [a manual deployment to a different environment](#configuring-manual-deployments). - - [Retry the deployment](#retrying-and-rolling-back). - - [Stop the environment](#stopping-an-environment). - -The information shown in the **Environments** page is limited to the latest -deployments, but an environment can have multiple deployments. - -> **Notes:** -> -> - While you can create environments manually in the web interface, we recommend -> that you define your environments in `.gitlab-ci.yml` first. They will -> be automatically created for you after the first deploy. -> - The environments page can only be viewed by users with [Reporter permission](../../user/permissions.md#project-members-permissions) -> and above. For more information on permissions, see the [permissions documentation](../../user/permissions.md). -> - Only deploys that happen after your `.gitlab-ci.yml` is properly configured -> show up in the **Environment** and **Last deployment** lists. - -### Viewing deployment history - -GitLab keeps track of your deployments, so you: - -- Always know what is currently being deployed on your servers. -- Can have the full history of your deployments for every environment. - -Clicking on an environment shows the history of its deployments. Here's an example **Environments** page -with multiple deployments: +### Environment rollback - +When you roll back a deployment on a specific commit, +a _new_ deployment is created. This deployment has its own unique job ID. +It points to the commit you're rolling back to. -This view is similar to the **Environments** page, but all deployments are shown. Also in this view -is a **Rollback** button. For more information, see [Retrying and rolling back](#retrying-and-rolling-back). +For the rollback to succeed, the deployment process must be defined in +the job's `script`. -### Retrying and rolling back +#### Retry or roll back a deployment If there is a problem with a deployment, you can retry it or roll it back. To retry or rollback a deployment: -1. Navigate to **Operations > Environments**. -1. Click on the environment. -1. In the deployment history list for the environment, click the: - - **Retry** button next to the last deployment, to retry that deployment. - - **Rollback** button next to a previously successful deployment, to roll back to that deployment. +1. Go to the project's **Operations > Environments**. +1. Select the environment. +1. To the right of the deployment name: + - To retry a deployment, select **Re-deploy to environment**. + - To roll back to a deployment, next to a previously successful deployment, select **Rollback environment**. -#### What to expect with a rollback +### Environment URL -Pressing the **Rollback** button on a specific commit triggers a _new_ deployment with its own -unique job ID. This new deployment points to the commit you're -rolling back to. +The [environment URL](../yaml/README.md#environmenturl) is displayed in a few +places in GitLab: -Note that the defined deployment process in the job's `script` determines whether the rollback -succeeds. - -### Using the environment URL - -The [environment URL](../yaml/README.md#environmenturl) is exposed in a few -places within GitLab: - -- In a merge request widget as a link: +- In a merge request as a link:  - In the Environments view as a button: -  +  - In the Deployments view as a button:  -You can see this information in a merge request itself if: +You can see this information in a merge request if: - The merge request is eventually merged to the default branch (usually `master`). - That branch also deploys to an environment (for example, `staging` or `production`). @@ -659,24 +363,27 @@ from source files to public pages in the environment set for Review Apps. ### Stopping an environment -Stopping an environment: +When you stop an environment: -- Moves it from the list of **Available** environments to the list of **Stopped** - environments on the [**Environments** page](#viewing-environments-and-deployments). -- Executes an [`on_stop` action](../yaml/README.md#environmenton_stop), if defined. +- On the **Environments** page, it moves from the list of **Available** environments + to the list of **Stopped** environments. +- An [`on_stop` action](../yaml/README.md#environmenton_stop), if defined, is executed. -This is often used when multiple developers are working on a project at the same time, -each of them pushing to their own branches, causing many dynamic environments to be created. - -Starting with GitLab 8.14, dynamic environments stop automatically when their associated branch is +Dynamic environments stop automatically when their associated branch is deleted. -#### Automatically stopping an environment +#### Stop an environment when a branch is deleted + +You can configure environments to stop when a branch is deleted. -Environments can be stopped automatically using special configuration. +The following example shows a `deploy_review` job that calls a `stop_review` job +to clean up and stop the environment. The `stop_review` job must be in the same +`stage` as the `deploy_review` job. -Consider the following example where the `deploy_review` job calls `stop_review` -to clean up and stop the environment: +Both jobs must have the same [`rules`](../yaml/README.md#onlyexcept-basic) +or [`only/except`](../yaml/README.md#onlyexcept-basic) configuration. Otherwise, +the `stop_review` job might not be included in all pipelines that include the +`deploy_review` job, and you cannot trigger `action: stop` to stop the environment automatically. ```yaml deploy_review: @@ -702,55 +409,30 @@ stop_review: when: manual ``` -If you can't use [Pipelines for merge requests](../merge_request_pipelines/index.md), -setting the [`GIT_STRATEGY`](../runners/README.md#git-strategy) to `none` is necessary in the -`stop_review` job so that the [runner](https://docs.gitlab.com/runner/) doesn't +If you can't use [pipelines for merge requests](../merge_request_pipelines/index.md), +set the [`GIT_STRATEGY`](../runners/README.md#git-strategy) to `none` in the +`stop_review` job. Then the [runner](https://docs.gitlab.com/runner/) doesn't try to check out the code after the branch is deleted. -When you have an environment that has a stop action defined (typically when -the environment describes a Review App), GitLab automatically triggers a -stop action when the associated branch is deleted. The `stop_review` job must -be in the same `stage` as the `deploy_review` job in order for the environment -to automatically stop. +Read more in the [`.gitlab-ci.yml` reference](../yaml/README.md#environmenton_stop). -Additionally, both jobs should have matching [`rules`](../yaml/README.md#onlyexcept-basic) -or [`only/except`](../yaml/README.md#onlyexcept-basic) configuration. In the example -above, if the configuration isn't identical, the `stop_review` job might not be -included in all pipelines that include the `deploy_review` job, and it isn't -possible to trigger `action: stop` to stop the environment automatically. - -You can read more in the [`.gitlab-ci.yml` reference](../yaml/README.md#environmenton_stop). - -#### Environments auto-stop +#### Stop an environment after a certain time period > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20956) in GitLab 12.8. -You can set an expiry time for environments and stop them automatically after a certain period. - -For example, consider the use of this feature with Review App environments. When you set up Review -Apps, sometimes they keep running for a long time because some merge requests are left open and -forgotten. Such idle environments waste resources and should be terminated as soon as possible. - -To address this problem, you can specify an optional expiration date for Review App environments. -When the expiry time is reached, GitLab automatically triggers a job to stop the environment, -eliminating the need of manually doing so. In case an environment is updated, the expiration is -renewed ensuring that only active merge requests keep running Review Apps. +You can set environments to stop automatically after a certain time period. -To enable this feature, you must specify the [`environment:auto_stop_in`](../yaml/README.md#environmentauto_stop_in) -keyword in `.gitlab-ci.yml`. You can specify a human-friendly date as the value, such as -`1 hour and 30 minutes` or `1 day`. `auto_stop_in` uses the same format of -[`artifacts:expire_in` docs](../yaml/README.md#artifactsexpire_in). +In your `.gitlab-ci.yml` file, specify the [`environment:auto_stop_in`](../yaml/README.md#environmentauto_stop_in) +keyword. You can specify a human-friendly date as the value, such as `1 hour and 30 minutes` or `1 day`. +After the time period passes, GitLab automatically triggers a job to stop the environment. -Note that due to resource limitation, a background worker for stopping environments only runs once +Due to resource limitations, a background worker for stopping environments only runs once every hour. This means that environments aren't stopped at the exact timestamp specified, but are instead stopped when the hourly cron worker detects expired environments. -##### Auto-stop example - -In the following example, there is a basic review app setup that creates a new environment -per merge request. The `review_app` job is triggered by every push and -creates or updates an environment named `review/your-branch-name`. -The environment keeps running until `stop_review_app` is executed: +In the following example, each merge request creates a new Review App environment. +Each push triggers the `review_app` job and an environment named `review/your-branch-name` +is created or updated. The environment runs until `stop_review_app` is executed: ```yaml review_app: @@ -772,54 +454,54 @@ stop_review_app: when: manual ``` -As long as a merge request is active and keeps getting new commits, -the review app doesn't stop, so developers don't need to worry about -re-initiating review app. +As long as the merge request is active and keeps getting new commits, +the Review App doesn't stop. Developers don't need to worry about +re-initiating Review App. -On the other hand, since `stop_review_app` is set to `auto_stop_in: 1 week`, -if a merge request becomes inactive for more than a week, +Because `stop_review_app` is set to `auto_stop_in: 1 week`, +if a merge request is inactive for more than a week, GitLab automatically triggers the `stop_review_app` job to stop the environment. -You can also check the expiration date of environments through the GitLab UI. To do so, -go to **Operations > Environments > Environment**. You can see the auto-stop period -at the left-top section and a pin-mark button at the right-top section. This pin-mark -button can be used to prevent auto-stopping the environment. By clicking this button, the -`auto_stop_in` setting is overwritten and the environment is active until it's stopped manually. +#### View a deployment's scheduled stop time - +You can view a deployment's expiration date in the GitLab UI. -#### Delete a stopped environment +1. Go to the project's **Operations > Environments** page. +1. Select the name of the deployment. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20620) in GitLab 12.10. +In the top left, next to the environment name, the expiration date is displayed. -You can delete [stopped environments](#stopping-an-environment) in one of two -ways: through the GitLab UI or through the API. +#### Override a deployment's scheduled stop time -##### Delete environments through the UI +You can manually override a deployment's expiration date. -To view the list of **Stopped** environments, navigate to **Operations > Environments** -and click the **Stopped** tab. +1. Go to the project's **Operations > Environments** page. +1. Select the deployment name. +1. In the top right, select the thumbtack (**{thumbtack}**). -From there, you can click the **Delete** button directly, or you can click the -environment name to see its details and **Delete** it from there. + -You can also delete environments by viewing the details for a -stopped environment: +The `auto_stop_in` setting is overwritten and the environment remains active until it's stopped manually. - 1. Navigate to **Operations > Environments**. - 1. Click on the name of an environment within the **Stopped** environments list. - 1. Click on the **Delete** button that appears at the top for all stopped environments. - 1. Finally, confirm your chosen environment in the modal that appears to delete it. +#### Delete a stopped environment -##### Delete environments through the API +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20620) in GitLab 12.10. + +You can delete [stopped environments](#stopping-an-environment) in the GitLab UI or by using +[the API](../../api/environments.md#delete-an-environment). -Environments can also be deleted by using the [Environments API](../../api/environments.md#delete-an-environment). +To delete a stopped environment in the GitLab UI: + +1. Go to the project's **Operations > Environments** page. +1. Select the **Stopped** tab. +1. Next to the environment you want to delete, select **Delete environment**. +1. On the confirmation dialog box, select **Delete environment**. ### Prepare an environment > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208655) in GitLab 13.2. -By default, GitLab creates a [deployment](#viewing-deployment-history) every time a +By default, GitLab creates a deployment every time a build with the specified environment runs. Newer deployments can also [cancel older ones](deployment_safety.md#skip-outdated-deployment-jobs). @@ -840,19 +522,19 @@ build: url: https://staging.example.com ``` -### Grouping similar environments +### Group similar environments > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7015) in GitLab 8.14. -As documented in [Configuring dynamic environments](#configuring-dynamic-environments), you can -prepend environment name with a word, followed by a `/`, and finally the branch -name, which is automatically defined by the `CI_COMMIT_REF_NAME` predefined CI/CD variable. +You can group environments into collapsible sections in the UI. + +For example, if all of your environments start with the name `review`, +then in the UI, the environments are grouped under that heading: -In short, environments that are named like `type/foo` are all presented under the same -group, named `type`. + -In our [minimal example](#example-configuration), we named the environments `review/$CI_COMMIT_REF_NAME` -where `$CI_COMMIT_REF_NAME` is the branch name. Here is a snippet of the example: +The following example shows how to start your environment names with `review`. +The `$CI_COMMIT_REF_NAME` variable is populated with the branch name at runtime: ```yaml deploy_review: @@ -863,11 +545,6 @@ deploy_review: name: review/$CI_COMMIT_REF_NAME ``` -In this case, if you visit the **Environments** page and the branches -exist, you should see something like: - - - ### Environment incident management You have successfully setup a Continuous Delivery/Deployment workflow in your project. @@ -897,7 +574,7 @@ severity is shown, so you can identify which environments need immediate attenti When the issue that triggered the alert is resolved, it is removed and is no longer visible on the environment page. -If the alert requires a [rollback](#retrying-and-rolling-back), you can select the +If the alert requires a [rollback](#retry-or-roll-back-a-deployment), you can select the deployment tab from the environment page and select which deployment to roll back to. #### Auto Rollback **(ULTIMATE)** @@ -939,8 +616,6 @@ Once configured, GitLab attempts to retrieve [supported performance metrics](../ for any environment that has had a successful deployment. If monitoring data was successfully retrieved, a **Monitoring** button appears for each environment. - - Clicking the **Monitoring** button displays a new page showing up to the last 8 hours of performance data. It may take a minute or two for data to appear after initial deployment. @@ -971,13 +646,13 @@ Note that container-based deployments often lack basic tools (like an editor), a be stopped or restarted at any time. If this happens, you lose all your changes. Treat this as a debugging tool, not a comprehensive online IDE. -Once enabled, your environments gain a **Terminal** button: +Once enabled, your environments display a **Terminal** button: - + You can also access the terminal button from the page for a specific environment: - + Wherever you find it, clicking the button takes you to a separate page to establish the terminal session: @@ -1054,24 +729,15 @@ such as [Review Apps](../review_apps/index.md) (`review/*`). Note that the most _specific_ spec takes precedence over the other wildcard matching. In this case, the `review/feature-1` spec takes precedence over `review/*` and `*` specs. -### Environments Dashboard **(PREMIUM)** - -See [Environments Dashboard](../environments/environments_dashboard.md) for a summary of each -environment's operational health. - -## Limitations - -In the `environment: name`, you are limited to only the [predefined CI/CD variables](../variables/predefined_variables.md). -Re-using variables defined inside `script` as part of the environment name doesn't work. - -## Further reading - -Below are some links you may find interesting: +## Related topics -- [The `.gitlab-ci.yml` definition of environments](../yaml/README.md#environment) -- [A blog post on Deployments & Environments](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/) -- [Review Apps - Use dynamic environments to deploy your code for every branch](../review_apps/index.md) -- [Deploy Boards for your applications running on Kubernetes](../../user/project/deploy_boards.md) +- [Use GitLab CI to deploy to multiple environments (blog post)](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/) +- [Review Apps](../review_apps/index.md): Use dynamic environments to deploy your code for every branch. +- [Deploy Boards](../../user/project/deploy_boards.md): View the status of your applications running on Kubernetes. +- [Protected environments](protected_environments.md): Determine who can deploy code to your environments. +- [Environments Dashboard](../environments/environments_dashboard.md): View a summary of each + environment's operational health. **(PREMIUM)** +- [Deployment safety](deployment_safety.md#restrict-write-access-to-a-critical-environment): Secure your deployments. <!-- ## Troubleshooting diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 2636e59723a..9a639fde5f6 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -70,7 +70,7 @@ Alternatively, you can use the API to protect an environment: name: ${CI_JOB_NAME} ``` -1. Use the UI to [create a new group](../../user/group/index.md#create-a-new-group). +1. Use the UI to [create a new group](../../user/group/index.md#create-a-group). For example, this group is called `protected-access-group` and has the group ID `9899826`. Note that the rest of the examples in these steps use this group. @@ -125,10 +125,18 @@ they have the following privileges: ## Deployment-only access to protected environments Users granted access to a protected environment, but not push or merge access -to the branch deployed to it, are only granted access to deploy the environment. +to the branch deployed to it, are only granted access to deploy the environment. An individual in a +group with the Reporter permission, or in groups added to the project with Reporter permissions, +appears in the dropdown menu for deployment-only access. -Note that deployment-only access is the only possible access level for users with -[Reporter permissions](../../user/permissions.md). +To add deployment-only access: + +1. Add a group with Reporter permissions. +1. Add user(s) to the group. +1. Invite the group to be a project member. +1. Follow the steps outlined in [Protecting Environments](#protecting-environments). + +Note that deployment-only access is the only possible access level for groups with [Reporter permissions](../../user/permissions.md). ## Modifying and unprotecting environments diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index b48dd561a66..fc6807fd191 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -25,7 +25,6 @@ The following table lists examples with step-by-step tutorials that are containe | Use case | Resource | |-------------------------------|----------| | Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | -| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | | Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | | GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | | End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | @@ -35,8 +34,6 @@ The following table lists examples with step-by-step tutorials that are containe | PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | | PHP with npm, SCP | [Running Composer and npm scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | | PHP with PHPunit, `atoum` | [Testing PHP projects](php.md). | -| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | -| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | | Secrets management with Vault | [Authenticating and Reading Secrets With HashiCorp Vault](authenticating-with-hashicorp-vault/index.md). | ### Contributed examples @@ -47,10 +44,13 @@ separate example projects: | Use case | Resource | |-------------------------------|----------| +| Clojure | [Test a Clojure application with GitLab CI/CD](https://gitlab.com/gitlab-examples/clojure-web-application). | | Game development | [DevOps and Game Development with GitLab CI/CD](https://gitlab.com/gitlab-examples/gitlab-game-demo/). | | Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](https://gitlab.com/gitlab-examples/maven/simple-maven-example). | | Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). | | Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](https://gitlab.com/gitlab-examples/python-getting-started). | +| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](https://gitlab.com/gitlab-examples/ruby-getting-started). | | Scala on Heroku | [Test and deploy a Scala application to Heroku](https://gitlab.com/gitlab-examples/scala-sbt). | ## CI/CD templates diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index 2d8c92a1a74..40ba7cff5f9 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -53,7 +53,9 @@ The JWT's payload looks like this: "job_id": "1212", # "ref": "auto-deploy-2020-04-01", # Git ref for this job "ref_type": "branch", # Git ref type, branch or tag - "ref_protected": "true" # true if this git ref is protected, false otherwise + "ref_protected": "true", # true if this git ref is protected, false otherwise + "environment": "production", # Environment this job deploys to, if present (GitLab 13.9 and later) + "environment_protected": "true" # true if deployed environment is protected, false otherwise (GitLab 13.9 and later) } ``` @@ -178,7 +180,7 @@ $ vault write auth/jwt/config \ For the full list of available configuration options, see Vault's [API documentation](https://www.vaultproject.io/api/auth/jwt#configure). -The following job, when run for the `master` branch, is able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: +The following job, when run for the default branch, is able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: ```yaml read_secrets: diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 779ca98084f..4d2c22a17f0 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -116,7 +116,7 @@ We also use two secure variables: ## Storing API keys To add secure variables, navigate to your project's -**Settings > CI / CD > Variables**. The variables that are defined +**Settings > CI/CD > Variables**. The variables that are defined in the project settings are sent along with the build script to the runner. The secure variables are stored out of the repository. Never store secrets in your project's `.gitlab-ci.yml`. It is also important that the secret's value @@ -128,4 +128,4 @@ or `%` (for Windows Batch runners): 1. `$VARIABLE` - use it for non-Windows runners 1. `%VARIABLE%` - use it for Windows Batch runners -Read more about the [CI variables](../../variables/README.md). +Read more about the [CI/CD variables](../../variables/README.md). diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 2d7ba2bc759..62607320410 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -122,7 +122,7 @@ Therefore, for a production environment we use additional steps to ensure that a Since this was a WordPress project, I gave real life code snippets. Some further ideas you can pursue: -- Having a slightly different script for `master` branch allows you to deploy to a production server from that branch and to a stage server from any other branches. +- Having a slightly different script for the default branch allows you to deploy to a production server from that branch and to a stage server from any other branches. - Instead of pushing it live, you can push it to WordPress official repository (with creating a SVN commit, etc.). - You could generate i18n text domains on the fly. diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index e20e86e8936..07bad3afc65 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -50,14 +50,14 @@ infrastructure is up and running, and that your units of code work well together [Selenium](https://www.selenium.dev/) is a piece of software that can control web browsers, e.g., to make them visit a specific URL or interact with elements on the page. It can be programmatically controlled from a variety of programming languages. In this article we're going to be using the -[WebdriverIO](https://webdriver.io/) JavaScript bindings, but the general concept should carry over +[WebdriverIO](http://v4.webdriver.io/) JavaScript bindings, but the general concept should carry over pretty well to [other programming languages supported by Selenium](https://www.selenium.dev/documentation/en/legacy_docs/selenium_rc/). ## Writing tests You can write tests using -[several testing frameworks supported by WebdriverIO](https://webdriver.io/guide/testrunner/frameworks.html). +[several testing frameworks supported by WebdriverIO](http://v4.webdriver.io/guide/testrunner/frameworks.html). We will be using [Jasmine](https://jasmine.github.io/) here: ```javascript @@ -82,14 +82,14 @@ multiple tests, such as making sure you are logged in. The function `it` defines an individual test. -[The `browser` object](https://webdriver.io/guide/testrunner/browserobject.html) is WebdriverIO's -special sauce. It provides most of [the WebdriverIO API methods](https://webdriver.io/api.html) that are the key to +[The `browser` object](http://v4.webdriver.io/guide/testrunner/browserobject.html) is WebdriverIO's +special sauce. It provides most of [the WebdriverIO API methods](http://v4.webdriver.io/docs/api/) that are the key to steering the browser. In this case, we can use -[`browser.url`](https://webdriver.io/api/protocol/url.html) to visit `/page-that-does-not-exist` to -hit our 404 page. We can then use [`browser.getUrl`](https://webdriver.io/api/property/getUrl.html) +[`browser.url`](http://v4.webdriver.io/api/protocol/url.html) to visit `/page-that-does-not-exist` to +hit our 404 page. We can then use [`browser.getUrl`](http://v4.webdriver.io/api/property/getUrl.html) to verify that the current page is indeed at the location we specified. To interact with the page, we can simply pass CSS selectors to -[`browser.element`](https://webdriver.io/api/protocol/element.html) to get access to elements on the +[`browser.element`](http://v4.webdriver.io/api/protocol/element.html) to get access to elements on the page and to interact with them - for example, to click on the link back to the home page. The simple test shown above @@ -111,9 +111,9 @@ you can use [the Webpack Dev Server WebdriverIO plugin](https://www.npmjs.com/pa that automatically starts a development server before executing the tests. The WebdriverIO documentation has -[an overview of all configuration options](https://webdriver.io/guide/getstarted/configuration.html), but the +[an overview of all configuration options](http://v4.webdriver.io/guide/getstarted/configuration.html), but the easiest way to get started is to start with -[WebdriverIO's default configuration](https://webdriver.io/guide/testrunner/configurationfile.html), which +[WebdriverIO's default configuration](http://v4.webdriver.io/guide/testrunner/configurationfile.html), which provides an overview of all available options. The two options that are going to be most relevant now are the `specs` option, which is an array of paths to your tests, and the `baseUrl` option, which points to where your app is running. And finally, we will need to tell WebdriverIO in which browsers we would like to run our @@ -186,7 +186,7 @@ e2e:chrome: Now that we have a job to run the end-to-end tests in, we need to tell WebdriverIO how to connect to the Selenium servers running alongside it. We've already cheated a bit above by -passing the value of the [`host`](https://webdriver.io/guide/getstarted/configuration.html#host) +passing the value of the [`host`](http://v4.webdriver.io/guide/getstarted/configuration.html#host) option as an argument to `npm run confidence-check` on the command line. However, we still need to tell WebdriverIO which browser is available for it to use. @@ -253,7 +253,7 @@ production project, see: - [Flockademic's `.gitlab-ci.yml`](https://gitlab.com/Flockademic/Flockademic/blob/dev/.gitlab-ci.yml) - [Flockademic's tests](https://gitlab.com/Flockademic/Flockademic/tree/dev/__e2e__) -There's plenty more that WebdriverIO can do. For example, you can configure a [`screenshotPath`](https://webdriver.io/guide/getstarted/configuration.html#screenshotPath) to tell WebdriverIO to take +There's plenty more that WebdriverIO can do. For example, you can configure a [`screenshotPath`](http://v4.webdriver.io/guide/getstarted/configuration.html#screenshotPath) to tell WebdriverIO to take a screenshot when tests are failing. Then tell GitLab CI/CD to store those [artifacts](../../yaml/README.md#artifacts), and you'll be able to see what went wrong within GitLab. diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index a02a5347734..2acd7315630 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -128,7 +128,7 @@ We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our  -We also need to add the public key to **Project** > **Settings** > **Repository** as a [Deploy Key](../../../ssh/README.md#deploy-keys), which gives us the ability to access our repository from the server through [SSH protocol](../../../gitlab-basics/command-line-commands.md#start-working-on-your-project). +We also need to add the public key to **Project** > **Settings** > **Repository** as a [Deploy Key](../../../user/project/deploy_keys/index.md), which gives us the ability to access our repository from the server through [SSH protocol](../../../gitlab-basics/command-line-commands.md#start-working-on-your-project). ```shell # As the deployer user on the server @@ -619,7 +619,7 @@ deploy_production: - master ``` -You may also want to add another job for [staging environment](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/), to final test your application before deploying to production. +You may also want to add another job for [staging environment](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/), to final test your application before deploying to production. ### Turn on GitLab CI/CD diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index c0fc93fe1b3..28a0080626a 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -91,7 +91,7 @@ As part of publishing a package, semantic-release increases the version number i 1. Navigate to **Project > Settings > Access Tokens**. 1. Give the token a name, and select the `api` scope. 1. Click **Create project access token** and copy its value. -1. Navigate to **Project > Settings > CI / CD > Variables**. +1. Navigate to **Project > Settings > CI/CD > Variables**. 1. Click **Add Variable**. 1. In the **Key** field, enter `GITLAB_TOKEN`. In the **Value** field, paste the token created above. Check the **Mask variable** option and click **Add variable**. @@ -126,7 +126,7 @@ Test the pipeline by creating a commit with a message like: fix: testing patch releases ``` -Push the commit to `master`. The pipeline should create a new release (`v1.0.0`) on the project's **Releases** page and publish a new version of the package to the project's **Package Registry** page. +Push the commit to the default branch. The pipeline should create a new release (`v1.0.0`) on the project's **Releases** page and publish a new version of the package to the project's **Package Registry** page. To create a minor release, use a commit message like: diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 28d00362309..4a6555a58a6 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -1,101 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: 'README.md#contributed-examples' --- -# Test and deploy a Python application with GitLab CI/CD +This document was moved to [another location](README.md#contributed-examples). -This example will guide you how to run tests in your Python application and deploy it automatically as Heroku application. - -You can also view or fork the complete [example source](https://gitlab.com/ayufan/python-getting-started). - -## Configure project - -This is what the `.gitlab-ci.yml` file looks like for this project: - -```yaml -stages: - - test - - deploy - -test: - stage: test - script: - # this configures Django application to use attached postgres database that is run on `postgres` host - - export DATABASE_URL=postgres://postgres:@postgres:5432/python-test-app - - apt-get update -qy - - apt-get install -y python-dev python-pip - - pip install -r requirements.txt - - python manage.py test - -staging: - stage: deploy - script: - - apt-get update -qy - - apt-get install -y ruby-dev - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-python-test-staging --api-key=$HEROKU_STAGING_API_KEY - only: - - master - -production: - stage: deploy - script: - - apt-get update -qy - - apt-get install -y ruby-dev - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-python-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY - only: - - tags -``` - -This project has three jobs: - -- `test` - used to test Django application. -- `staging` - used to automatically deploy staging environment every push to `master` branch. -- `production` - used to automatically deploy production environment for every created tag. - -## Store API keys - -You'll need to create two variables in **Settings > CI/CD > Variables** in your GitLab project: - -- `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. -- `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. - -Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). - -## Create Heroku application - -For each of your environments, you'll need to create a new Heroku application. -You can do this through the [Dashboard](https://dashboard.heroku.com/). - -## Create a runner - -First install [Docker Engine](https://docs.docker.com/installation/). - -To build this project you also need to have [GitLab Runner](https://docs.gitlab.com/runner/index.html). -You can use public runners available on `gitlab.com` or you can register your own: - -```shell -cat > /tmp/test-config.template.toml << EOF -[[runners]] -[runners.docker] -[[runners.docker.services]] -name = "postgres:latest" -EOF - -gitlab-runner register \ - --non-interactive \ - --url "https://gitlab.com/" \ - --registration-token "PROJECT_REGISTRATION_TOKEN" \ - --description "python-3.5" \ - --executor "docker" \ - --template-config /tmp/test-config.template.toml \ - --docker-image python:3.5 -``` - -With the command above, you create a runner that uses the [`python:3.5`](https://hub.docker.com/_/python) image and uses a [PostgreSQL](https://hub.docker.com/_/postgres) database. - -To access the PostgreSQL database, connect to `host: postgres` as user `postgres` with no password. +<!-- This redirect file can be deleted after 2021-06-01. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 5bf0b3d01be..4a6555a58a6 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -1,97 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: 'README.md#contributed-examples' --- -# Test and deploy a Ruby application with GitLab CI/CD +This document was moved to [another location](README.md#contributed-examples). -This example will guide you through how to run tests in your Ruby on Rails application and deploy it automatically as a Heroku application. - -You can also view or fork the complete [example source](https://gitlab.com/ayufan/ruby-getting-started) and view the logs of its past [CI jobs](https://gitlab.com/ayufan/ruby-getting-started/-/jobs?scope=finished). - -## Configure the project - -This is what the `.gitlab-ci.yml` file looks like for this project: - -```yaml -test: - stage: test - script: - - apt-get update -qy - - apt-get install -y nodejs - - bundle install --path /cache - - bundle exec rake db:create RAILS_ENV=test - - bundle exec rake test - -staging: - stage: deploy - script: - - gem install dpl --pre - - dpl heroku api --app=gitlab-ci-ruby-test-staging --api-key=$HEROKU_STAGING_API_KEY - only: - - master - -production: - stage: deploy - script: - - gem install dpl --pre - - dpl heroku api --app=gitlab-ci-ruby-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY - only: - - tags -``` - -This project has three jobs: - -- `test` - used to test Rails application. -- `staging` - used to automatically deploy staging environment every push to `master` branch. -- `production` - used to automatically deploy production environment for every created tag. - -## Store API keys - -You'll need to create two CI/CD variables in your project's **Settings > CI/CD > Variables** and do not check **Protect variable** or **Mask variable**: - -- `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. -- `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. - -Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). - -## Create Heroku application - -For each of your environments, you'll need to create a new Heroku application. -You can do this through the [Heroku Dashboard](https://dashboard.heroku.com/). - -## Create a runner - -First install [Docker Engine](https://docs.docker.com/installation/). - -To build this project you also need to have [GitLab Runner](https://docs.gitlab.com/runner/). -You can use public runners available on `gitlab.com` or register your own. Start by -creating a template configuration file to pass complex configuration: - -```shell -cat > /tmp/test-config.template.toml << EOF -[[runners]] -[runners.docker] -[[runners.docker.services]] -name = "postgres:latest" -EOF -``` - -Finally, register the runner, passing the newly-created template configuration file: - -```shell -gitlab-runner register \ - --non-interactive \ - --url "https://gitlab.com/" \ - --registration-token "PROJECT_REGISTRATION_TOKEN" \ - --description "ruby:2.6" \ - --executor "docker" \ - --template-config /tmp/test-config.template.toml \ - --docker-image ruby:2.6 -``` - -With the command above, you create a runner that uses the [`ruby:2.6`](https://hub.docker.com/_/ruby) image and uses a [PostgreSQL](https://hub.docker.com/_/postgres) database. - -To access the PostgreSQL database, connect to `host: postgres` as user `postgres` with no password. +<!-- This redirect file can be deleted after 2021-06-01. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/test-clojure-application.md b/doc/ci/examples/test-clojure-application.md index b6691930a2c..8aa1fb21275 100644 --- a/doc/ci/examples/test-clojure-application.md +++ b/doc/ci/examples/test-clojure-application.md @@ -1,46 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: 'README.md#contributed-examples' --- -NOTE: -This document has not been updated recently and could be out of date. For the latest documentation, see the [GitLab CI/CD](../README.md) page and the [GitLab CI/CD Pipeline Configuration Reference](../yaml/README.md). +This document was moved to [another location](README.md#contributed-examples). -# Test a Clojure application with GitLab CI/CD - -This example will guide you how to run tests on your Clojure application. - -You can view or fork the [example source](https://gitlab.com/dzaporozhets/clojure-web-application) and view the logs of its past [CI jobs](https://gitlab.com/dzaporozhets/clojure-web-application/builds?scope=finished). - -## Configure the project - -This is what the `.gitlab-ci.yml` file looks like for this project: - -```yaml -variables: - POSTGRES_DB: sample-test - DATABASE_URL: "postgresql://postgres@postgres:5432/sample-test" - -before_script: - - apt-get update -y - - apt-get install default-jre postgresql-client -y - - wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein - - chmod a+x lein - - export LEIN_ROOT=1 - - PATH=$PATH:. - - lein deps - - lein migratus migrate - -test: - script: - - lein test -``` - -In `before_script`, we install JRE and [Leiningen](https://leiningen.org/). - -The sample project uses the [migratus](https://github.com/yogthos/migratus) library to manage database migrations, and -we have added a database migration as the last step of `before_script`. - -You can use public runners available on `gitlab.com` for testing your application with this configuration. +<!-- This redirect file can be deleted after 2021-05-26. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index d9a40c1feb6..01df4f63c92 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -5,38 +5,23 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Using Git submodules with GitLab CI +# Using Git submodules with GitLab CI/CD -> **Notes:** -> -> - GitLab 8.12 introduced a new [CI job permissions model](../user/project/new_ci_build_permissions_model.md) and you -> are encouraged to upgrade your GitLab instance if you haven't done already. -> If you are **not** using GitLab 8.12 or higher, you would need to work your way -> around submodules in order to access the sources of e.g., `gitlab.com/group/project` -> with the use of [SSH keys](ssh_keys/index.md). -> - With GitLab 8.12 onward, your permissions are used to evaluate what a CI job -> can access. More information about how this system works can be found in the -> [Jobs permissions model](../user/permissions.md#job-permissions). -> - The HTTP(S) Git protocol [must be enabled](../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols) in your GitLab instance. +Use [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) to keep +a Git repository as a subdirectory of another Git repository. You can clone another +repository into your project and keep your commits separate. -## Configuring the `.gitmodules` file +## Configure the `.gitmodules` file -If dealing with [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules), your project probably has a file -named `.gitmodules`. +When you use Git submodules, your project should have a file named `.gitmodules`. +You might need to modify it to work in a GitLab CI/CD job. -Let's consider the following example: +For example, your `.gitmodules` configuration might look like the following if: -1. Your project is located at `https://gitlab.com/secret-group/my-project`. -1. To checkout your sources you usually use an SSH address like - `git@gitlab.com:secret-group/my-project.git`. -1. Your project depends on `https://gitlab.com/group/project`, which you want - to include as a submodule. - -If you are using GitLab 8.12+ and your submodule is on the same GitLab server, -you must update your `.gitmodules` file to use **relative URLs**. -Since Git allows the usage of relative URLs for your `.gitmodules` configuration, -this easily allows you to use HTTP(S) for cloning all your CI jobs and SSH -for all your local checkouts. The `.gitmodules` would look like: +- Your project is located at `https://gitlab.com/secret-group/my-project`. +- Your project depends on `https://gitlab.com/group/project`, which you want + to include as a submodule. +- You check out your sources with an SSH address like `git@gitlab.com:secret-group/my-project.git`. ```ini [submodule "project"] @@ -44,14 +29,16 @@ for all your local checkouts. The `.gitmodules` would look like: url = ../../group/project.git ``` -The above configuration instructs Git to automatically deduce the URL that -should be used when cloning sources. Whether you use HTTP(S) or SSH, Git uses -that same channel and it makes all your CI jobs use HTTP(S). -GitLab CI/CD only uses HTTP(S) for cloning your sources, and all your local -clones continue using SSH. +When your submodule is on the same GitLab server, you should use relative URLs in +your `.gitmodules` file. Then you can clone with HTTPS in all your CI/CD jobs. You +can also use SSH for all your local checkouts. + +The above configuration instructs Git to automatically deduce the URL to +use when cloning sources. Git uses the same configuration for both HTTPS and SSH. +GitLab CI/CD uses HTTPS for cloning your sources, and you can continue to use SSH +to clone locally. -For all other submodules not located on the same GitLab server, use the full -HTTP(S) protocol URL: +For submodules not located on the same GitLab server, use the full URL: ```ini [submodule "project-x"] @@ -59,45 +46,16 @@ HTTP(S) protocol URL: url = https://gitserver.com/group/project-x.git ``` -Once `.gitmodules` is correctly configured, you can move on to -[configuring your `.gitlab-ci.yml`](#using-git-submodules-in-your-ci-jobs). - -## Using Git submodules in your CI jobs +## Use Git submodules in CI/CD jobs -There are a few steps you need to take in order to make submodules work -correctly with your CI jobs: +To make submodules work correctly in CI/CD jobs: -1. First, make sure you have used [relative URLs](#configuring-the-gitmodules-file) - for the submodules located in the same GitLab server. -1. Next, if you are using `gitlab-runner` v1.10+, you can set the - `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` to tell - the runner to fetch your submodules before the job: +1. Make sure you use [relative URLs](#configure-the-gitmodules-file) + for submodules located in the same GitLab server. +1. You can set the `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` + to tell the runner to [fetch your submodules before the job](runners/README.md#git-submodule-strategy): ```yaml variables: GIT_SUBMODULE_STRATEGY: recursive ``` - - See the [GitLab Runner documentation](runners/README.md#git-submodule-strategy) - for more details about `GIT_SUBMODULE_STRATEGY`. - -1. If you are using an older version of `gitlab-runner`, then use - `git submodule sync/update` in `before_script`: - - ```yaml - before_script: - - git submodule sync --recursive - - git submodule update --init --recursive - ``` - - `--recursive` should be used in either both or none (`sync/update`) depending on - whether you have recursive submodules. - -The rationale to set the `sync` and `update` in `before_script` is because of -the way Git submodules work. On a fresh runner workspace, Git sets the -submodule URL including the token in `.git/config` -(or `.git/modules/<submodule>/config`) based on `.gitmodules` and the current -remote URL. On subsequent jobs on the same runner, `.git/config` is cached -and already contains a full URL for the submodule, corresponding to the previous -job, and to **a token from a previous job**. `sync` allows to force updating -the full URL. diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differindex 9e2b7e89577..4d49f5ea5bd 100644 --- a/doc/ci/img/deployments_view.png +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environment_auto_stop_v12_8.png b/doc/ci/img/environment_auto_stop_v12_8.png Binary files differdeleted file mode 100644 index f098938ef04..00000000000 --- a/doc/ci/img/environment_auto_stop_v12_8.png +++ /dev/null diff --git a/doc/ci/img/environments_available_13_10.png b/doc/ci/img/environments_available_13_10.png Binary files differnew file mode 100644 index 00000000000..94ffb0032fa --- /dev/null +++ b/doc/ci/img/environments_available_13_10.png diff --git a/doc/ci/img/environments_available_13_7.png b/doc/ci/img/environments_available_13_7.png Binary files differdeleted file mode 100644 index 2e1f56c5894..00000000000 --- a/doc/ci/img/environments_available_13_7.png +++ /dev/null diff --git a/doc/ci/img/environments_dynamic_groups.png b/doc/ci/img/environments_dynamic_groups.png Binary files differdeleted file mode 100644 index 37828ccd0c1..00000000000 --- a/doc/ci/img/environments_dynamic_groups.png +++ /dev/null diff --git a/doc/ci/img/environments_manual_action_deployments.png b/doc/ci/img/environments_manual_action_deployments.png Binary files differdeleted file mode 100644 index c5959c0003e..00000000000 --- a/doc/ci/img/environments_manual_action_deployments.png +++ /dev/null diff --git a/doc/ci/img/environments_manual_action_environments.png b/doc/ci/img/environments_manual_action_environments.png Binary files differdeleted file mode 100644 index b2ec27cc721..00000000000 --- a/doc/ci/img/environments_manual_action_environments.png +++ /dev/null diff --git a/doc/ci/img/environments_manual_action_jobs.png b/doc/ci/img/environments_manual_action_jobs.png Binary files differdeleted file mode 100644 index d948ee5da9e..00000000000 --- a/doc/ci/img/environments_manual_action_jobs.png +++ /dev/null diff --git a/doc/ci/img/environments_manual_action_pipelines.png b/doc/ci/img/environments_manual_action_pipelines.png Binary files differdeleted file mode 100644 index 332850afb7f..00000000000 --- a/doc/ci/img/environments_manual_action_pipelines.png +++ /dev/null diff --git a/doc/ci/img/environments_manual_action_single_pipeline.png b/doc/ci/img/environments_manual_action_single_pipeline.png Binary files differdeleted file mode 100644 index 8c1c0c1d993..00000000000 --- a/doc/ci/img/environments_manual_action_single_pipeline.png +++ /dev/null diff --git a/doc/ci/img/environments_terminal_button_on_index.png b/doc/ci/img/environments_terminal_button_on_index.png Binary files differdeleted file mode 100644 index 40110ff325f..00000000000 --- a/doc/ci/img/environments_terminal_button_on_index.png +++ /dev/null diff --git a/doc/ci/img/environments_terminal_button_on_show.png b/doc/ci/img/environments_terminal_button_on_show.png Binary files differdeleted file mode 100644 index e96ca9c9c7e..00000000000 --- a/doc/ci/img/environments_terminal_button_on_show.png +++ /dev/null diff --git a/doc/ci/img/pipelines_junit_test_report_ui_v12_5.png b/doc/ci/img/pipelines_junit_test_report_ui_v12_5.png Binary files differdeleted file mode 100644 index 5b1e3254f8b..00000000000 --- a/doc/ci/img/pipelines_junit_test_report_ui_v12_5.png +++ /dev/null diff --git a/doc/ci/img/pipelines_junit_test_report_v13_10.png b/doc/ci/img/pipelines_junit_test_report_v13_10.png Binary files differnew file mode 100644 index 00000000000..ef79a2547af --- /dev/null +++ b/doc/ci/img/pipelines_junit_test_report_v13_10.png diff --git a/doc/ci/img/pipelines_junit_test_report_with_errors_v13_10.png b/doc/ci/img/pipelines_junit_test_report_with_errors_v13_10.png Binary files differnew file mode 100644 index 00000000000..cfcf3bec76c --- /dev/null +++ b/doc/ci/img/pipelines_junit_test_report_with_errors_v13_10.png diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index d1fe6db3ee4..0c5fa59da8e 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -153,7 +153,7 @@ Job grouping is evaluated with an improved regular expression to group jobs by n The new implementation removes one or more `: [...]`, `X Y`, `X/Y`, or `X\Y` sequences from the **end** of job names only. -Matching substrings occuring at the beginning or in the middle of build names are +Matching substrings occurring at the beginning or in the middle of build names are no longer removed. ## Specifying variables when running manual jobs @@ -187,7 +187,7 @@ For example, if you start rolling out new code and: - Users do not experience trouble, GitLab can automatically complete the deployment from 0% to 100%. - Users experience trouble with the new code, you can stop the timed incremental rollout by canceling the pipeline - and [rolling](../environments/index.md#retrying-and-rolling-back) back to the last stable version. + and [rolling](../environments/index.md#retry-or-roll-back-a-deployment) back to the last stable version.  diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index 5c3cf6ec02e..7e76efe8b50 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -19,6 +19,12 @@ you can use *pipelines for merge requests*. In the UI, these pipelines are labeled as `detached`. Otherwise, these pipelines appear the same as other pipelines. +Pipelines for merge requests can run when you: + +- Create a new merge request. +- Commit changes to the source branch for the merge request. +- Select the **Run pipeline** button from the **Pipelines** tab in the merge request. + Any user who has developer [permissions](../../user/permissions.md) can run a pipeline for merge requests. @@ -213,7 +219,7 @@ The variable names begin with the `CI_MERGE_REQUEST_` prefix. ### Two pipelines created when pushing to a merge request If you are experiencing duplicated pipelines when using `rules`, take a look at -the [important differences between `rules` and `only`/`except`](../yaml/README.md#prevent-duplicate-pipelines), +the [important differences between `rules` and `only`/`except`](../yaml/README.md#avoid-duplicate-pipelines), which helps you get your starting configuration correct. If you are seeing two pipelines when using `only/except`, please see the caveats diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index e83789efdbf..72603ed94c0 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -32,7 +32,7 @@ can still be successfully merged into the target. When the merge request can't be merged, the pipeline runs against the source branch only. For example, when: - The target branch has changes that conflict with the changes in the source branch. -- The merge request is a [**Draft** merge request](../../../user/project/merge_requests/work_in_progress_merge_requests.md). +- The merge request is a [**Draft** merge request](../../../user/project/merge_requests/drafts.md). In these cases, the pipeline runs as a [pipeline for merge requests](../index.md) and is labeled as `detached`. If these cases no longer exist, new pipelines diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 3966e5e3e89..ed14e61c54b 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -26,12 +26,14 @@ Consider the following examples of data that can use Metrics Reports: ## How it works -Metrics are read from the metrics report (default: `metrics.txt`). They are parsed and displayed in the MR widget. +Metrics for a branch are read from the latest metrics report artifact (default filename: `metrics.txt`) as string values. -All values are considered strings and string compare is used to find differences between the latest available `metrics` artifact from: +For an MR, the values of these metrics from the feature branch are compared to the values from the target branch. Then they are displayed in the MR widget in this order: -- `master` -- The feature branch +- Existing metrics with changed values. +- Metrics that have been added by the MR. Marked with a **New** badge. +- Metrics that have been removed by the MR. Marked with a **Removed** badge. +- Existing metrics with unchanged values. ## How to set it up diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 4c186b8a64e..9736f8c1418 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -314,7 +314,7 @@ Some features are not implemented yet. For example, support for environments. You can trigger a pipeline in your project whenever a pipeline finishes for a new tag in a different project: -1. Go to the project's **Settings > CI / CD** page, and expand the **Pipeline subscriptions** section. +1. Go to the project's **Settings > CI/CD** page, and expand the **Pipeline subscriptions** section. 1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. 1. Click subscribe. diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index 9f4a1afe2f9..aabdc6cd36b 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -8,10 +8,7 @@ type: reference # Pipeline Editor **(FREE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4540) in GitLab 13.8. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-pipeline-editor). **(FREE SELF)** +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/270059) in GitLab 13.10. WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -30,7 +27,7 @@ From the pipeline editor page you can: NOTE: You must already have [a `.gitlab-ci.yml` file](../quick_start/index.md#create-a-gitlab-ciyml-file) -on the default branch (usually `master`) of your project to use the editor. +on the default branch of your project to use the editor. ## Validate CI configuration @@ -68,7 +65,6 @@ reflected in the CI lint. It displays the same results as the existing [CI Lint WARNING: This feature might not be available to you. Check the **version history** note above for details. -It is not accessible if the [pipeline editor is disabled](#enable-or-disable-pipeline-editor). To view a visualization of your `gitlab-ci.yml` configuration, in your project, go to **CI/CD > Editor**, and then select the **Visualize** tab. The @@ -107,7 +103,7 @@ Feature.enable(:ci_config_visualization_tab) ## View expanded configuration > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/246801) in GitLab 13.9. -> - It is [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It is [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. > - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-expanded-configuration). **(FREE SELF)** To view the fully expanded CI/CD configuration as one combined file, go to the @@ -122,20 +118,20 @@ where: ### Enable or disable expanded configuration **(FREE SELF)** Expanded CI/CD configuration is under development and not ready for production use. -It is deployed behind a feature flag that is **disabled by default**. +It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) can opt to enable it. -To enable it: +To disable it: ```ruby -Feature.enable(:ci_config_visualization_tab) +Feature.disable(:ci_config_merged_tab) ``` -To disable it: +To enable it: ```ruby -Feature.disable(:ci_config_visualization_tab) +Feature.enable(:ci_config_merged_tab) ``` ## Commit changes to CI configuration @@ -150,22 +146,3 @@ If you enter a new branch name, the **Start a new merge request with these chang checkbox appears. Select it to start a new merge request after you commit the changes.  - -## Enable or disable pipeline editor **(FREE SELF)** - -The pipeline editor is under development but ready for production use. It is -deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can disable it. - -To disable it: - -```ruby -Feature.disable(:ci_pipeline_editor_page) -``` - -To enable it: - -```ruby -Feature.enable(:ci_pipeline_editor_page) -``` diff --git a/doc/ci/pipelines/img/pipelines_settings_test_coverage.png b/doc/ci/pipelines/img/pipelines_settings_test_coverage.png Binary files differdeleted file mode 100644 index 13ed69be810..00000000000 --- a/doc/ci/pipelines/img/pipelines_settings_test_coverage.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 6bc0d9bddd9..c2fcbbcf72f 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -6,9 +6,7 @@ disqus_identifier: 'https://docs.gitlab.com/ee/ci/pipelines.html' type: reference --- -# CI/CD pipelines - -> Introduced in GitLab 8.8. +# CI/CD pipelines **(FREE)** NOTE: Watch the @@ -96,7 +94,7 @@ This table lists the refspecs injected for each pipeline type: The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a running pipeline job. This ref can be created even after the associated branch or tag has been -deleted. It's therefore useful in some features such as [automatically stopping an environment](../environments/index.md#automatically-stopping-an-environment), +deleted. It's therefore useful in some features such as [automatically stopping an environment](../environments/index.md#stopping-an-environment), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md) that might run pipelines after branch deletion. @@ -137,10 +135,10 @@ To execute a pipeline manually: 1. Navigate to your project's **CI/CD > Pipelines**. 1. Select the **Run Pipeline** button. 1. On the **Run Pipeline** page: - 1. Select the branch to run the pipeline for in the **Create for** field. + 1. Select the branch or tag to run the pipeline for in the **Run for branch name or tag** field. 1. Enter any [environment variables](../variables/README.md) required for the pipeline run. You can set specific variables to have their [values prefilled in the form](#prefill-variables-in-manual-pipelines). - 1. Click the **Create pipeline** button. + 1. Click the **Run pipeline** button. The pipeline now executes the jobs as configured. @@ -199,8 +197,6 @@ For each `var` or `file_var`, a key and value are required. ### Add manual interaction to your pipeline -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7931) in GitLab 8.15. - Manual actions, configured using the [`when:manual`](../yaml/README.md#whenmanual) keyword, allow you to require manual interaction before moving forward in the pipeline. @@ -208,7 +204,7 @@ You can do this straight from the pipeline graph. Just click the play button to execute that particular job. For example, your pipeline might start automatically, but it requires manual action to -[deploy to production](../environments/index.md#configuring-manual-deployments). In the example below, the `production` +[deploy to production](../environments/index.md#configure-manual-deployments). In the example below, the `production` stage has a job with a manual action.  @@ -324,8 +320,6 @@ runners do not use regular runners, they must be tagged accordingly. ## Visualize pipelines -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5742) in GitLab 8.11. - Pipelines can be complex structures with many sequential and parallel jobs. To make it easier to understand the flow of a pipeline, GitLab has pipeline graphs for viewing pipelines diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index a8e310c1f0d..752e2a97fad 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -36,7 +36,7 @@ Otherwise the pipeline is not created. To schedule a pipeline for project: -1. Navigate to the project's **CI / CD > Schedules** page. +1. Navigate to the project's **CI/CD > Schedules** page. 1. Click the **New schedule** button. 1. Fill in the **Schedule a new pipeline** form. 1. Click the **Save pipeline schedule** button. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 32221b78039..e607bae53bd 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -6,7 +6,7 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/settings.h type: reference, howto --- -# Pipeline settings +# Pipeline settings **(FREE)** To reach the pipelines settings navigate to your project's **Settings > CI/CD**. @@ -17,6 +17,11 @@ The following settings can be configured per project. For an overview, watch the video [GitLab CI Pipeline, Artifacts, and Environments](https://www.youtube.com/watch?v=PCKDICEe10s). Watch also [GitLab CI pipeline tutorial for beginners](https://www.youtube.com/watch?v=Jav4vbUrqII). +You can use the pipeline status to determine if a merge request can be merged: + +- [Merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md). +- [Only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds). + ## Git strategy With Git strategy, you can choose the default way your repository is fetched @@ -24,7 +29,7 @@ from GitLab in a job. There are two options. Using: -- `git clone`, which is slower since it clones the repository from scratch +- `git clone`, which is slower because it clones the repository from scratch for every job, ensuring that the local working copy is always pristine. - `git fetch`, which is default in GitLab and faster as it re-uses the local working copy (falling back to clone if it doesn't exist). @@ -70,20 +75,19 @@ For information about setting a maximum artifact size for a project, see ## Custom CI/CD configuration path -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12509) in GitLab 9.4. -> - [Support for external `.gitlab-ci.yml` locations](https://gitlab.com/gitlab-org/gitlab/-/issues/14376) introduced in GitLab 12.6. +> [Support for external `.gitlab-ci.yml` locations](https://gitlab.com/gitlab-org/gitlab/-/issues/14376) introduced in GitLab 12.6. By default we look for the `.gitlab-ci.yml` file in the project's root directory. If needed, you can specify an alternate path and filename, including locations outside the project. To customize the path: -1. Go to the project's **Settings > CI / CD**. +1. Go to the project's **Settings > CI/CD**. 1. Expand the **General pipelines** section. 1. Provide a value in the **CI/CD configuration file** field. 1. Click **Save changes**. -If the CI configuration is stored within the repository in a non-default +If the CI configuration is stored in the repository in a non-default location, the path must be relative to the root directory. Examples of valid paths and file names include: @@ -96,7 +100,7 @@ If hosting the CI configuration on an external site, the URL link must end with - `http://example.com/generate/ci/config.yml` -If hosting the CI configuration in a different project within GitLab, the path must be relative +If hosting the CI configuration in a different project in GitLab, the path must be relative to the root directory in the other project. Include the group and project name at the end: - `.gitlab-ci.yml@mygroup/another-project` @@ -114,10 +118,10 @@ able to edit it. ## Test coverage parsing If you use test coverage in your code, GitLab can capture its output in the -job log using a regular expression. In the pipelines settings, search for the -"Test coverage parsing" section. +job log using a regular expression. - +In your project, go to **Settings > CI/CD** and expand the **General pipelines** +section. Enter the regular expression in the **Test coverage parsing** field. Leave blank if you want to disable it or enter a Ruby regular expression. You can use <https://rubular.com> to test your regex. The regex returns the **last** @@ -133,19 +137,21 @@ averaged. <!-- vale gitlab.Spelling = NO --> -| Coverage Tool | Sample regular expression | -|------------------------------------------------|---------------------------| -| Simplecov (Ruby) | `\(\d+.\d+\%\) covered` | -| pytest-cov (Python) | `^TOTAL.+?(\d+\%)$` | +| Coverage Tool | Sample regular expression | +|------------------------------------------------|-----------------------------------------------| +| Simplecov (Ruby) | `\(\d+.\d+\%\) covered` | +| pytest-cov (Python) | `^TOTAL.+?(\d+\%)$` | | Scoverage (Scala) | `Statement coverage[A-Za-z\.*]\s*:\s*([^%]+)` | -| `phpunit --coverage-text --colors=never` (PHP) | `^\s*Lines:\s*\d+.\d+\%` | -| gcovr (C/C++) | `^TOTAL.*\s+(\d+\%)$` | -| `tap --coverage-report=text-summary` (NodeJS) | `^Statements\s*:\s*([^%]+)` | -| `nyc npm test` (NodeJS) | `All files[^|]*\|[^|]*\s+([\d\.]+)` | -| excoveralls (Elixir) | `\[TOTAL\]\s+(\d+\.\d+)%` | -| `mix test --cover` (Elixir) | `\d+.\d+\%\s+\|\s+Total` | -| JaCoCo (Java/Kotlin) | `Total.*?([0-9]{1,3})%` | -| `go test -cover` (Go) | `coverage: \d+.\d+% of statements` | +| `phpunit --coverage-text --colors=never` (PHP) | `^\s*Lines:\s*\d+.\d+\%` | +| gcovr (C/C++) | `^TOTAL.*\s+(\d+\%)$` | +| `tap --coverage-report=text-summary` (NodeJS) | `^Statements\s*:\s*([^%]+)` | +| `nyc npm test` (NodeJS) | `All files[^|]*\|[^|]*\s+([\d\.]+)` | +| excoveralls (Elixir) | `\[TOTAL\]\s+(\d+\.\d+)%` | +| `mix test --cover` (Elixir) | `\d+.\d+\%\s+\|\s+Total` | +| JaCoCo (Java/Kotlin) | `Total.*?([0-9]{1,3})%` | +| `go test -cover` (Go) | `coverage: \d+.\d+% of statements` | +| .Net (OpenCover) | `(Visited Points).*\((.*)\)` | +| .Net (`dotnet test` line coverage) | `Total\s*\|\s*(\d+\.?\d+)` | <!-- vale gitlab.Spelling = YES --> @@ -157,11 +163,13 @@ averaged. To see the evolution of your project code coverage over time, you can view a graph or download a CSV file with this data. From your project: -1. Go to **{chart}** **Project Analytics > Repository** to see the historic data for each job listed in the dropdown above the graph. +1. Go to **Project Analytics > Repository** to see the historic data for each job listed in the dropdown above the graph. 1. If you want a CSV file of that data, click **Download raw data (`.csv`)**  +Code coverage data is also [available at the group level](../../user/group/repositories_analytics/index.md). + ### Removing color codes Some test coverage tools output with ANSI color codes that aren't @@ -215,11 +223,9 @@ If **Public pipelines** is disabled: ## Auto-cancel redundant pipelines -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9362) in GitLab 9.1. - You can set pending or running pipelines to cancel automatically when a new pipeline runs on the same branch. You can enable this in the project settings: -1. Go to **Settings > CI / CD**. +1. Go to **Settings > CI/CD**. 1. Expand **General Pipelines**. 1. Check the **Auto-cancel redundant pipelines** checkbox. 1. Click **Save changes**. @@ -232,14 +238,14 @@ running job can be cancelled before it completes. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25276) in GitLab 12.9. Your project may have multiple concurrent deployment jobs that are -scheduled to run within the same time frame. +scheduled to run in the same time frame. This can lead to a situation where an older deployment job runs after a newer one, which may not be what you want. To avoid this scenario: -1. Go to **Settings > CI / CD**. +1. Go to **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Check the **Skip outdated deployment jobs** checkbox. 1. Click **Save changes**. @@ -272,15 +278,15 @@ pages. ### Pipeline status badge -Depending on the status of your job, a badge can have the following values: +Depending on the status of your pipeline, a badge can have the following values: -- pending -- running -- passed -- failed -- skipped -- canceled -- unknown +- `pending` +- `running` +- `passed` +- `failed` +- `skipped` +- `canceled` +- `unknown` You can access a pipeline status badge image using the following link: @@ -321,29 +327,27 @@ into your `README.md`: Pipeline badges can be rendered in different styles by adding the `style=style_name` parameter to the URL. Two styles are available: -#### Flat (default) +- Flat (default): -```plaintext -https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg?style=flat -``` + ```plaintext + https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg?style=flat + ``` - +  -#### Flat square +- Flat square ([Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30120) in GitLab 11.8): -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30120) in GitLab 11.8. + ```plaintext + https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg?style=flat-square + ``` -```plaintext -https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg?style=flat-square -``` - - +  ### Custom badge text > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17555) in GitLab 13.1. -The text for a badge can be customized. This can be useful to differentiate between multiple coverage jobs that run in the same pipeline. Customize the badge text and width by adding the `key_text=custom_text` and `key_width=custom_key_width` parameters to the URL: +The text for a badge can be customized to differentiate between multiple coverage jobs that run in the same pipeline. Customize the badge text and width by adding the `key_text=custom_text` and `key_width=custom_key_width` parameters to the URL: ```plaintext https://gitlab.com/gitlab-org/gitlab/badges/master/coverage.svg?job=karma&key_text=Frontend+Coverage&key_width=130 @@ -351,10 +355,6 @@ https://gitlab.com/gitlab-org/gitlab/badges/master/coverage.svg?job=karma&key_te  -## CI/CD Variables - -[CI/CD variables](../variables/README.md) can be set to be available to a runner. - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index 711bf0c0e7a..664523a08e4 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -66,7 +66,7 @@ In this file, you define: - The decisions the runner should make when specific conditions are encountered. For example, you might want to run a suite of tests when you commit to -any branch except `master`. When you commit to `master`, you want +any branch except the default branch. When you commit to the default branch, you want to run the same suite, but also publish your application. All of this is defined in the `.gitlab-ci.yml` file. @@ -117,15 +117,22 @@ The pipeline starts when the commit is committed. #### `.gitlab-ci.yml` tips -- If you want the runner to use a Docker image to run the jobs, edit the `.gitlab-ci.yml` file - to include your image name: +- If you want the runner to [use a Docker container to run the jobs](../docker/using_docker_images.md), + edit the `.gitlab-ci.yml` file + to include an image name: ```yaml default: image: ruby:2.7.2 ``` - This command tells the runner to use a Ruby image from Docker Hub. + This command tells the runner to use a Ruby image from Docker Hub + and to run the jobs in a container that's generated from the image. + + This process is different than + [building an application as a Docker container](../docker/using_docker_build.md). + Your application does not need to be built as a Docker container to + run CI/CD jobs in Docker containers. - To validate your `.gitlab-ci.yml` file, use the [CI Lint tool](../lint.md), which is available in every project. diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 9de6a1162bd..122f9caebe7 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -12,6 +12,10 @@ type: reference Review Apps is a collaboration tool that takes the hard work out of providing an environment to showcase product changes. +NOTE: +If you have a Kubernetes cluster, you can automate this feature in your applications +by using [Auto DevOps](../../topics/autodevops/index.md). + ## Introduction Review Apps: @@ -27,8 +31,8 @@ In the above example: - A Review App is built every time a commit is pushed to `topic branch`. - The reviewer fails two reviews before passing the third review. -- After the review has passed, `topic branch` is merged into `master` where it is deployed to staging. -- After having been approved in staging, the changes that were merged into `master` are deployed in to production. +- After the review passes, `topic branch` is merged into the default branch, where it's deployed to staging. +- After its approval in staging, the changes that were merged into the default branch are deployed to production. ## How Review Apps work @@ -52,7 +56,7 @@ After adding Review Apps to your workflow, you follow the branched Git flow. Tha ## Configuring Review Apps -Review Apps are built on [dynamic environments](../environments/index.md#configuring-dynamic-environments), which allow you to dynamically create a new environment for each branch. +Review Apps are built on [dynamic environments](../environments/index.md#create-a-dynamic-environment), which allow you to dynamically create a new environment for each branch. The process of configuring Review Apps is as follows: @@ -85,7 +89,7 @@ you can copy and paste into `.gitlab-ci.yml` as a starting point. To do so: ## Review Apps auto-stop -See how to [configure Review Apps environments to expire and auto-stop](../environments/index.md#environments-auto-stop) +See how to [configure Review Apps environments to expire and auto-stop](../environments/index.md#stop-an-environment-after-a-certain-time-period) after a given period of time. ## Review Apps examples @@ -282,8 +286,8 @@ The visual review tools retrieve the merge request ID from the `data-merge-reque data attribute included in the `script` HTML tag used to add the visual review tools to your review app. -​After determining the ID for the merge request to link to a visual review app, you -can supply the ID by either:​​ +After determining the ID for the merge request to link to a visual review app, you +can supply the ID by either: - Hard-coding it in the script tag via the data attribute `data-merge-request-id` of the app. - Dynamically adding the `data-merge-request-id` value during the build of the app. @@ -317,7 +321,3 @@ the user must enter a [personal access token](../../user/profile/personal_access with `api` scope before submitting feedback. This same method can be used to require authentication for any public projects. - -## Limitations - -Review App limitations are the same as [environments limitations](../environments/index.md#limitations). diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index bf7552ad609..d09daea9a75 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -38,7 +38,7 @@ multiple projects. If you are using a self-managed instance of GitLab: - Your administrator can install and register shared runners by - going to your project's **Settings > CI / CD**, expanding the **Runners** section, + going to your project's **Settings > CI/CD**, expanding the **Runners** section, and clicking **Show runner installation instructions**. These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html). - The administrator can also configure a maximum number of shared runner [pipeline minutes for @@ -220,7 +220,7 @@ Specific runners process jobs by using a first in, first out ([FIFO](https://en. NOTE: Specific runners do not get shared with forked projects automatically. -A fork *does* copy the CI / CD settings of the cloned repository. +A fork *does* copy the CI/CD settings of the cloned repository. #### Create a specific runner diff --git a/doc/ci/ssh_keys/index.md b/doc/ci/ssh_keys/index.md index 2cdef176a22..72a99efc9bc 100644 --- a/doc/ci/ssh_keys/index.md +++ b/doc/ci/ssh_keys/index.md @@ -33,7 +33,7 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/) 1. Run the [`ssh-agent`](https://linux.die.net/man/1/ssh-agent) during job to load the private key. 1. Copy the public key to the servers you want to have access to (usually in - `~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys) + `~/.ssh/authorized_keys`) or add it as a [deploy key](../../user/project/deploy_keys/index.md) if you are accessing a private GitLab repository. The private key is displayed in the job log, unless you enable @@ -101,7 +101,7 @@ to access it. This is where an SSH key pair comes in handy. 1. As a final step, add the _public_ key from the one you created in the first step to the services that you want to have an access to from within the build environment. If you are accessing a private GitLab repository you need to add - it as a [deploy key](../../ssh/README.md#deploy-keys). + it as a [deploy key](../../user/project/deploy_keys/index.md). That's it! You can now have access to private servers or repositories in your build environment. @@ -130,7 +130,7 @@ on, and use that key for all projects that are run on this machine. 1. As a final step, add the _public_ key from the one you created earlier to the services that you want to have an access to from within the build environment. If you are accessing a private GitLab repository you need to add it as a - [deploy key](../../ssh/README.md#deploy-keys). + [deploy key](../../user/project/deploy_keys/index.md). After generating the key, try to sign in to the remote server to accept the fingerprint: diff --git a/doc/ci/test_cases/img/test_case_list_v13_6.png b/doc/ci/test_cases/img/test_case_list_v13_6.png Binary files differdeleted file mode 100644 index b5d89582558..00000000000 --- a/doc/ci/test_cases/img/test_case_list_v13_6.png +++ /dev/null diff --git a/doc/ci/test_cases/img/test_case_show_v13_10.png b/doc/ci/test_cases/img/test_case_show_v13_10.png Binary files differnew file mode 100644 index 00000000000..81e87dfcff8 --- /dev/null +++ b/doc/ci/test_cases/img/test_case_show_v13_10.png diff --git a/doc/ci/test_cases/img/test_case_show_v13_6.png b/doc/ci/test_cases/img/test_case_show_v13_6.png Binary files differdeleted file mode 100644 index bbd7601cf30..00000000000 --- a/doc/ci/test_cases/img/test_case_show_v13_6.png +++ /dev/null diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md index 6385b1638ff..4bd38f4043a 100644 --- a/doc/ci/test_cases/index.md +++ b/doc/ci/test_cases/index.md @@ -13,7 +13,7 @@ type: reference Test cases in GitLab can help your teams create testing scenarios in their existing development platform. -This can help the Implementation and Testing teams collaborate, because they no longer have to +Now your Implementation and Testing teams can collaborate better, as they no longer have to use external test planning tools, which require additional overhead, context switching, and expense. ## Create a test case @@ -22,7 +22,7 @@ Users with Reporter or higher [permissions](../../user/permissions.md) can creat To create a test case in a GitLab project: -1. Navigate to **CI/CD > Test Cases**. +1. Go to **CI/CD > Test Cases**. 1. Select the **New test case** button. You are taken to the new test case form. Here you can enter the new case's title, [description](../../user/markdown.md), attach a file, and assign [labels](../../user/project/labels.md). 1. Select the **Submit test case** button. You are taken to view the new test case. @@ -34,14 +34,12 @@ issue list with a search query, including labels or the test case's title. Users with Guest or higher [permissions](../../user/permissions.md) can view test cases. - - To view a test case: -1. In a project, navigate to **CI/CD > Test Cases**. +1. In a project, go to **CI/CD > Test Cases**. 1. Select the title of the test case you want to view. You are taken to the test case page. - + ## Edit a test case @@ -68,7 +66,7 @@ To archive a test case, on the test case's page, select the **Archive test case* To view archived test cases: -1. Navigate to **CI/CD > Test Cases**. +1. Go to **CI/CD > Test Cases**. 1. Select **Archived**. ## Reopen an archived test case diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index b4cea48a362..fa97cbdfcec 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -188,10 +188,10 @@ source repository. Be sure to URL-encode `ref` if it contains slashes. ### Using webhook payload in the triggered pipeline > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31197) in GitLab 13.9. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-the-trigger_payload-variable). **(FREE SELF)** +> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-the-trigger_payload-variable). **(FREE SELF)** WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -203,21 +203,21 @@ so you can access the data with `cat $TRIGGER_PAYLOAD` or a similar command. #### Enable or disable the `TRIGGER_PAYLOAD` variable -The `TRIGGER_PAYLOAD` CI/CD variable is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. +The `TRIGGER_PAYLOAD` CI/CD variable is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can enable it. +can opt to disable it. -To enable it: +To disable it: ```ruby -Feature.enable(:ci_trigger_payload_into_pipeline) +Feature.disable(:ci_trigger_payload_into_pipeline) ``` -To disable it: +To enable it: ```ruby -Feature.disable(:ci_trigger_payload_into_pipeline) +Feature.enable(:ci_trigger_payload_into_pipeline) ``` ## Making use of trigger variables diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index cddcf76236a..d4b9282ac38 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -119,7 +119,7 @@ associated with it. Usually one pipeline is a merge request pipeline, and the ot is a branch pipeline. This is usually caused by the `rules` configuration, and there are several ways to -[prevent duplicate pipelines](yaml/README.md#prevent-duplicate-pipelines). +[prevent duplicate pipelines](yaml/README.md#avoid-duplicate-pipelines). #### A job is not in the pipeline @@ -141,7 +141,7 @@ be checked to make sure the jobs are added to the correct pipeline type. For example, if a merge request pipeline did not run, the jobs may have been added to a branch pipeline instead. -It's also possible that your [`workflow: rules`](yaml/README.md#workflowrules) configuration +It's also possible that your [`workflow: rules`](yaml/README.md#workflow) configuration blocked the pipeline, or allowed the wrong pipeline type. ### A job runs unexpectedly @@ -164,7 +164,7 @@ a branch to its remote repository. To illustrate the problem, suppose you've had 1. A user creates a feature branch named `example` and pushes it to a remote repository. 1. A new pipeline starts running on the `example` branch. -1. A user rebases the `example` branch on the latest `master` branch and force-pushes it to its remote repository. +1. A user rebases the `example` branch on the latest default branch and force-pushes it to its remote repository. 1. A new pipeline starts running on the `example` branch again, however, the previous pipeline (2) fails because of `fatal: reference is not a tree:` error. @@ -258,8 +258,8 @@ When you use [`rules`](yaml/README.md#rules) with a `when:` clause without an `i clause, multiple pipelines may run. Usually this occurs when you push a commit to a branch that has an open merge request associated with it. -To [prevent duplicate pipelines](yaml/README.md#prevent-duplicate-pipelines), use -[`workflow: rules`](yaml/README.md#workflowrules) or rewrite your rules to control +To [prevent duplicate pipelines](yaml/README.md#avoid-duplicate-pipelines), use +[`workflow: rules`](yaml/README.md#workflow) or rewrite your rules to control which pipelines can run. ### Console workaround if job using resource_group gets stuck diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index ee060f33d01..2aee5e364ad 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -28,7 +28,7 @@ in the pipeline detail view. Consider the following workflow: -1. Your `master` branch is rock solid, your project is using GitLab CI/CD and +1. Your default branch is rock solid, your project is using GitLab CI/CD and your pipelines indicate that there isn't anything broken. 1. Someone from your team submits a merge request, a test fails and the pipeline gets the known red icon. To investigate more, you have to go through the job @@ -44,7 +44,7 @@ First, GitLab Runner uploads all [JUnit report format XML files](https://www.ibm as [artifacts](pipelines/job_artifacts.md#artifactsreportsjunit) to GitLab. Then, when you visit a merge request, GitLab starts comparing the head and base branch's JUnit report format XML files, where: -- The base branch is the target branch (usually `master`). +- The base branch is the target branch (usually the default branch). - The head branch is the source branch (the latest pipeline in each merge request). The reports panel has a summary showing how many tests failed, how many had errors @@ -197,7 +197,7 @@ There are a few tools that can produce JUnit report format XML files in C/C++. #### GoogleTest In the following example, `gtest` is used to generate the test reports. -If there are multiple gtest executables created for different architectures (`x86`, `x64` or `arm`), +If there are multiple `gtest` executables created for different architectures (`x86`, `x64` or `arm`), you will be required to run each test providing a unique filename. The results will then be aggregated together. @@ -316,13 +316,21 @@ If JUnit report format XML files are generated and uploaded as part of a pipelin can be viewed inside the pipelines details page. The **Tests** tab on this page will display a list of test suites and cases reported from the XML file. - + You can view all the known test suites and click on each of these to see further details, including the cases that make up the suite. You can also retrieve the reports via the [GitLab API](../api/pipelines.md#get-a-pipelines-test-report). +### Unit test reports parsing errors + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263457) in GitLab 13.10. + +If parsing JUnit report XML results in an error, an indicator is shown next to the job name. Hovering over the icon shows the parser error in a tooltip. If multiple parsing errors come from [grouped jobs](jobs/index.md#group-jobs-in-a-pipeline), GitLab shows only the first error from the group. + + + ## Viewing JUnit screenshots on GitLab > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202114) in GitLab 13.0. diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index b6228e26175..5da501d4d8b 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD variables +# GitLab CI/CD variables **(FREE)** CI/CD variables are part of the environment in which [pipelines](../pipelines/index.md) and jobs run. For example, you could: @@ -161,7 +161,7 @@ You can use tools like [the AWS CLI](https://docs.aws.amazon.com/cli/latest/user and [`kubectl`](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#the-kubeconfig-environment-variable) to customize your configuration by using **File** type variables. -Previously, a common pattern was to read the value of a CI variable, save it in a file, and then +Previously, a common pattern was to read the value of a CI/CD variable, save it in a file, and then use that file in your script: ```shell diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 8d2df82a212..9d598c43ef0 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -7,142 +7,160 @@ type: reference # Predefined variables reference -For an introduction on this subject, read through the -[CI/CD variables](README.md) document. +Predefined [CI/CD variables](README.md) are available in every GitLab CI/CD pipeline. -Some of the predefined variables are available only if a minimum -version of [GitLab Runner](https://docs.gitlab.com/runner/) is used. Consult the table below to find the -version of GitLab Runner that's required. +Some variables are only available with more recent versions of [GitLab Runner](https://docs.gitlab.com/runner/). -You can add a command to your `.gitlab-ci.yml` file to -[output the values of all variables available for a job](README.md#list-all-environment-variables). +You can [output the values of all variables available for a job](README.md#list-all-environment-variables) +with a `script` command. -Kubernetes-specific variables are detailed in the -[Kubernetes deployment variables](../../user/project/clusters/index.md#deployment-variables) section. +There are also [Kubernetes-specific deployment variables](../../user/project/clusters/index.md#deployment-variables). + +| Variable | GitLab | Runner | Description | +|------------------------------------------|--------|--------|-------------| +| `CHAT_CHANNEL` | 10.6 | all | The Source chat channel that triggered the [ChatOps](../chatops/index.md) command. | +| `CHAT_INPUT` | 10.6 | all | The additional arguments passed with the [ChatOps](../chatops/index.md) command. | +| `CI` | all | 0.4 | Available for all jobs executed in CI/CD. `true` when available. | +| `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL. | +| `CI_BUILDS_DIR` | all | 11.10 | The top-level directory where builds are executed. | +| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch. Is always `0000000000000000000000000000000000000000` in pipelines for merge requests. | +| `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Available in branch pipelines, including pipelines for the default branch. Not available in merge request pipelines or tag pipelines. | +| `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit. If the title is shorter than 100 characters, the message without the first line. | +| `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | +| `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built. | +| `CI_COMMIT_REF_PROTECTED` | 11.11 | all | `true` if the job is running for a protected reference. | +| `CI_COMMIT_REF_SLUG` | 9.0 | all | `CI_COMMIT_REF_NAME` in lowercase, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | +| `CI_COMMIT_SHA` | 9.0 | all | The commit revision the project is built for. | +| `CI_COMMIT_SHORT_SHA` | 11.7 | all | The first eight characters of `CI_COMMIT_SHA`. | +| `CI_COMMIT_TAG` | 9.0 | 0.5 | The commit tag name. Available only in pipelines for tags. | +| `CI_COMMIT_TIMESTAMP` | 13.4 | all | The timestamp of the commit in the ISO 8601 format. | +| `CI_COMMIT_TITLE` | 10.8 | all | The title of the commit. The full first line of the message. | +| `CI_CONCURRENT_ID` | all | 11.10 | The unique ID of build execution in a single executor. | +| `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | The unique ID of build execution in a single executor and project. | +| `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to the CI/CD configuration file. Defaults to `.gitlab-ci.yml`. | +| `CI_DEBUG_TRACE` | all | 1.7 | `true` if [debug logging (tracing)](README.md#debug-logging) is enabled. | +| `CI_DEFAULT_BRANCH` | 12.4 | all | The name of the project's default branch. | +| `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | 13.7 | all | The image prefix for pulling images through the Dependency Proxy. | +| `CI_DEPENDENCY_PROXY_PASSWORD` | 13.7 | all | The password to pull images through the Dependency Proxy. | +| `CI_DEPENDENCY_PROXY_SERVER` | 13.7 | all | The server for logging in to the Dependency Proxy. This is equivalent to `$CI_SERVER_HOST:$CI_SERVER_PORT`. | +| `CI_DEPENDENCY_PROXY_USER` | 13.7 | all | The username to pull images through the Dependency Proxy. | +| `CI_DEPLOY_FREEZE` | 13.2 | all | Only available if the pipeline runs during a [deploy freeze window](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). `true` when available. | +| `CI_DEPLOY_PASSWORD` | 10.8 | all | The authentication password of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), if the project has one. | +| `CI_DEPLOY_USER` | 10.8 | all | The authentication username of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), if the project has one. | +| `CI_DISPOSABLE_ENVIRONMENT` | all | 10.1 | Only available if the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). `true` when available. | +| `CI_ENVIRONMENT_NAME` | 8.15 | all | The name of the environment for this job. Available if [`environment:name`](../yaml/README.md#environmentname) is set. | +| `CI_ENVIRONMENT_SLUG` | 8.15 | all | The simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, and so on. Available if [`environment:name`](../yaml/README.md#environmentname) is set. | +| `CI_ENVIRONMENT_URL` | 9.3 | all | The URL of the environment for this job. Available if [`environment:url`](../yaml/README.md#environmenturl) is set. | +| `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Only available if the pipeline's project has an open [requirement](../../user/project/requirements/index.md). `true` when available. | +| `CI_JOB_ID` | 9.0 | all | The internal ID of the job, unique across all jobs in the GitLab instance. | +| `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. | +| `CI_JOB_JWT` | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | +| `CI_JOB_MANUAL` | 8.12 | all | `true` if a job was started manually. | +| `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job. | +| `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the job's stage. | +| `CI_JOB_STATUS` | all | 13.5 | The status of the job as each runner stage is executed. Use with [`after_script`](../yaml/README.md#after_script). Can be `success`, `failed`, or `canceled`. | +| `CI_JOB_TOKEN` | 9.0 | 1.2 | A token to authenticate with [certain API endpoints](../../api/README.md#gitlab-ci-job-token) or download [dependent repositories](../../user/project/new_ci_build_permissions_model.md#dependent-repositories). The token is valid as long as the job is running. | +| `CI_JOB_URL` | 11.1 | 0.5 | The job details URL. | +| `CI_JOB_STARTED_AT` | 13.10 | all | The UTC datetime when a job started, in [ISO 8601](https://tools.ietf.org/html/rfc3339#appendix-A) format. | +| `CI_KUBERNETES_ACTIVE` | 13.0 | all | Only available if the pipeline has a Kubernetes cluster available for deployments. `true` when available. | +| `CI_NODE_INDEX` | 11.5 | all | The index of the job in the job set. Only available if the job uses [`parallel`](../yaml/README.md#parallel). | +| `CI_NODE_TOTAL` | 11.5 | all | The total number of instances of this job running in parallel. Set to `1` if the job does not use [`parallel`](../yaml/README.md#parallel). | +| `CI_OPEN_MERGE_REQUESTS` | 13.8 | all | A comma-separated list of up to four merge requests that use the current branch and project as the merge request source. Only available in branch and merge request pipelines if the branch has an associated merge request. For example, `gitlab-org/gitlab!333,gitlab-org/gitlab-foss!11`. | +| `CI_PAGES_DOMAIN` | 11.8 | all | The configured domain that hosts GitLab Pages. | +| `CI_PAGES_URL` | 11.8 | all | The URL for a GitLab Pages site. Always a subdomain of `CI_PAGES_DOMAIN`. | +| `CI_PIPELINE_ID` | 8.10 | all | The instance-level ID of the current pipeline. This ID is unique across all projects on the GitLab instance. | +| `CI_PIPELINE_IID` | 11.0 | all | The project-level IID (internal ID) of the current pipeline. This ID is unique only within the current project. | +| `CI_PIPELINE_SOURCE` | 10.0 | all | How the pipeline was triggered. Can be `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/README.md#authentication-tokens). | +| `CI_PIPELINE_TRIGGERED` | all | all | `true` if the job was [triggered](../triggers/README.md). | +| `CI_PIPELINE_URL` | 11.1 | 0.5 | The URL for the pipeline details. | +| `CI_PIPELINE_CREATED_AT` | 13.10 | all | The UTC datetime when the pipeline was created, in [ISO 8601](https://tools.ietf.org/html/rfc3339#appendix-A) format. | +| `CI_PROJECT_CONFIG_PATH` | 13.8 | all | (Deprecated) The CI configuration path for the project. [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/321334) in GitLab 13.10. [Removal planned](https://gitlab.com/gitlab-org/gitlab/-/issues/322807) for GitLab 14.0. | +| `CI_PROJECT_DIR` | all | all | The full path the repository is cloned to, and where the job runs from. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see the [Advanced GitLab Runner configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | +| `CI_PROJECT_ID` | all | all | The ID of the current project. This ID is unique across all projects on the GitLab instance. | +| `CI_PROJECT_NAME` | 8.10 | 0.5 | The name of the directory for the project. For example if the project URL is `gitlab.example.com/group-name/project-1`, `CI_PROJECT_NAME` is `project-1`. | +| `CI_PROJECT_NAMESPACE` | 8.10 | 0.5 | The project namespace (username or group name) of the job. | +| `CI_PROJECT_PATH_SLUG` | 9.3 | all | `$CI_PROJECT_PATH` in lowercase with characters that are not `a-z` or `0-9` replaced with `-`. Use in URLs and domain names. | +| `CI_PROJECT_PATH` | 8.10 | 0.5 | The project namespace with the project name included. | +| `CI_PROJECT_REPOSITORY_LANGUAGES` | 12.3 | all | A comma-separated, lowercase list of the languages used in the repository. For example `ruby,javascript,html,css`. | +| `CI_PROJECT_ROOT_NAMESPACE` | 13.2 | 0.5 | The root project namespace (username or group name) of the job. For example, if `CI_PROJECT_NAMESPACE` is `root-group/child-group/grandchild-group`, `CI_PROJECT_ROOT_NAMESPACE` is `root-group`. | +| `CI_PROJECT_TITLE` | 12.4 | all | The human-readable project name as displayed in the GitLab web interface. | +| `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address of the project. | +| `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility. Can be `internal`, `private`, or `public`. | +| `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | The address of the project's Container Registry. Only available if the Container Registry is enabled for the project. | +| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. | +| `CI_REGISTRY_USER` | 9.0 | all | The username to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. | +| `CI_REGISTRY` | 8.10 | 0.5 | The address of the GitLab Container Registry. Only available if the Container Registry is enabled for the project. This variable includes a `:port` value if one is specified in the registry configuration. | +| `CI_REPOSITORY_URL` | 9.0 | all | The URL to clone the Git repository. | +| `CI_RUNNER_DESCRIPTION` | 8.10 | 0.5 | The description of the runner. | +| `CI_RUNNER_EXECUTABLE_ARCH` | all | 10.6 | The OS/architecture of the GitLab Runner executable. Might not be the same as the environment of the executor. | +| `CI_RUNNER_ID` | 8.10 | 0.5 | The unique ID of the runner being used. | +| `CI_RUNNER_REVISION` | all | 10.6 | The revision of the runner running the job. | +| `CI_RUNNER_SHORT_TOKEN` | all | 12.3 | First eight characters of the runner's token used to authenticate new job requests. Used as the runner's unique ID. | +| `CI_RUNNER_TAGS` | 8.10 | 0.5 | A comma-separated list of the runner tags. | +| `CI_RUNNER_VERSION` | all | 10.6 | The version of the GitLab Runner running the job. | +| `CI_SERVER_HOST` | 12.1 | all | The host of the GitLab instance URL, without protocol or port. For example `gitlab.example.com`. | +| `CI_SERVER_NAME` | all | all | The name of CI/CD server that coordinates jobs. | +| `CI_SERVER_PORT` | 12.8 | all | The port of the GitLab instance URL, without host or protocol. For example `8080`. | +| `CI_SERVER_PROTOCOL` | 12.8 | all | The protocol of the GitLab instance URL, without host or port. For example `https`. | +| `CI_SERVER_REVISION` | all | all | GitLab revision that schedules jobs. | +| `CI_SERVER_URL` | 12.7 | all | The base URL of the GitLab instance, including protocol and port. For example `https://gitlab.example.com:8080`. | +| `CI_SERVER_VERSION_MAJOR` | 11.4 | all | The major version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MAJOR` is `13`. | +| `CI_SERVER_VERSION_MINOR` | 11.4 | all | The minor version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MINOR` is `6`. | +| `CI_SERVER_VERSION_PATCH` | 11.4 | all | The patch version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_PATCH` is `1`. | +| `CI_SERVER_VERSION` | all | all | The full version of the GitLab instance. | +| `CI_SERVER` | all | all | Available for all jobs executed in CI/CD. `yes` when available. | +| `CI_SHARED_ENVIRONMENT` | all | 10.1 | Only available if the job is executed in a shared environment (something that is persisted across CI/CD invocations, like the `shell` or `ssh` executor). `true` when available. | +| `GITLAB_CI` | all | all | Available for all jobs executed in CI/CD. `true` when available. | +| `GITLAB_FEATURES` | 10.6 | all | The comma-separated list of licensed features available for the GitLab instance and license. | +| `GITLAB_USER_EMAIL` | 8.12 | all | The email of the user who started the job. | +| `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the job. | +| `GITLAB_USER_LOGIN` | 10.0 | all | The username of the user who started the job. | +| `GITLAB_USER_NAME` | 10.0 | all | The name of the user who started the job. | +| `TRIGGER_PAYLOAD` | 13.9 | all | The webhook payload. Only available when a pipeline is [triggered with a webhook](../triggers/README.md#using-webhook-payload-in-the-triggered-pipeline). | + +## Predefined variables for merge request pipelines + +These variables are available when: + +- The pipelines [are merge request pipelines](../merge_request_pipelines/index.md). +- The merge request is open. + +| Variable | GitLab | Runner | Description | +|----------------------------------------|--------|--------|-------------| +| `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of usernames of assignees for the merge request. | +| `CI_MERGE_REQUEST_ID` | 11.6 | all | The instance-level ID of the merge request. This is a unique ID across all projects on GitLab. | +| `CI_MERGE_REQUEST_IID` | 11.6 | all | The project-level IID (internal ID) of the merge request. This ID is unique for the current project. | +| `CI_MERGE_REQUEST_LABELS` | 11.9 | all | Comma-separated label names of the merge request. | +| `CI_MERGE_REQUEST_MILESTONE` | 11.9 | all | The milestone title of the merge request. | +| `CI_MERGE_REQUEST_PROJECT_ID` | 11.6 | all | The ID of the project of the merge request. | +| `CI_MERGE_REQUEST_PROJECT_PATH` | 11.6 | all | The path of the project of the merge request. For example `namespace/awesome-project`. | +| `CI_MERGE_REQUEST_PROJECT_URL` | 11.6 | all | The URL of the project of the merge request. For example, `http://192.168.10.15:3000/namespace/awesome-project`. | +| `CI_MERGE_REQUEST_REF_PATH` | 11.6 | all | The ref path of the merge request. For example, `refs/merge-requests/1/head`. | +| `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME` | 11.6 | all | The source branch name of the merge request. | +| `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the source branch of the merge request. The variable is empty in merge request pipelines. The SHA is present only in [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | +| `CI_MERGE_REQUEST_SOURCE_PROJECT_ID` | 11.6 | all | The ID of the source project of the merge request. | +| `CI_MERGE_REQUEST_SOURCE_PROJECT_PATH` | 11.6 | all | The path of the source project of the merge request. | +| `CI_MERGE_REQUEST_SOURCE_PROJECT_URL` | 11.6 | all | The URL of the source project of the merge request. | +| `CI_MERGE_REQUEST_TARGET_BRANCH_NAME` | 11.6 | all | The target branch name of the merge request. | +| `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the target branch of the merge request. The variable is empty in merge request pipelines. The SHA is present only in [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | +| `CI_MERGE_REQUEST_TITLE` | 11.9 | all | The title of the merge request. | +| `CI_MERGE_REQUEST_EVENT_TYPE` | 12.3 | all | The event type of the merge request. Can be `detached`, `merged_result` or `merge_train`. | +| `CI_MERGE_REQUEST_DIFF_ID` | 13.7 | all | The version of the merge request diff. | +| `CI_MERGE_REQUEST_DIFF_BASE_SHA` | 13.7 | all | The base SHA of the merge request diff. | + +## Predefined variables for external pull request pipelines + +These variables are only available when: + +- The pipelines are [external pull requests pipelines](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests) +- The pull request is open. | Variable | GitLab | Runner | Description | |-----------------------------------------------|--------|--------|-------------| -| `CHAT_CHANNEL` | 10.6 | all | Source chat channel which triggered the [ChatOps](../chatops/index.md) command. | -| `CHAT_INPUT` | 10.6 | all | Additional arguments passed in the [ChatOps](../chatops/index.md) command. | -| `CI` | all | 0.4 | Mark that job is executed in CI environment. | -| `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL. | -| `CI_BUILDS_DIR` | all | 11.10 | Top-level directory where builds are executed. | -| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch. Is always `0000000000000000000000000000000000000000` in pipelines for merge requests. | -| `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. | -| `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | -| `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built. | -| `CI_COMMIT_REF_PROTECTED` | 11.11 | all | `true` if the job is running on a protected reference, `false` if not. | -| `CI_COMMIT_REF_SLUG` | 9.0 | all | `$CI_COMMIT_REF_NAME` in lowercase, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | -| `CI_COMMIT_SHA` | 9.0 | all | The commit revision for which project is built. | -| `CI_COMMIT_SHORT_SHA` | 11.7 | all | The first eight characters of `CI_COMMIT_SHA`. | -| `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Present in branch pipelines, including pipelines for the default branch. Not present in merge request pipelines or tag pipelines. | -| `CI_COMMIT_TAG` | 9.0 | 0.5 | The commit tag name. Present only when building tags. | -| `CI_COMMIT_TITLE` | 10.8 | all | The title of the commit - the full first line of the message. | -| `CI_COMMIT_TIMESTAMP` | 13.4 | all | The timestamp of the commit in the ISO 8601 format. | -| `CI_CONCURRENT_ID` | all | 11.10 | Unique ID of build execution in a single executor. | -| `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | Unique ID of build execution in a single executor and project. | -| `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to CI configuration file. Defaults to `.gitlab-ci.yml`. | -| `CI_DEBUG_TRACE` | all | 1.7 | Whether [debug logging (tracing)](README.md#debug-logging) is enabled. | -| `CI_DEFAULT_BRANCH` | 12.4 | all | The name of the default branch for the project. | -| `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | 13.7 | all | The image prefix for pulling images through the Dependency Proxy. | -| `CI_DEPENDENCY_PROXY_SERVER` | 13.7 | all | The server for logging in to the Dependency Proxy. This is equivalent to `$CI_SERVER_HOST:$CI_SERVER_PORT`. | -| `CI_DEPENDENCY_PROXY_PASSWORD` | 13.7 | all | The password to use to pull images through the Dependency Proxy. | -| `CI_DEPENDENCY_PROXY_USER` | 13.7 | all | The username to use to pull images through the Dependency Proxy. | -| `CI_DEPLOY_FREEZE` | 13.2 | all | Included with the value `true` if the pipeline runs during a [deploy freeze window](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). | -| `CI_DEPLOY_PASSWORD` | 10.8 | all | Authentication password of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | -| `CI_DEPLOY_USER` | 10.8 | all | Authentication username of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | -| `CI_DISPOSABLE_ENVIRONMENT` | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. | -| `CI_ENVIRONMENT_NAME` | 8.15 | all | The name of the environment for this job. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. | -| `CI_ENVIRONMENT_SLUG` | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, and so on. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. | -| `CI_ENVIRONMENT_URL` | 9.3 | all | The URL of the environment for this job. Only present if [`environment:url`](../yaml/README.md#environmenturl) is set. | -| `CI_EXTERNAL_PULL_REQUEST_IID` | 12.3 | all | Pull Request ID from GitHub if the [pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_SOURCE_REPOSITORY` | 13.3 | all | The source repository name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_TARGET_REPOSITORY` | 13.3 | all | The target repository name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_NAME` | 12.3 | all | The source branch name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the source branch of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_NAME` | 12.3 | all | The target branch name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the target branch of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Included with the value `true` only if the pipeline's project has any open [requirements](../../user/project/requirements/index.md). Not included if there are no open requirements for the pipeline's project. | -| `CI_OPEN_MERGE_REQUESTS` | 13.8 | all | Available in branch and merge request pipelines. Contains a comma-separated list of up to four merge requests that use the current branch and project as the merge request source. For example `gitlab-org/gitlab!333,gitlab-org/gitlab-foss!11`. | -| `CI_JOB_ID` | 9.0 | all | The unique ID of the current job that GitLab CI/CD uses internally. | -| `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the image running the CI job. | -| `CI_JOB_MANUAL` | 8.12 | all | The flag to indicate that job was manually started. | -| `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml`. | -| `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml`. | -| `CI_JOB_STATUS` | all | 13.5 | The state of the job as each runner stage is executed. Use with [`after_script`](../yaml/README.md#after_script) where `CI_JOB_STATUS` can be either: `success`, `failed` or `canceled`. | -| `CI_JOB_TOKEN` | 9.0 | 1.2 | Token used for authenticating with [a few API endpoints](../../api/README.md#gitlab-ci-job-token) and downloading [dependent repositories](../../user/project/new_ci_build_permissions_model.md#dependent-repositories). The token is valid as long as the job is running. | -| `CI_JOB_JWT` | 12.10 | all | RS256 JSON web token that can be used for authenticating with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | -| `CI_JOB_URL` | 11.1 | 0.5 | Job details URL. | -| `CI_KUBERNETES_ACTIVE` | 13.0 | all | Included with the value `true` only if the pipeline has a Kubernetes cluster available for deployments. Not included if no cluster is available. Can be used as an alternative to [`only:kubernetes`/`except:kubernetes`](../yaml/README.md#onlykubernetesexceptkubernetes) with [`rules:if`](../yaml/README.md#rulesif). | -| `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of username(s) of assignee(s) for the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_ID` | 11.6 | all | The instance-level ID of the merge request. Only available if [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. This is a unique ID across all projects on GitLab. | -| `CI_MERGE_REQUEST_IID` | 11.6 | all | The project-level IID (internal ID) of the merge request. Only available If [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. This ID is unique for the current project. | -| `CI_MERGE_REQUEST_LABELS` | 11.9 | all | Comma-separated label names of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_MILESTONE` | 11.9 | all | The milestone title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_PROJECT_ID` | 11.6 | all | The ID of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_PROJECT_PATH` | 11.6 | all | The path of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) (for example `namespace/awesome-project`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_PROJECT_URL` | 11.6 | all | The URL of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) (for example `http://192.168.10.15:3000/namespace/awesome-project`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_REF_PATH` | 11.6 | all | The ref path of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). (for example `refs/merge-requests/1/head`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME` | 11.6 | all | The source branch name of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the source branch of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used, the merge request is created, and the pipeline is a [merged result pipeline](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | -| `CI_MERGE_REQUEST_SOURCE_PROJECT_ID` | 11.6 | all | The ID of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_PROJECT_PATH` | 11.6 | all | The path of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_PROJECT_URL` | 11.6 | all | The URL of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_TARGET_BRANCH_NAME` | 11.6 | all | The target branch name of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the target branch of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used, the merge request is created, and the pipeline is a [merged result pipeline](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | -| `CI_MERGE_REQUEST_TITLE` | 11.9 | all | The title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_EVENT_TYPE` | 12.3 | all | The event type of the merge request, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Can be `detached`, `merged_result` or `merge_train`. | -| `CI_MERGE_REQUEST_DIFF_ID` | 13.7 | all | The version of the merge request diff, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). | -| `CI_MERGE_REQUEST_DIFF_BASE_SHA` | 13.7 | all | The base SHA of the merge request diff, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). | -| `CI_NODE_INDEX` | 11.5 | all | Index of the job in the job set. If the job is not parallelized, this variable is not set. | -| `CI_NODE_TOTAL` | 11.5 | all | Total number of instances of this job running in parallel. If the job is not parallelized, this variable is set to `1`. | -| `CI_PAGES_DOMAIN` | 11.8 | all | The configured domain that hosts GitLab Pages. | -| `CI_PAGES_URL` | 11.8 | all | URL to GitLab Pages-built pages. Always belongs to a subdomain of `CI_PAGES_DOMAIN`. | -| `CI_PIPELINE_ID` | 8.10 | all | The instance-level ID of the current pipeline. This is a unique ID across all projects on GitLab. | -| `CI_PIPELINE_IID` | 11.0 | all | The project-level IID (internal ID) of the current pipeline. This ID is unique for the current project. | -| `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/README.md#authentication-tokens). For pipelines created before GitLab 9.5, this is displayed as `unknown`. | -| `CI_PIPELINE_TRIGGERED` | all | all | The flag to indicate that job was [triggered](../triggers/README.md). | -| `CI_PIPELINE_URL` | 11.1 | 0.5 | Pipeline details URL. | -| `CI_PROJECT_CONFIG_PATH` | 13.8 | all | The CI configuration path for the project. | -| `CI_PROJECT_DIR` | all | all | The full path where the repository is cloned and where the job is run. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see [Advanced configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) for GitLab Runner. | -| `CI_PROJECT_ID` | all | all | The unique ID of the current project that GitLab CI/CD uses internally. | -| `CI_PROJECT_NAME` | 8.10 | 0.5 | The name of the directory for the project that is being built. For example, if the project URL is `gitlab.example.com/group-name/project-1`, the `CI_PROJECT_NAME` would be `project-1`. | -| `CI_PROJECT_NAMESPACE` | 8.10 | 0.5 | The project namespace (username or group name) that is being built. | -| `CI_PROJECT_ROOT_NAMESPACE` | 13.2 | 0.5 | The **root** project namespace (username or group name) that is being built. For example, if `CI_PROJECT_NAMESPACE` is `root-group/child-group/grandchild-group`, `CI_PROJECT_ROOT_NAMESPACE` would be `root-group`. | -| `CI_PROJECT_PATH` | 8.10 | 0.5 | The namespace with project name. | -| `CI_PROJECT_PATH_SLUG` | 9.3 | all | `$CI_PROJECT_PATH` in lowercase and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | -| `CI_PROJECT_REPOSITORY_LANGUAGES` | 12.3 | all | Comma-separated, lowercase list of the languages used in the repository (for example `ruby,javascript,html,css`). | -| `CI_PROJECT_TITLE` | 12.4 | all | The human-readable project name as displayed in the GitLab web interface. | -| `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address to access project. | -| `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility (internal, private, public). | -| `CI_REGISTRY` | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of the GitLab Container Registry. This variable includes a `:port` value if one has been specified in the registry configuration. | -| `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project. | -| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to use to push containers to the GitLab Container Registry, for the current project. | -| `CI_REGISTRY_USER` | 9.0 | all | The username to use to push containers to the GitLab Container Registry, for the current project. | -| `CI_REPOSITORY_URL` | 9.0 | all | The URL to clone the Git repository. | -| `CI_RUNNER_DESCRIPTION` | 8.10 | 0.5 | The description of the runner as saved in GitLab. | -| `CI_RUNNER_EXECUTABLE_ARCH` | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor). | -| `CI_RUNNER_ID` | 8.10 | 0.5 | The unique ID of runner being used. | -| `CI_RUNNER_REVISION` | all | 10.6 | GitLab Runner revision that is executing the current job. | -| `CI_RUNNER_SHORT_TOKEN` | all | 12.3 | First eight characters of the runner's token used to authenticate new job requests. Used as the runner's unique ID. | -| `CI_RUNNER_TAGS` | 8.10 | 0.5 | The defined runner tags. | -| `CI_RUNNER_VERSION` | all | 10.6 | GitLab Runner version that is executing the current job. | -| `CI_SERVER` | all | all | Mark that job is executed in CI environment. | -| `CI_SERVER_URL` | 12.7 | all | The base URL of the GitLab instance, including protocol and port (like `https://gitlab.example.com:8080`). | -| `CI_SERVER_HOST` | 12.1 | all | Host component of the GitLab instance URL, without protocol and port (like `gitlab.example.com`). | -| `CI_SERVER_PORT` | 12.8 | all | Port component of the GitLab instance URL, without host and protocol (like `3000`). | -| `CI_SERVER_PROTOCOL` | 12.8 | all | Protocol component of the GitLab instance URL, without host and port (like `https`). | -| `CI_SERVER_NAME` | all | all | The name of CI server that is used to coordinate jobs. | -| `CI_SERVER_REVISION` | all | all | GitLab revision that is used to schedule jobs. | -| `CI_SERVER_VERSION` | all | all | GitLab version that is used to schedule jobs. | -| `CI_SERVER_VERSION_MAJOR` | 11.4 | all | GitLab version major component. | -| `CI_SERVER_VERSION_MINOR` | 11.4 | all | GitLab version minor component. | -| `CI_SERVER_VERSION_PATCH` | 11.4 | all | GitLab version patch component. | -| `CI_SHARED_ENVIRONMENT` | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | -| `GITLAB_CI` | all | all | Mark that job is executed in GitLab CI/CD environment. | -| `GITLAB_FEATURES` | 10.6 | all | The comma separated list of licensed features available for your instance and plan. | -| `GITLAB_USER_EMAIL` | 8.12 | all | The email of the user who started the job. | -| `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the job. | -| `GITLAB_USER_LOGIN` | 10.0 | all | The login username of the user who started the job. | -| `GITLAB_USER_NAME` | 10.0 | all | The real name of the user who started the job. | -| `TRIGGER_PAYLOAD` | 13.9 | all | This variable is available when a pipeline is [triggered with a webhook](../triggers/README.md#using-webhook-payload-in-the-triggered-pipeline) | +| `CI_EXTERNAL_PULL_REQUEST_IID` | 12.3 | all | Pull request ID from GitHub. | +| `CI_EXTERNAL_PULL_REQUEST_SOURCE_REPOSITORY` | 13.3 | all | The source repository name of the pull request. | +| `CI_EXTERNAL_PULL_REQUEST_TARGET_REPOSITORY` | 13.3 | all | The target repository name of the pull request. | +| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_NAME` | 12.3 | all | The source branch name of the pull request. | +| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the source branch of the pull request. | +| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_NAME` | 12.3 | all | The target branch name of the pull request. | +| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the target branch of the pull request. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index d057fe9c4cc..16c02b3482b 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -109,6 +109,7 @@ The following variables are known as "persisted": - `CI_PIPELINE_ID` - `CI_JOB_ID` - `CI_JOB_TOKEN` +- `CI_JOB_STARTED_AT` - `CI_BUILD_ID` - `CI_BUILD_TOKEN` - `CI_REGISTRY_USER` diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 2abfbd3a5b4..9329748e396 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -7,7 +7,7 @@ type: reference <!-- markdownlint-disable MD044 --> <!-- vale gitlab.Spelling = NO --> -# Keyword reference for the .gitlab-ci.yml file +# Keyword reference for the .gitlab-ci.yml file **(FREE)** <!-- vale gitlab.Spelling = YES --> <!-- markdownlint-enable MD044 --> @@ -26,41 +26,43 @@ A job is defined as a list of keywords that define the job's behavior. The keywords available for jobs are: -| Keyword | Description | -|:---------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [`script`](#script) | Shell script that is executed by a runner. | -| [`after_script`](#after_script) | Override a set of commands that are executed after job. | -| [`allow_failure`](#allow_failure) | Allow job to fail. A failed job does not cause the pipeline to fail. | -| [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, and `artifacts:reports`. | -| [`before_script`](#before_script) | Override a set of commands that are executed before job. | -| [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, `cache:when`, and `cache:policy`. | -| [`coverage`](#coverage) | Code coverage settings for a given job. | -| [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. | -| [`environment`](#environment) | Name of an environment to which the job deploys. Also available: `environment:name`, `environment:url`, `environment:on_stop`, `environment:auto_stop_in`, and `environment:action`. | -| [`except`](#onlyexcept-basic) | Limit when jobs are not created. Also available: [`except:refs`, `except:kubernetes`, `except:variables`, and `except:changes`](#onlyexcept-advanced). | -| [`extends`](#extends) | Configuration entries that this job inherits from. | -| [`image`](#image) | Use Docker images. Also available: `image:name` and `image:entrypoint`. | -| [`include`](#include) | Include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. | -| [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | -| [`only`](#onlyexcept-basic) | Limit when jobs are created. Also available: [`only:refs`, `only:kubernetes`, `only:variables`, and `only:changes`](#onlyexcept-advanced). | -| [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | -| [`parallel`](#parallel) | How many instances of a job should be run in parallel. | -| [`release`](#release) | Instructs the runner to generate a [Release](../../user/project/releases/index.md) object. | -| [`resource_group`](#resource_group) | Limit job concurrency. | -| [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. | -| [`rules`](#rules) | List of conditions to evaluate and determine selected attributes of a job, and whether or not it's created. | -| [`services`](#services) | Use Docker services images. Also available: `services:name`, `services:alias`, `services:entrypoint`, and `services:command`. | -| [`stage`](#stage) | Defines a job stage (default: `test`). | -| [`tags`](#tags) | List of tags that are used to select a runner. | -| [`timeout`](#timeout) | Define a custom job-level timeout that takes precedence over the project-wide setting. | -| [`trigger`](#trigger) | Defines a downstream pipeline trigger. | -| [`variables`](#variables) | Define job variables on a job level. | -| [`when`](#when) | When to run job. Also available: `when:manual` and `when:delayed`. | +| Keyword | Description | +| :-----------------------------------|:------------| +| [`after_script`](#after_script) | Override a set of commands that are executed after job. | +| [`allow_failure`](#allow_failure) | Allow job to fail. A failed job does not cause the pipeline to fail. | +| [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. | +| [`before_script`](#before_script) | Override a set of commands that are executed before job. | +| [`cache`](#cache) | List of files that should be cached between subsequent runs. | +| [`coverage`](#coverage) | Code coverage settings for a given job. | +| [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. | +| [`environment`](#environment) | Name of an environment to which the job deploys. | +| [`except`](#onlyexcept-basic) | Limit when jobs are not created. | +| [`extends`](#extends) | Configuration entries that this job inherits from. | +| [`image`](#image) | Use Docker images. | +| [`include`](#include) | Include external YAML files. | +| [`inherit`](#inherit) | Select which global defaults all jobs inherit. | +| [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | +| [`needs`](#needs) | Execute jobs earlier than the stage ordering. | +| [`only`](#onlyexcept-basic) | Limit when jobs are created. | +| [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | +| [`parallel`](#parallel) | How many instances of a job should be run in parallel. | +| [`release`](#release) | Instructs the runner to generate a [release](../../user/project/releases/index.md) object. | +| [`resource_group`](#resource_group) | Limit job concurrency. | +| [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. | +| [`rules`](#rules) | List of conditions to evaluate and determine selected attributes of a job, and whether or not it's created. | +| [`script`](#script) | Shell script that is executed by a runner. | +| [`secrets`](#secrets) | The CI/CD secrets the job needs. | +| [`services`](#services) | Use Docker services images. | +| [`stage`](#stage) | Defines a job stage. | +| [`tags`](#tags) | List of tags that are used to select a runner. | +| [`timeout`](#timeout) | Define a custom job-level timeout that takes precedence over the project-wide setting. | +| [`trigger`](#trigger) | Defines a downstream pipeline trigger. | +| [`variables`](#variables) | Define job variables on a job level. | +| [`when`](#when) | When to run job. | ### Unavailable names for jobs -Each job must have a unique name, but there are a few reserved `keywords` that -can't be used as job names: +You can't use these keywords as job names: - `image` - `services` @@ -72,38 +74,27 @@ can't be used as job names: - `cache` - `include` -### Reserved keywords +### Custom default keyword values -If you get a validation error when you use specific values (for example, `true` or `false`), try to: +You can set global defaults for some keywords. Jobs that do not define one or more +of the listed keywords use the value defined in the `default:` section. -- Quote them. -- Change them to a different form. For example, `/bin/true`. +These job keywords can be defined inside a `default:` section: -## Global keywords - -Some keywords are defined at a global level and affect all jobs in the pipeline. - -### Global defaults - -Some keywords can be set globally as the default for all jobs with the -`default:` keyword. Default keywords can then be overridden by job-specific -configuration. - -The following job keywords can be defined inside a `default:` block: - -- [`image`](#image) -- [`services`](#services) -- [`before_script`](#before_script) - [`after_script`](#after_script) -- [`tags`](#tags) -- [`cache`](#cache) - [`artifacts`](#artifacts) +- [`before_script`](#before_script) +- [`cache`](#cache) +- [`image`](#image) +- [`interruptible`](#interruptible) - [`retry`](#retry) +- [`services`](#services) +- [`tags`](#tags) - [`timeout`](#timeout) -- [`interruptible`](#interruptible) -In the following example, the `ruby:2.5` image is set as the default for all -jobs except the `rspec 2.6` job, which uses the `ruby:2.6` image: +The following example sets the `ruby:2.5` image as the default for all jobs in the pipeline. +The `rspec 2.6` job does not use the default, because it overrides the default with +a job-specific `image:` section: ```yaml default: @@ -117,87 +108,16 @@ rspec 2.6: script: bundle exec rspec ``` -#### `inherit` - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207484) in GitLab 12.9. - -You can disable inheritance of globally defined defaults -and variables with the `inherit:` keyword. - -To enable or disable the inheritance of all `default:` or `variables:` keywords, use: - -- `default: true` or `default: false` -- `variables: true` or `variables: false` - -To inherit only a subset of `default:` keywords or `variables:`, specify what -you wish to inherit. Anything not listed is **not** inherited. Use -one of the following formats: - -```yaml -inherit: - default: [keyword1, keyword2] - variables: [VARIABLE1, VARIABLE2] -``` - -Or: - -```yaml -inherit: - default: - - keyword1 - - keyword2 - variables: - - VARIABLE1 - - VARIABLE2 -``` - -In the example below: - -- `rubocop`: - - inherits: Nothing. -- `rspec`: - - inherits: the default `image` and the `WEBHOOK_URL` variable. - - does **not** inherit: the default `before_script` and the `DOMAIN` variable. -- `capybara`: - - inherits: the default `before_script` and `image`. - - does **not** inherit: the `DOMAIN` and `WEBHOOK_URL` variables. -- `karma`: - - inherits: the default `image` and `before_script`, and the `DOMAIN` variable. - - does **not** inherit: `WEBHOOK_URL` variable. - -```yaml -default: - image: 'ruby:2.4' - before_script: - - echo Hello World - -variables: - DOMAIN: example.com - WEBHOOK_URL: https://my-webhook.example.com - -rubocop: - inherit: - default: false - variables: false - script: bundle exec rubocop - -rspec: - inherit: - default: [image] - variables: [WEBHOOK_URL] - script: bundle exec rspec +## Global keywords -capybara: - inherit: - variables: false - script: bundle exec capybara +Some keywords are not defined in a job. These keywords control pipeline behavior +or import additional pipeline configuration: -karma: - inherit: - default: true - variables: [DOMAIN] - script: karma -``` +| Keyword | Description | +|-------------------------|:------------| +| [`stages`](#stages) | The names and order of the pipeline stages. | +| [`workflow`](#workflow) | Control what types of pipeline run. | +| [`include`](#include) | Import configuration from other YAML files. | ### `stages` @@ -235,13 +155,13 @@ If a job does not specify a [`stage`](#stage), the job is assigned the `test` st To make a job start earlier and ignore the stage order, use the [`needs`](#needs) keyword. -### `workflow:rules` +### `workflow` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29654) in GitLab 12.5 -The top-level `workflow:` keyword determines whether or not a pipeline is created. -It accepts a single `rules:` keyword that is similar to [`rules:` defined in jobs](#rules). -Use it to define what can trigger a new pipeline. +Use `workflow:` to determine whether or not a pipeline is created. +Define this keyword at the top level, with a single `rules:` keyword that +is similar to [`rules:` defined in jobs](#rules). You can use the [`workflow:rules` templates](#workflowrules-templates) to import a preconfigured `workflow: rules` entry. @@ -266,15 +186,15 @@ Some example `if` clauses for `workflow: rules`: See the [common `if` clauses for `rules`](#common-if-clauses-for-rules) for more examples. -For example, in the following configuration, pipelines run for all `push` events (changes to -branches and new tags). Pipelines for push events with `-wip` in the commit message +In the following example, pipelines run for all `push` events (changes to +branches and new tags). Pipelines for push events with `-draft` in the commit message don't run, because they are set to `when: never`. Pipelines for schedules or merge requests don't run either, because no rules evaluate to true for them: ```yaml workflow: rules: - - if: $CI_COMMIT_MESSAGE =~ /-wip$/ + - if: $CI_COMMIT_MESSAGE =~ /-draft$/ when: never - if: '$CI_PIPELINE_SOURCE == "push"' ``` @@ -300,13 +220,13 @@ The final `when: always` rule runs all other pipeline types, **including** merge request pipelines. If your rules match both branch pipelines and merge request pipelines, -[duplicate pipelines](#prevent-duplicate-pipelines) can occur. +[duplicate pipelines](#avoid-duplicate-pipelines) can occur. #### `workflow:rules` templates > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217732) in GitLab 13.0. -We provide templates that set up `workflow: rules` +GitLab provides templates that set up `workflow: rules` for common scenarios. These templates help prevent duplicate pipelines. The [`Branch-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/Branch-Pipelines.gitlab-ci.yml) @@ -314,12 +234,12 @@ makes your pipelines run for branches and tags. Branch pipeline status is displayed in merge requests that use the branch as a source. However, this pipeline type does not support any features offered by -[Merge Request Pipelines](../merge_request_pipelines/), like -[Pipelines for Merge Results](../merge_request_pipelines/#pipelines-for-merged-results) -or [Merge Trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/). -Use this template if you are intentionally avoiding those features. +[merge request pipelines](../merge_request_pipelines/), like +[pipelines for merge results](../merge_request_pipelines/#pipelines-for-merged-results) +or [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/). +This template intentionally avoids those features. -It is [included](#include) as follows: +To [include](#include) it: ```yaml include: @@ -327,25 +247,74 @@ include: ``` The [`MergeRequest-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml) -makes your pipelines run for the default branch (usually `master`), tags, and +makes your pipelines run for the default branch, tags, and all types of merge request pipelines. Use this template if you use any of the -the [Pipelines for Merge Requests features](../merge_request_pipelines/), as mentioned -above. +the [pipelines for merge requests features](../merge_request_pipelines/). -It is [included](#include) as follows: +To [include](#include) it: ```yaml include: - template: 'Workflows/MergeRequest-Pipelines.gitlab-ci.yml' ``` +#### Switch between branch pipelines and merge request pipelines + +> [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/201845) GitLab 13.8. + +To make the pipeline switch from branch pipelines to merge request pipelines after +a merge request is created, add a `workflow: rules` section to your `.gitlab-ci.yml` file. + +If you use both pipeline types at the same time, [duplicate pipelines](#avoid-duplicate-pipelines) +might run at the same time. To prevent duplicate pipelines, use the +[`CI_OPEN_MERGE_REQUESTS` variable](../variables/predefined_variables.md). + +The following example is for a project that runs branch and merge request pipelines only, +but does not run pipelines for any other case. It runs: + +- Branch pipelines when a merge request is not open for the branch. +- Merge request pipelines when a merge request is open for the branch. + +```yaml +workflow: + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS' + when: never + - if: '$CI_COMMIT_BRANCH' +``` + +If the pipeline is triggered by: + +- A merge request, run a merge request pipeline. For example, a merge request pipeline + can be triggered by a push to a branch with an associated open merge request. +- A change to a branch, but a merge request is open for that branch, do not run a branch pipeline. +- A change to a branch, but without any open merge requests, run a branch pipeline. + +You can also add a rule to an existing `workflow` section to switch from branch pipelines +to merge request pipelines when a merge request is created. + +Add this rule to the top of the `workflow` section, followed by the other rules that +were already present: + +```yaml +workflow: + rules: + - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push" + when: never + - ... # Previously defined workflow rules here +``` + +[Triggered pipelines](../triggers/README.md) that run on a branch have a `$CI_COMMIT_BRANCH` +set and could be blocked by a similar rule. Triggered pipelines have a pipeline source +of `trigger` or `pipeline`, so `&& $CI_PIPELINE_SOURCE == "push"` ensures the rule +does not block triggered pipelines. + ### `include` -> - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. -> - Available for Starter, Premium, and Ultimate in GitLab 10.6 and later. -> - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Free in 11.4. +> [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Free in 11.4. -Use the `include` keyword to include external YAML files in your CI/CD configuration. +Use `include` to include external YAML files in your CI/CD configuration. You can break down one long `gitlab-ci.yml` file into multiple files to increase readability, or reduce duplication of the same configuration in multiple places. @@ -355,8 +324,8 @@ You can also store template files in a central repository and `include` them in otherwise the external file is not included. You can't use [YAML anchors](#anchors) across different YAML files sourced by `include`. -You can only refer to anchors in the same file. Instead of YAML anchors, you can -use the [`extends` keyword](#extends). +You can only refer to anchors in the same file. To reuse configuration from different +YAML files, use [`!reference` tags](#reference-tags) or the [`extends` keyword](#extends). `include` supports the following inclusion methods: @@ -367,11 +336,11 @@ use the [`extends` keyword](#extends). | [`remote`](#includeremote) | Include a file from a remote URL. Must be publicly accessible. | | [`template`](#includetemplate) | Include templates that are provided by GitLab. | -The `.gitlab-ci.yml` file configuration included by all methods is evaluated when the pipeline is created. -The configuration is a snapshot in time and persisted in the database. Any changes to -the referenced `.gitlab-ci.yml` file configuration is not reflected in GitLab until the next pipeline is created. +When the pipeline starts, the `.gitlab-ci.yml` file configuration included by all methods is evaluated. +The configuration is a snapshot in time and persists in the database. GitLab does not reflect any changes to +the referenced `.gitlab-ci.yml` file configuration until the next pipeline starts. -The files defined by `include` are: +The `include` files are: - Deep merged with those in the `.gitlab-ci.yml` file. - Always evaluated first and merged with the content of the `.gitlab-ci.yml` file, @@ -395,13 +364,13 @@ include: file: '.compliance-gitlab-ci.yml' ``` -For an example of how you can include these predefined variables, and their impact on CI jobs, -see the following [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos). +For an example of how you can include these predefined variables, and the variables' impact on CI/CD jobs, +see this [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos). #### `include:local` -`include:local` includes a file that is in the same repository as the `.gitlab-ci.yml` file. -It's referenced with full paths relative to the root directory (`/`). +Use `include:local` to include a file that is in the same repository as the `.gitlab-ci.yml` file. +Use a full path relative to the root directory (`/`). If you use `include:local`, make sure that both the `.gitlab-ci.yml` file and the local file are on the same branch. @@ -418,7 +387,7 @@ include: - local: '/templates/.gitlab-ci-template.yml' ``` -This can be defined as a short local include: +You can also use shorter syntax to define the path: ```yaml include: '.gitlab-ci-production.yml' @@ -432,8 +401,9 @@ Use local includes instead of symbolic links. To include files from another private project on the same GitLab instance, use `include:file`. You can use `include:file` in combination with `include:project` only. +Use a full path, relative to the root directory (`/`). -The included file is referenced with a full path, relative to the root directory (`/`). For example: +For example: ```yaml include: @@ -441,7 +411,7 @@ include: file: '/templates/.gitlab-ci-template.yml' ``` -You can also specify a `ref`. If not specified, it defaults to the `HEAD` of the project: +You can also specify a `ref`. If you do not specify a value, the ref defaults to the `HEAD` of the project: ```yaml include: @@ -530,15 +500,15 @@ to resolve all files is 30 seconds. #### Additional `includes` examples -There is a list of [additional `includes` examples](includes.md) available. +View [additional `includes` examples](includes.md). ## Keyword details -The following are detailed explanations for keywords used to configure CI/CD pipelines. +The following topics explain how to use keywords to configure CI/CD pipelines. ### `image` -Used to specify [a Docker image](../docker/using_docker_images.md#what-is-an-image) to use for the job. +Use `image` to specify [a Docker image](../docker/using_docker_images.md#what-is-an-image) to use for the job. For: @@ -559,13 +529,13 @@ For more information, see [Available settings for `image`](../docker/using_docke #### `services` -Used to specify a [service Docker image](../docker/using_docker_images.md#what-is-a-service), linked to a base image specified in [`image`](#image). +Use `services` to specify a [service Docker image](../docker/using_docker_images.md#what-is-a-service), linked to a base image specified in [`image`](#image). For: - Usage examples, see [Define `image` and `services` from `.gitlab-ci.yml`](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml). - Detailed usage information, refer to [Docker integration](../docker/index.md) documentation. -- For example services, see [GitLab CI/CD Services](../services/index.md). +- Example services, see [GitLab CI/CD Services](../services/index.md). ##### `services:name` @@ -593,8 +563,11 @@ For more information, see [Available settings for `services`](../docker/using_do ### `script` -`script` is the only required keyword that a job needs. It's a shell script -that is executed by the runner. For example: +Use `script` to specify a shell script for the runner to execute. + +All jobs except [trigger jobs](#trigger) require a `script` keyword. + +For example: ```yaml job: @@ -603,7 +576,7 @@ job: You can use [YAML anchors with `script`](#yaml-anchors-for-scripts). -This keyword can also contain several commands in an array: +The `script` keyword can also contain several commands in an array: ```yaml job: @@ -637,7 +610,7 @@ job: You can verify the syntax is valid with the [CI Lint](../lint.md) tool. -Be careful when using these special characters as well: +Be careful when using these characters as well: - `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. @@ -657,10 +630,10 @@ job: Use `before_script` to define an array of commands that should run before each job, but after [artifacts](#artifacts) are restored. -Scripts specified in `before_script` are concatenated with any scripts specified -in the main [`script`](#script), and executed together in a single shell. +Scripts you specify in `before_script` are concatenated with any scripts you specify +in the main [`script`](#script). The combine scripts execute together in a single shell. -It's possible to overwrite a globally defined `before_script` if you define it in a job: +You can overwrite a globally-defined `before_script` if you define it in a job: ```yaml default: @@ -685,11 +658,11 @@ You can use [YAML anchors with `before_script`](#yaml-anchors-for-scripts). Use `after_script` to define an array of commands that run after each job, including failed jobs. -If a job times out or is cancelled, the `after_script` commands are not executed. -Support for executing `after_script` commands for timed-out or cancelled jobs -[is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/15603). +If a job times out or is cancelled, the `after_script` commands do not execute. +An [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/15603) exists to support +executing `after_script` commands for timed-out or cancelled jobs. -Scripts specified in `after_script` are executed in a new shell, separate from any +Scripts you specify in `after_script` execute in a new shell, separate from any `before_script` or `script` scripts. As a result, they: - Have a current working directory set back to the default. @@ -722,7 +695,7 @@ You can use [YAML anchors with `after_script`](#yaml-anchors-for-scripts). #### Script syntax -You can use special syntax in [`script`](README.md#script) sections to: +You can use syntax in [`script`](README.md#script) sections to: - [Split long commands](script.md#split-long-commands) into multiline commands. - [Use color codes](script.md#add-color-codes-to-script-output) to make job logs easier to review. @@ -731,9 +704,19 @@ You can use special syntax in [`script`](README.md#script) sections to: ### `stage` -`stage` is defined per-job and relies on [`stages`](#stages), which is defined -globally. Use `stage` to define which stage a job runs in, and jobs of the same -`stage` are executed in parallel (subject to [certain conditions](#use-your-own-runners)). For example: +Use `stage` to define which stage a job runs in. Jobs in the same +`stage` can execute in parallel (subject to [certain conditions](#use-your-own-runners)). + +Jobs without a `stage` entry use the `test` stage by default. If you do not define +[`stages`](#stages) in the pipeline, you can use the 5 default stages, which execute in +this order: + +- [`.pre`](#pre-and-post) +- `build` +- `test` +- `deploy` +- [`.post`](#pre-and-post) +For example: ```yaml stages: @@ -779,45 +762,39 @@ is greater than `1`. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31441) in GitLab 12.4. -The following stages are available to every pipeline: +Use `pre` and `post` for jobs that need to run first or last in a pipeline. -- `.pre`, which is guaranteed to always be the first stage in a pipeline. -- `.post`, which is guaranteed to always be the last stage in a pipeline. +- `.pre` is guaranteed to always be the first stage in a pipeline. +- `.post` is guaranteed to always be the last stage in a pipeline. User-defined stages are executed after `.pre` and before `.post`. -A pipeline is not created if all jobs are in `.pre` or `.post` stages. +You must have a job in at least one stage other than `.pre` or `.post`. -The order of `.pre` and `.post` can't be changed, even if defined out of order in the `.gitlab-ci.yml` file. -For example, the following are equivalent configuration: +You can't change the order of `.pre` and `.post`, even if you define them out of order in the `.gitlab-ci.yml` file. +For example, the following configurations are equivalent: -- Configured in order: - - ```yaml - stages: - - .pre - - a - - b - - .post - ``` - -- Configured out of order: - - ```yaml - stages: - - a - - .pre - - b - - .post - ``` +```yaml +stages: + - .pre + - a + - b + - .post +``` -- Not explicitly configured: +```yaml +stages: + - a + - .pre + - b + - .post +``` - ```yaml - stages: - - a - - b - ``` +```yaml +stages: + - a + - b +``` ### `extends` @@ -827,7 +804,7 @@ Use `extends` to reuse configuration sections. It's an alternative to [YAML anch and is a little more flexible and readable. You can use `extends` to reuse configuration from [included configuration files](#use-extends-and-include-together). -In this example, the `rspec` job uses the configuration from the `.tests` template job. +In the following example, the `rspec` job uses the configuration from the `.tests` template job. GitLab: - Performs a reverse deep merge based on the keys. @@ -898,8 +875,8 @@ In GitLab 12.0 and later, it's also possible to use multiple parents for #### Merge details -`extends` is able to merge hashes but not arrays. -The algorithm used for merge is "closest scope wins", so +You can use `extends` to merge hashes but not arrays. +The algorithm used for merge is "closest scope wins," so keys from the last member always override anything defined on other levels. For example: @@ -951,7 +928,7 @@ rspec: - rake rspec ``` -Note that in the example above: +In this example: - The `variables` sections merge, but `URL: "http://docker-url.internal"` overwrites `URL: "http://my-url.internal"`. - `tags: ['docker']` overwrites `tags: ['production']`. @@ -963,8 +940,8 @@ Note that in the example above: To reuse configuration from different configuration files, combine `extends` and [`include`](#include). -In this example, a `script` is defined in the `included.yml` file. -Then, in the `.gitlab-ci.yml` file, you use `extends` to refer +In the following example, a `script` is defined in the `included.yml` file. +Then, in the `.gitlab-ci.yml` file, `extends` refers to the contents of the `script`: - `included.yml`: @@ -989,11 +966,11 @@ to the contents of the `script`: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3. -Use the `rules` keyword to include or exclude jobs in pipelines. +Use `rules` to include or exclude jobs in pipelines. -Rules are evaluated *in order* until the first match. When matched, the job +Rules are evaluated *in order* until the first match. When a match is found, the job is either included or excluded from the pipeline, depending on the configuration. -If included, the job also has [certain attributes](#rules-attributes) +The job can also have [certain attributes](#rules-attributes) added to it. `rules` replaces [`only/except`](#onlyexcept-basic) and they can't be used together @@ -1052,7 +1029,7 @@ The job is not added to the pipeline: `when: always`. - If a rule matches, and has `when: never` as the attribute. -This example uses `if` to strictly limit when jobs run: +The following example uses `if` to strictly limit when jobs run: ```yaml job: @@ -1100,15 +1077,14 @@ WARNING: If you use a `when:` clause as the final rule (not including `when: never`), two simultaneous pipelines may start. Both push pipelines and merge request pipelines can be triggered by the same event (a push to the source branch for an open merge request). -See how to [prevent duplicate pipelines](#prevent-duplicate-pipelines) +See how to [prevent duplicate pipelines](#avoid-duplicate-pipelines) for more details. -#### Prevent duplicate pipelines +#### Avoid duplicate pipelines -Jobs defined with `rules` can trigger multiple pipelines with the same action. You -don't have to explicitly configure rules for each type of pipeline to trigger them -accidentally. Rules that are too broad could cause simultaneous pipelines of a different -type to run unexpectedly. +If a job uses `rules`, a single action, like pushing a commit to a branch, can trigger +multiple pipelines. You don't have to explicitly configure rules for multiple types +of pipeline to trigger them accidentally. Some configurations that have the potential to cause duplicate pipelines cause a [pipeline warning](../troubleshooting.md#pipeline-warnings) to be displayed. @@ -1130,12 +1106,10 @@ other pipelines, including **both** push (branch) and merge request pipelines. W this configuration, every push to an open merge request's source branch causes duplicated pipelines. -There are multiple ways to avoid duplicate pipelines: - -- Use [`workflow: rules`](#workflowrules) to specify which types of pipelines - can run. To eliminate duplicate pipelines, use merge request pipelines only - or push (branch) pipelines only. +To avoid duplicate pipelines, you can: +- Use [`workflow`](#workflow) to specify which types of pipelines + can run. - Rewrite the rules to run the job only in very specific cases, and avoid a final `when:` rule: @@ -1146,7 +1120,7 @@ There are multiple ways to avoid duplicate pipelines: - if: '$CUSTOM_VARIABLE == "true" && $CI_PIPELINE_SOURCE == "merge_request_event"' ``` -You can prevent duplicate pipelines by changing the job rules to avoid either push (branch) +You can also avoid duplicate pipelines by changing the job rules to avoid either push (branch) pipelines or merge request pipelines. However, if you use a `- when: always` rule without `workflow: rules`, GitLab still displays a [pipeline warning](../troubleshooting.md#pipeline-warnings). @@ -1162,7 +1136,8 @@ job: - when: always ``` -Do not include both push and merge request pipelines in the same job: +You should not include both push and merge request pipelines in the same job without +[`workflow:rules` that prevent duplicate pipelines](#switch-between-branch-pipelines-and-merge-request-pipelines): ```yaml job: @@ -1192,22 +1167,13 @@ runs the other job (`job-with-rules`). Jobs with no rules default to [`except: merge_requests`](#onlyexcept-basic), so `job-with-no-rules` runs in all cases except merge requests. -It is not possible to define rules based on whether or not a branch has an open -merge request associated with it. You can't configure a job to be included in: - -- Only branch pipelines when the branch doesn't have a merge request associated with it. -- Only merge request pipelines when the branch has a merge request associated with it. - -See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/201845) for more details. - #### `rules:if` -`rules:if` clauses determine whether or not jobs are added to a pipeline by evaluating -an `if` statement. If the `if` statement is true, the job is either included -or excluded from a pipeline. In plain English, `if` rules can be interpreted as one of: +Use `rules:if` clauses to specify when to add a job to a pipeline: -- "If this rule evaluates to true, add the job" (default). -- "If this rule evaluates to true, do not add the job" (by adding `when: never`). +- If an `if` statement is true, add the job to the pipeline. +- If an `if` statement is true, but it's combined with `when: never`, do not add the job to the pipeline. +- If no `if` statements are true, do not add the job to the pipeline. `rules:if` differs slightly from `only:variables` by accepting only a single expression string per rule, rather than an array of them. Any set of expressions to be @@ -1265,7 +1231,9 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. | | `webide` | For pipelines created by using the [WebIDE](../../user/project/web_ide/index.md). | -For example: +The following example runs the job as a manual job in scheduled pipelines or in push +pipelines (to branches or tags), with `when: on_success` (default). It does not +add the job to any other pipeline type. ```yaml job: @@ -1277,11 +1245,8 @@ job: - if: '$CI_PIPELINE_SOURCE == "push"' ``` -This example runs the job as a manual job in scheduled pipelines or in push -pipelines (to branches or tags), with `when: on_success` (default). It does not -add the job to any other pipeline type. - -Another example: +The following example runs the job as a `when: on_success` job in [merge request pipelines](../merge_request_pipelines/index.md) +and scheduled pipelines. It does not run in any other pipeline type. ```yaml job: @@ -1291,16 +1256,13 @@ job: - if: '$CI_PIPELINE_SOURCE == "schedule"' ``` -This example runs the job as a `when: on_success` job in [merge request pipelines](../merge_request_pipelines/index.md) -and scheduled pipelines. It does not run in any other pipeline type. - Other commonly used variables for `if` clauses: - `if: $CI_COMMIT_TAG`: If changes are pushed for a tag. - `if: $CI_COMMIT_BRANCH`: If changes are pushed to any branch. -- `if: '$CI_COMMIT_BRANCH == "master"'`: If changes are pushed to `master`. +- `if: '$CI_COMMIT_BRANCH == "main"'`: If changes are pushed to `main`. - `if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'`: If changes are pushed to the default - branch (usually `master`). Use when you want to have the same configuration in multiple + branch. Use when you want to have the same configuration in multiple projects with different default branches. - `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression. - `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/README.md#custom-cicd-variables) @@ -1310,11 +1272,11 @@ Other commonly used variables for `if` clauses: #### `rules:changes` -`rules:changes` determines whether or not to add jobs to a pipeline by checking for +Use `rules:changes` to specify when to add a job to a pipeline by checking for changes to specific files. -`rules: changes` works exactly the same way as [`only: changes` and `except: changes`](#onlychangesexceptchanges), -accepting an array of paths. It's recommended to only use `rules: changes` with branch +`rules: changes` works the same way as [`only: changes` and `except: changes`](#onlychangesexceptchanges). +It accepts an array of paths. You should use `rules: changes` only with branch pipelines or merge request pipelines. For example, it's common to use `rules: changes` with merge request pipelines: @@ -1337,7 +1299,7 @@ In this example: - If `Dockerfile` has not changed, do not add job to any pipeline (same as `when: never`). To use `rules: changes` with branch pipelines instead of merge request pipelines, -change the `if:` clause in the example above to: +change the `if:` clause in the previous example to: ```yaml rules: @@ -1359,7 +1321,7 @@ if there is no `if:` statement that limits the job to branch or merge request pi > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34272) in GitLab 13.6. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.7. -CI/CD variables can be used in `rules:changes` expressions to determine when +You can use CI/CD variables in `rules:changes` expressions to determine when to add jobs to a pipeline: ```yaml @@ -1372,7 +1334,7 @@ docker build: - $DOCKERFILES_DIR/* ``` -You can use The `$` character for both variables and paths. For example, if the +You can use the `$` character for both variables and paths. For example, if the `$DOCKERFILES_DIR` variable exists, its value is used. If it does not exist, the `$` is interpreted as being part of a path. @@ -1380,10 +1342,10 @@ You can use The `$` character for both variables and paths. For example, if the > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4. -`exists` accepts an array of paths and matches if any of these paths exist -as files in the repository. +Use `exists` to run a job when certain files exist in the repository. +You can use an array of paths. -In this example, `job` runs if a `Dockerfile` exists anywhere in the repository: +In the following example, `job` runs if a `Dockerfile` exists anywhere in the repository: ```yaml job: @@ -1416,11 +1378,11 @@ For performance reasons, GitLab matches a maximum of 10,000 `exists` patterns. A You can use [`allow_failure: true`](#allow_failure) in `rules:` to allow a job to fail, or a manual job to wait for action, without stopping the pipeline itself. All jobs that use `rules:` default to `allow_failure: false` -if `allow_failure:` is not defined. +if you do not define `allow_failure:`. The rule-level `rules:allow_failure` option overrides the job-level -[`allow_failure`](#allow_failure) option, and is only applied when the job is -triggered by the particular rule. +[`allow_failure`](#allow_failure) option, and is only applied when +the particular rule triggers the job. ```yaml job: @@ -1436,16 +1398,9 @@ In this example, if the first rule matches, then the job has `when: manual` and #### `rules:variables` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/209864) in GitLab 13.7. -> - It was [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/289803) on GitLab 13.8. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-rulesvariables). **(FREE SELF)** +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/289803) in GitLab 13.10. -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - -You can use [`variables`](#variables) in `rules:` to define variables for specific conditions. +Use [`variables`](#variables) in `rules:` to define variables for specific conditions. For example: @@ -1465,25 +1420,6 @@ job: - echo "Run another script if $IS_A_FEATURE exists" ``` -##### Enable or disable rules:variables **(FREE SELF)** - -rules:variables is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:ci_rules_variables) -``` - -To disable it: - -```ruby -Feature.disable(:ci_rules_variables) -``` - #### Complex rule clauses To conjoin `if`, `changes`, and `exists` clauses with an `AND`, use them in the @@ -1540,14 +1476,14 @@ to add jobs to pipelines, use [`rules`](#rules). 1. `except` defines the names of branches and tags the job does **not** run for. -There are a few rules that apply to the usage of job policy: +A few rules apply to the usage of job policy: - `only` and `except` are inclusive. If both `only` and `except` are defined in a job specification, the ref is filtered by `only` and `except`. - `only` and `except` can use regular expressions ([supported regexp syntax](#supported-onlyexcept-regexp-syntax)). - `only` and `except` can specify a repository path to filter jobs for forks. -In addition, `only` and `except` can use special keywords: +In addition, `only` and `except` can use these keywords: | **Value** | **Description** | |--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -1568,8 +1504,8 @@ Scheduled pipelines run on specific branches, so jobs configured with `only: bra run on scheduled pipelines too. Add `except: schedules` to prevent jobs with `only: branches` from running on scheduled pipelines. -In the example below, `job` runs only for refs that start with `issue-`, -whereas all branches are skipped: +In the following example, `job` runs only for refs that start with `issue-`. +All branches are skipped: ```yaml job: @@ -1581,8 +1517,8 @@ job: - branches ``` -Pattern matching is case-sensitive by default. Use `i` flag modifier, like -`/pattern/i` to make a pattern case-insensitive: +Pattern matching is case-sensitive by default. Use the `i` flag modifier, like +`/pattern/i`, to make a pattern case-insensitive: ```yaml job: @@ -1594,8 +1530,11 @@ job: - branches ``` -In this example, `job` runs only for refs that are tagged, or if a build is -explicitly requested by an API trigger or a [Pipeline Schedule](../pipelines/schedules.md): +In the following example, `job` runs only for: + +- Git tags +- [Triggers](../triggers/README.md#trigger-token) +- [Scheduled pipelines](../pipelines/schedules.md) ```yaml job: @@ -1606,8 +1545,7 @@ job: - schedules ``` -Use the repository path to have jobs executed only for the parent -repository and not forks: +To execute jobs only for the parent repository and not forks: ```yaml job: @@ -1618,11 +1556,11 @@ job: - /^release/.*$/@gitlab-org/gitlab ``` -The above example runs `job` for all branches on `gitlab-org/gitlab`, -except `master` and those with names prefixed with `release/`. +This example runs `job` for all branches on `gitlab-org/gitlab`, +except `master` and branches that start with `release/`. If a job does not have an `only` rule, `only: ['branches', 'tags']` is set by -default. If it does not have an `except` rule, it's empty. +default. If the job does not have an `except` rule, it's empty. For example, `job1` and `job2` are essentially the same: @@ -1698,7 +1636,7 @@ the pipeline if the following is true: - `(any listed refs are true) AND (any listed variables are true) AND (any listed changes are true) AND (any chosen Kubernetes status matches)` -In the example below, the `test` job is `only` created when **all** of the following are true: +In the following example, the `test` job is `only` created when **all** of the following are true: - The pipeline is [scheduled](../pipelines/schedules.md) **or** runs for `master`. - The `variables` keyword matches. @@ -1721,7 +1659,7 @@ added if the following is true: - `(any listed refs are true) OR (any listed variables are true) OR (any listed changes are true) OR (a chosen Kubernetes status matches)` -In the example below, the `test` job is **not** created when **any** of the following are true: +In the following example, the `test` job is **not** created when **any** of the following are true: - The pipeline runs for the `master` branch. - There are changes to the `README.md` file in the root directory of the repository. @@ -1743,7 +1681,7 @@ test: The `refs` strategy can take the same values as the [simplified only/except configuration](#onlyexcept-basic). -In the example below, the `deploy` job is created only when the +In the following example, the `deploy` job is created only when the pipeline is [scheduled](../pipelines/schedules.md) or runs for the `master` branch: ```yaml @@ -1760,7 +1698,7 @@ deploy: The `kubernetes` strategy accepts only the `active` keyword. -In the example below, the `deploy` job is created only when the +In the following example, the `deploy` job is created only when the Kubernetes service is active in the project: ```yaml @@ -1831,7 +1769,7 @@ In pipelines with [sources other than the three above](../variables/predefined_v You can configure jobs to use `only: changes` with other `only: refs` keywords. However, those jobs ignore the changes and always run. -In this example, when you push commits to an existing branch, the `docker build` job +In the following example, when you push commits to an existing branch, the `docker build` job runs only if any of these files change: - The `Dockerfile` file. @@ -1928,7 +1866,7 @@ docker build service one: - service-one/**/* ``` -In the example above, the pipeline might fail because of changes to a file in `service-one/**/*`. +In this example, the pipeline might fail because of changes to a file in `service-one/**/*`. A later commit that doesn't have changes in `service-one/**/*` but does have changes to the `Dockerfile` can pass. The job @@ -1962,13 +1900,22 @@ All files are considered to have changed when a scheduled pipeline runs. > - In GitLab 12.3, maximum number of jobs in `needs` array raised from five to 50. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30631) in GitLab 12.8, `needs: []` lets jobs start immediately. -Use the `needs:` keyword to execute jobs out-of-order. Relationships between jobs +Use `needs:` to execute jobs out-of-order. Relationships between jobs that use `needs` can be visualized as a [directed acyclic graph](../directed_acyclic_graph/index.md). You can ignore stage ordering and run some jobs without waiting for others to complete. Jobs in multiple stages can run concurrently. -Let's consider the following example: +The following example creates four paths of execution: + +- Linter: the `lint` job runs immediately without waiting for the `build` stage + to complete because it has no needs (`needs: []`). +- Linux path: the `linux:rspec` and `linux:rubocop` jobs runs as soon as the `linux:build` + job finishes without waiting for `mac:build` to finish. +- macOS path: the `mac:rspec` and `mac:rubocop` jobs runs as soon as the `mac:build` + job finishes, without waiting for `linux:build` to finish. +- The `production` job runs as soon as all previous jobs finish; in this case: + `linux:build`, `linux:rspec`, `linux:rubocop`, `mac:build`, `mac:rspec`, `mac:rubocop`. ```yaml linux:build: @@ -2001,25 +1948,10 @@ production: stage: deploy ``` -This example creates four paths of execution: - -- Linter: the `lint` job runs immediately without waiting for the `build` stage to complete because it has no needs (`needs: []`). - -- Linux path: the `linux:rspec` and `linux:rubocop` jobs runs as soon - as the `linux:build` job finishes without waiting for `mac:build` to finish. - -- macOS path: the `mac:rspec` and `mac:rubocop` jobs runs as soon - as the `mac:build` job finishes, without waiting for `linux:build` to finish. - -- The `production` job runs as soon as all previous jobs - finish; in this case: `linux:build`, `linux:rspec`, `linux:rubocop`, - `mac:build`, `mac:rspec`, `mac:rubocop`. - #### Requirements and limitations -- If `needs:` is set to point to a job that is not instantiated - because of `only/except` rules or otherwise does not exist, the - pipeline is not created and a YAML error is shown. +- In GitLab 13.9 and older, if `needs:` refers to a job that might not be added to + a pipeline because of `only`, `except`, or `rules`, the pipeline might fail to create. - The maximum number of jobs that a single job can need in the `needs:` array is limited: - For GitLab.com, the limit is 50. For more information, see our [infrastructure issue](https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/7541). @@ -2031,7 +1963,7 @@ This example creates four paths of execution: - `needs:` is similar to `dependencies:` in that it must use jobs from prior stages, meaning it's impossible to create circular dependencies. Depending on jobs in the current stage is not possible either, but support [is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/30632). -- Related to the above, stages must be explicitly defined for all jobs +- Stages must be explicitly defined for all jobs that have the keyword `needs:` or are referred to by one. ##### Changing the `needs:` job limit **(FREE SELF)** @@ -2054,7 +1986,7 @@ To disable directed acyclic graphs (DAG), set the limit to `0`. Use `artifacts: true` (default) or `artifacts: false` to control when artifacts are downloaded in jobs that use `needs`. -In this example, the `rspec` job downloads the `build_job` artifacts, but the +In the following example, the `rspec` job downloads the `build_job` artifacts, but the `rubocop` job does not: ```yaml @@ -2077,7 +2009,7 @@ rubocop: artifacts: false ``` -In this example, the `rspec` job downloads the artifacts from all three `build_jobs`. +In the following example, the `rspec` job downloads the artifacts from all three `build_jobs`. `artifacts` is: - Set to true for `build_job_1`. @@ -2128,7 +2060,7 @@ The user running the pipeline must have at least `reporter` access to the group Use `needs` to download artifacts from different pipelines in the current project. Set the `project` keyword as the current project's name, and specify a ref. -In this example, `build_job` downloads the artifacts for the latest successful +In the following example, `build_job` downloads the artifacts for the latest successful `build-1` job with the `other-ref` ref: ```yaml @@ -2160,12 +2092,13 @@ build_job: artifacts: true ``` -Downloading artifacts from jobs that are run in [`parallel:`](#parallel) is not supported. +You can't download artifacts from jobs that run in [`parallel:`](#parallel). + +To download artifacts between [parent-child pipelines](../parent_child_pipelines.md), +use [`needs:pipeline`](#artifact-downloads-to-child-pipelines). -To download artifacts between [parent-child pipelines](../parent_child_pipelines.md) use [`needs:pipeline`](#artifact-downloads-to-child-pipelines). -Downloading artifacts from the same ref as the currently running pipeline is not -recommended because artifacts could be overridden by concurrent pipelines running -on the same ref. +You should not download artifacts from the same ref as a running pipeline. Concurrent +pipelines running on the same ref could override the artifacts. ##### Artifact downloads to child pipelines @@ -2209,6 +2142,68 @@ in the same parent-child pipeline hierarchy of the given pipeline. The `pipeline` attribute does not accept the current pipeline ID (`$CI_PIPELINE_ID`). To download artifacts from a job in the current pipeline, use the basic form of [`needs`](#artifact-downloads-with-needs). +#### Optional `needs` + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30680) in GitLab 13.10. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-optional-needs). **(FREE SELF)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +To need a job that sometimes does not exist in the pipeline, add `optional: true` +to the `needs` configuration. If not defined, `optional: false` is the default. + +Jobs that use [`rules`](#rules), [`only`, or `except`](#onlyexcept-basic), might +not always exist in a pipeline. When the pipeline starts, it checks the `needs` +relationships before running. Without `optional: true`, needs relationships that +point to a job that does not exist stops the pipeline from starting and causes a pipeline +error similar to: + +- `'job1' job needs 'job2' job, but it was not added to the pipeline` + +In this example: + +- When the branch is `master`, the `build` job exists in the pipeline, and the `rspec` + job waits for it to complete before starting. +- When the branch is not `master`, the `build` job does not exist in the pipeline. + The `rspec` job runs immediately (similar to `needs: []`) because its `needs` + relationship to the `build` job is optional. + +```yaml +build: + stage: build + rules: + - if: $CI_COMMIT_REF_NAME == "master" + +rspec: + stage: test + needs: + - job: build + optional: true +``` + +#### Enable or disable optional needs **(FREE SELF)** + +Optional needs is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:ci_needs_optional) +``` + +To disable it: + +```ruby +Feature.disable(:ci_needs_optional) +``` + ### `tags` Use `tags` to select a specific runner from the list of all runners that are @@ -2217,7 +2212,7 @@ available for the project. When you register a runner, you can specify the runner's tags, for example `ruby`, `postgres`, `development`. -In this example, the job is run by a runner that +In the following example, the job is run by a runner that has both `ruby` and `postgres` tags defined. ```yaml @@ -2266,7 +2261,7 @@ Assuming all other jobs are successful, the job's stage and its pipeline show the same orange warning. However, the associated commit is marked as "passed", without warnings. -In the example below, `job1` and `job2` run in parallel, but if `job1` +In the following example, `job1` and `job2` run in parallel. If `job1` fails, it doesn't stop the next stage from running, because it's marked with `allow_failure: true`: @@ -2331,9 +2326,14 @@ The valid values of `when` are: Added in GitLab 11.14. 1. `never`: - With [`rules`](#rules), don't execute job. - - With [`workflow:rules`](#workflowrules), don't run pipeline. + - With [`workflow`](#workflow), don't run pipeline. -For example: +In the following example, the script: + +1. Executes `cleanup_build_job` only when `build_job` fails. +1. Always executes `cleanup_job` as the last step in pipeline regardless of + success or failure. +1. Executes `deploy_job` when you run it manually in the GitLab UI. ```yaml stages: @@ -2372,13 +2372,6 @@ cleanup_job: when: always ``` -The above script: - -1. Executes `cleanup_build_job` only when `build_job` fails. -1. Always executes `cleanup_job` as the last step in pipeline regardless of - success or failure. -1. Executes `deploy_job` when you run it manually in the GitLab UI. - #### `when:manual` A manual job is a type of job that is not executed automatically and must be explicitly @@ -2386,7 +2379,8 @@ started by a user. You might want to use manual jobs for things like deploying t To make a job manual, add `when: manual` to its configuration. -Manual jobs can be started from the pipeline, job, [environment](../environments/index.md#configuring-manual-deployments), +When the pipeline starts, manual jobs display as skipped and do not run automatically. +They can be started from the pipeline, job, [environment](../environments/index.md#configure-manual-deployments), and deployment views. Manual jobs can be either optional or blocking: @@ -2441,7 +2435,7 @@ To protect a manual job: ``` 1. In the [protected environments settings](../environments/protected_environments.md#protecting-environments), - select the environment (`production` in the example above) and add the users, roles or groups + select the environment (`production` in this example) and add the users, roles or groups that are authorized to trigger the manual job to the **Allowed to Deploy** list. Only those in this list can trigger this manual job, as well as GitLab administrators who are always able to use protected environments. @@ -2492,10 +2486,7 @@ Soon GitLab Runner picks up and starts the job. ### `environment` Use `environment` to define the [environment](../environments/index.md) that a job deploys to. -If `environment` is specified and no environment under that name exists, a new -one is created automatically. - -In its simplest form, the `environment` keyword can be defined like: +For example: ```yaml deploy to production: @@ -2504,34 +2495,20 @@ deploy to production: environment: production ``` -In the above example, the `deploy to production` job is marked as doing a -deployment to the `production` environment. - -#### `environment:name` +You can assign a value to the `environment` keyword by using: -The `environment: name` keyword can use any of the defined CI/CD [variables](#variables), -including predefined, secure, or variables defined in the `.gitlab-ci.yml` file. +- Plain text, like `production`. +- Variables, including CI/CD variables, predefined, secure, or variables + defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. -The `environment` name can contain: +If you specify an `environment` and no environment with that name exists, +an environment is created. -- letters -- digits -- spaces -- `-` -- `_` -- `/` -- `$` -- `{` -- `}` - -Common names are `qa`, `staging`, and `production`, but you can use whatever -name works with your workflow. +#### `environment:name` -Instead of defining the name of the environment right after the `environment` -keyword, it's also possible to define it as a separate value. For that, use -the `name` keyword under `environment`: +Set a name for an [environment](../environments/index.md). For example: ```yaml deploy to production: @@ -2541,18 +2518,32 @@ deploy to production: name: production ``` -#### `environment:url` +Common environment names are `qa`, `staging`, and `production`, but you can use any +name you want. -The `environment:url` keyword can use any of the defined CI/CD [variables](#variables), -including predefined, secure, or variables defined in the `.gitlab-ci.yml` file. +You can assign a value to the `name` keyword by using: + +- Plain text, like `staging`. +- Variables, including CI/CD variables, predefined, secure, or variables + defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. -This optional value exposes buttons that take you to the defined URL +The environment `name` can contain: + +- Letters +- Digits +- Spaces +- `-` +- `_` +- `/` +- `$` +- `{` +- `}` + +#### `environment:url` -In this example, if the job finishes successfully, it creates buttons -in the merge requests and in the environments/deployments pages that point -to `https://prod.example.com`. +Set a URL for an [environment](../environments/index.md). For example: ```yaml deploy to production: @@ -2563,12 +2554,18 @@ deploy to production: url: https://prod.example.com ``` -#### `environment:on_stop` +After the job completes, you can access the URL by using a button in the merge request, +environment, or deployment pages. + +You can assign a value to the `url` keyword by using: -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/22191) in GitLab 8.13. -> - Starting with GitLab 8.14, when you have an environment that has a stop action -> defined, GitLab automatically triggers a stop action when the associated -> branch is deleted. +- Plain text, like `https://prod.example.com`. +- Variables, including CI/CD variables, predefined, secure, or variables + defined in the `.gitlab-ci.yml` file. + +You can't use variables defined in a `script` section. + +#### `environment:on_stop` Closing (stopping) environments can be achieved with the `on_stop` keyword defined under `environment`. It declares a different job that runs to close the @@ -2578,8 +2575,6 @@ Read the `environment:action` section for an example. #### `environment:action` -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/22191) in GitLab 8.13. - Use the `action` keyword to specify jobs that prepare, start, or stop environments. | **Value** | **Description** | @@ -2618,7 +2613,7 @@ it is set to `manual`, so it needs a [manual action](#whenmanual) from the GitLab UI to run. Also in the example, `GIT_STRATEGY` is set to `none`. If the -`stop_review_app` job is [automatically triggered](../environments/index.md#automatically-stopping-an-environment), +`stop_review_app` job is [automatically triggered](../environments/index.md#stopping-an-environment), the runner won’t try to check out the code after the branch is deleted. The example also overwrites global variables. If your `stop` `environment` job depends @@ -2627,14 +2622,18 @@ to change the job without overriding the global variables. The `stop_review_app` job is **required** to have the following keywords defined: -- `when` - [reference](#when) +- `when`, defined at either: + - [The job level](#when). + - [In a rules clause](#rules). If you use `rules:` and `when: manual`, you should + also set [`allow_failure: true`](#allow_failure) so the pipeline can complete + even if the job doesn't run. - `environment:name` - `environment:action` Additionally, both jobs should have matching [`rules`](../yaml/README.md#onlyexcept-basic) or [`only/except`](../yaml/README.md#onlyexcept-basic) configuration. -In the example above, if the configuration is not identical: +In the examples above, if the configuration is not identical: - The `stop_review_app` job might not be included in all pipelines that include the `review_app` job. - It is not possible to trigger the `action: stop` to stop the environment automatically. @@ -2660,7 +2659,7 @@ When the environment for `review_app` is created, the environment's lifetime is Every time the review app is deployed, that lifetime is also reset to `1 day`. For more information, see -[the environments auto-stop documentation](../environments/index.md#environments-auto-stop) +[the environments auto-stop documentation](../environments/index.md#stop-an-environment-after-a-certain-time-period) #### `environment:kubernetes` @@ -2686,7 +2685,7 @@ environment, using the `production` [Kubernetes namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/). For more information, see -[Available settings for `kubernetes`](../environments/index.md#configuring-kubernetes-deployments). +[Available settings for `kubernetes`](../environments/index.md#configure-kubernetes-deployments). NOTE: Kubernetes configuration is not supported for Kubernetes clusters @@ -2694,6 +2693,23 @@ that are [managed by GitLab](../../user/project/clusters/index.md#gitlab-managed To follow progress on support for GitLab-managed clusters, see the [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/38054). +#### `environment:deployment_tier` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 13.10. + +Use the `deployment_tier` keyword to specify the tier of the deployment environment: + +```yaml +deploy: + script: echo + environment: + name: customer-portal + deployment_tier: production +``` + +For more information, +see [Deployment tier of environments](../environments/index.md#deployment-tier-of-environments). + #### Dynamic environments Use CI/CD [variables](../variables/README.md) to dynamically name environments. @@ -2722,7 +2738,7 @@ as Review Apps. You can see an example that uses Review Apps at ### `cache` -Use the `cache` keyword to specify a list of files and directories to +Use `cache` to specify a list of files and directories to cache between jobs. You can only use paths that are in the local working copy. If `cache` is defined outside the scope of jobs, it's set @@ -2812,7 +2828,62 @@ URI-encoded `%2F`. A value made only of dots (`.`, `%2E`) is also forbidden. You can specify a [fallback cache key](#fallback-cache-key) to use if the specified `cache:key` is not found. -##### Fallback cache key +##### Multiple caches + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32814) in GitLab 13.10. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-multiple-caches). **(FREE SELF)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +You can have a maximum of four caches: + +```yaml +test-job: + stage: build + cache: + - key: + files: + - Gemfile.lock + paths: + - vendor/ruby + - key: + files: + - yarn.lock + paths: + - .yarn-cache/ + script: + - bundle install --path=vendor + - yarn install --cache-folder .yarn-cache + - echo Run tests... +``` + +If multiple caches are combined with a [Fallback cache key](#fallback-cache-key), +the fallback is fetched multiple times if multiple caches are not found. + +##### Enable or disable multiple caches **(FREE SELF)** + +The multiple caches feature is under development and not ready for production use. +It is deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:multiple_cache_per_job) +``` + +To disable it: + +```ruby +Feature.disable(:multiple_cache_per_job) +``` + +#### Fallback cache key > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4. @@ -2823,7 +2894,7 @@ to download cache that's tagged with `test`. If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to specify a cache to use when none exists. -In this example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined +In the following example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined by the `CACHE_FALLBACK_KEY` variable: ```yaml @@ -2860,7 +2931,7 @@ cache: - node_modules ``` -In this example we're creating a cache for Ruby and Node.js dependencies that +This example creates a cache for Ruby and Node.js dependencies that is tied to current versions of the `Gemfile.lock` and `package.json` files. Whenever one of these files changes, a new cache key is computed and a new cache is created. Any future job runs that use the same `Gemfile.lock` and `package.json` with `cache:key:files` @@ -2994,7 +3065,7 @@ To do so, add `policy: push` to the job. ### `artifacts` -Use the `artifacts` keyword to specify a list of files and directories that are +Use `artifacts` to specify a list of files and directories that are attached to the job when it [succeeds, fails, or always](#artifactswhen). The artifacts are sent to GitLab after the job finishes. They are @@ -3461,8 +3532,7 @@ for more details. ### `retry` -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/3442) in GitLab 9.5. -> - [Behavior expanded](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3515) in GitLab 11.5 to control which failures to retry on. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3515) in GitLab 11.5, you can control which failures to retry on. Use `retry` to configure how many times a job is retried in case of a failure. @@ -3474,7 +3544,7 @@ If `retry` is set to `2`, and a job succeeds in a second run (first retry), it i The `retry` value must be a positive integer, from `0` to `2` (two retries maximum, three runs in total). -This example retries all failure cases: +The following example retries all failure cases: ```yaml test: @@ -3636,7 +3706,7 @@ deploystacks: STACK: [data, processing] ``` -This example generates 10 parallel `deploystacks` jobs, each with different values +The following example generates 10 parallel `deploystacks` jobs, each with different values for `PROVIDER` and `STACK`: ```plaintext @@ -3670,6 +3740,40 @@ deploystacks: - PROVIDER: [aws, ovh, gcp, vultr] ``` +##### Parallel `matrix` trigger jobs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/270957) in GitLab 13.10. + +Use `matrix:` to run a [trigger](#trigger) job multiple times in parallel in a single pipeline, +but with different variable values for each instance of the job. + +```yaml +deploystacks: + stage: deploy + trigger: + include: path/to/child-pipeline.yml + parallel: + matrix: + - PROVIDER: aws + STACK: [monitoring, app1] + - PROVIDER: ovh + STACK: [monitoring, backup] + - PROVIDER: [gcp, vultr] + STACK: [data] +``` + +This example generates 6 parallel `deploystacks` trigger jobs, each with different values +for `PROVIDER` and `STACK`, and they create 6 different child pipelines with those variables. + +```plaintext +deploystacks: [aws, monitoring] +deploystacks: [aws, app1] +deploystacks: [ovh, monitoring] +deploystacks: [ovh, backup] +deploystacks: [gcp, data] +deploystacks: [vultr, data] +``` + ### `trigger` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. @@ -3821,6 +3925,10 @@ To force the `trigger` job to wait for the downstream (multi-project or child) p pipeline completes. At that point, the `trigger` job completes and displays the same status as the downstream job. +This setting can help keep your pipeline execution linear. In the following example, jobs from +subsequent stages wait for the triggered pipeline to successfully complete before +starting, which reduces parallelization. + ```yaml trigger_job: trigger: @@ -3828,10 +3936,6 @@ trigger_job: strategy: depend ``` -This setting can help keep your pipeline execution linear. In the example above, jobs from -subsequent stages wait for the triggered pipeline to successfully complete before -starting, which reduces parallelization. - #### Trigger a pipeline by API call To force a rebuild of a specific branch, tag, or commit, you can use an API call @@ -3858,7 +3962,12 @@ When enabled, a pipeline is immediately canceled when a new pipeline starts on t Set jobs as interruptible that can be safely canceled once started (for instance, a build job). -For example: +In the following example, a new pipeline run causes an existing running pipeline to be: + +- Canceled, if only `step-1` is running or pending. +- Not canceled, once `step-2` starts running. + +After an uninterruptible job starts running, the pipeline cannot be canceled. ```yaml stages: @@ -3884,13 +3993,6 @@ step-3: interruptible: true ``` -In the example above, a new pipeline run causes an existing running pipeline to be: - -- Canceled, if only `step-1` is running or pending. -- Not canceled, once `step-2` starts running. - -When an uninterruptible job is running, the pipeline cannot be canceled, regardless of the final job's state. - ### `resource_group` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7. @@ -3937,7 +4039,7 @@ executions. The [`trigger` keyword](#trigger) can trigger downstream pipelines. [`resource_group` keyword](#resource_group) can co-exist with it. This is useful to control the concurrency for deployment pipelines, while running non-sensitive jobs concurrently. -This example has two pipeline configurations in a project. When a pipeline starts running, +The following example has two pipeline configurations in a project. When a pipeline starts running, non-sensitive jobs are executed first and aren't affected by concurrent executions in other pipelines. However, GitLab ensures that there are no other deployment pipelines running before triggering a deployment (child) pipeline. If other deployment pipelines are running, GitLab waits @@ -3978,7 +4080,7 @@ deployment: script: echo "Deploying..." ``` -Note that you must define [`strategy: depend`](#linking-pipelines-with-triggerstrategy) +You must define [`strategy: depend`](#linking-pipelines-with-triggerstrategy) with the `trigger` keyword. This ensures that the lock isn't released until the downstream pipeline finishes. @@ -3986,7 +4088,7 @@ finishes. > [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/19298) in GitLab 13.2. -`release` indicates that the job creates a [Release](../../user/project/releases/index.md). +Use `release` to create a [release](../../user/project/releases/index.md). These keywords are supported: @@ -3997,18 +4099,18 @@ These keywords are supported: - [`milestones`](#releasemilestones) (optional) - [`released_at`](#releasereleased_at) (optional) -The Release is created only if the job processes without error. If the Rails API -returns an error during Release creation, the `release` job fails. +The release is created only if the job processes without error. If the Rails API +returns an error during release creation, the `release` job fails. #### `release-cli` Docker image -The Docker image to use for the `release-cli` must be specified, using the following directive: +You must specify the Docker image to use for the `release-cli`: ```yaml image: registry.gitlab.com/gitlab-org/release-cli:latest ``` -#### Script +#### `script` All jobs except [trigger](#trigger) jobs must have the `script` keyword. A `release` job can use the output from script commands, but you can use a placeholder script if @@ -4041,11 +4143,12 @@ android-release: #### `release:tag_name` -The `tag_name` must be specified. It can refer to an existing Git tag or can be specified by the user. +You must specify a `tag_name` for the release. The tag can refer to an existing Git tag or +you can specify a new tag. When the specified tag doesn't exist in the repository, a new tag is created from the associated SHA of the pipeline. -For example, when creating a Release from a Git tag: +For example, when creating a release from a Git tag: ```yaml job: @@ -4064,17 +4167,17 @@ job: description: 'Release description' ``` -- The Release is created only if the job's main script succeeds. -- If the Release already exists, it is not updated and the job with the `release` keyword fails. +- The release is created only if the job's main script succeeds. +- If the release already exists, it is not updated and the job with the `release` keyword fails. - The `release` section executes after the `script` tag and before the `after_script`. #### `release:name` -The Release name. If omitted, it is populated with the value of `release: tag_name`. +The release name. If omitted, it is populated with the value of `release: tag_name`. #### `release:description` -Specifies the long description of the Release. You can also specify a file that contains the +Specifies the long description of the release. You can also specify a file that contains the description. ##### Read description from a file @@ -4112,8 +4215,7 @@ released_at: '2021-03-15T08:00:00Z' #### Complete example for `release` -Combining the individual examples given above for `release` results in the following -code snippets. There are two options, depending on how you generate the +If you combine the previous examples for `release`, you get two options, depending on how you generate the tags. You can't use these options together, so choose one: - To create a release when you push a Git tag, or when you add a Git tag @@ -4197,7 +4299,7 @@ The entries under the `release` node are transformed into a `bash` command line to the Docker container, which contains the [release-cli](https://gitlab.com/gitlab-org/release-cli). You can also call the `release-cli` directly from a `script` entry. -For example, using the YAML described above: +For example, if you use the YAML described previously: ```shell release-cli create --name "Release $CI_COMMIT_SHA" --description "Created using the release-cli $EXTRA_DESCRIPTION" --tag-name "v${MAJOR}.${MINOR}.${REVISION}" --ref "$CI_COMMIT_SHA" --released-at "2020-07-15T08:00:00Z" --milestone "m1" --milestone "m2" --milestone "m3" @@ -4207,7 +4309,7 @@ release-cli create --name "Release $CI_COMMIT_SHA" --description "Created using > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33014) in GitLab 13.4. -`secrets` indicates the [CI/CD Secrets](../secrets/index.md) this job needs. It should be a hash, +Use `secrets` to specify the [CI/CD Secrets](../secrets/index.md) the job needs. It should be a hash, and the keys should be the names of the variables that are made available to the job. The value of each secret is saved in a temporary file. This file's path is stored in these variables. @@ -4216,7 +4318,8 @@ variables. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28321) in GitLab 13.4. -`vault` keyword specifies secrets provided by [Hashicorp's Vault](https://www.vaultproject.io/). +Use `vault` to specify secrets provided by [Hashicorp's Vault](https://www.vaultproject.io/). + This syntax has multiple forms. The shortest form assumes the use of the [KV-V2](https://www.vaultproject.io/docs/secrets/kv/kv-v2) secrets engine, mounted at the default path `kv-v2`. The last part of the secret's path is the @@ -4254,14 +4357,13 @@ job: ### `pages` -`pages` is a special job that uploads static content to GitLab that -is then published as a website. It has a special syntax, so the two -requirements below must be met: +Use `pages` to upload static content to GitLab. The content +is then published as a website. You must: -- Any static content must be placed under a `public/` directory. -- `artifacts` with a path to the `public/` directory must be defined. +- Place any static content in a `public/` directory. +- Define [`artifacts`](#artifacts) with a path to the `public/` directory. -The example below moves all files from the root of the project to the +The following example moves all files from the root of the project to the `public/` directory. The `.public` workaround is so `cp` does not also copy `public/` to itself in an infinite loop: @@ -4279,7 +4381,89 @@ pages: - master ``` -Read more on [GitLab Pages user documentation](../../user/project/pages/index.md). +View the [GitLab Pages user documentation](../../user/project/pages/index.md). + +### `inherit` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207484) in GitLab 12.9. + +Use `inherit:` to control inheritance of globally-defined defaults +and variables. + +To enable or disable the inheritance of all `default:` or `variables:` keywords, use: + +- `default: true` or `default: false` +- `variables: true` or `variables: false` + +To inherit only a subset of `default:` keywords or `variables:`, specify what +you wish to inherit. Anything not listed is **not** inherited. Use +one of the following formats: + +```yaml +inherit: + default: [keyword1, keyword2] + variables: [VARIABLE1, VARIABLE2] +``` + +Or: + +```yaml +inherit: + default: + - keyword1 + - keyword2 + variables: + - VARIABLE1 + - VARIABLE2 +``` + +In the following example: + +- `rubocop`: + - inherits: Nothing. +- `rspec`: + - inherits: the default `image` and the `WEBHOOK_URL` variable. + - does **not** inherit: the default `before_script` and the `DOMAIN` variable. +- `capybara`: + - inherits: the default `before_script` and `image`. + - does **not** inherit: the `DOMAIN` and `WEBHOOK_URL` variables. +- `karma`: + - inherits: the default `image` and `before_script`, and the `DOMAIN` variable. + - does **not** inherit: `WEBHOOK_URL` variable. + +```yaml +default: + image: 'ruby:2.4' + before_script: + - echo Hello World + +variables: + DOMAIN: example.com + WEBHOOK_URL: https://my-webhook.example.com + +rubocop: + inherit: + default: false + variables: false + script: bundle exec rubocop + +rspec: + inherit: + default: [image] + variables: [WEBHOOK_URL] + script: bundle exec rspec + +capybara: + inherit: + variables: false + script: bundle exec capybara + +karma: + inherit: + default: true + variables: [DOMAIN] + script: karma +``` ## `variables` @@ -4337,7 +4521,7 @@ You can use [YAML anchors for variables](#yaml-anchors-for-variables). > [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/30101) GitLab 13.7. -You can use the `value` and `description` keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) +Use the `value` and `description` keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) when [running a pipeline manually](../pipelines/index.md#run-a-pipeline-manually): ```yaml @@ -4349,7 +4533,7 @@ variables: ### Configure runner behavior with variables -You can use [CI/CD variables](../variables/README.md) to configure runner Git behavior: +You can use [CI/CD variables](../variables/README.md) to configure how the runner processes Git requests: - [`GIT_STRATEGY`](../runners/README.md#git-strategy) - [`GIT_SUBMODULE_STRATEGY`](../runners/README.md#git-submodule-strategy) @@ -4365,16 +4549,16 @@ You can use [CI/CD variables](../variables/README.md) to configure runner Git be You can also use variables to configure how many times a runner [attempts certain stages of job execution](../runners/README.md#job-stages-attempts). -## Special YAML features +## YAML-specific features -It's possible to use special YAML features like anchors (`&`), aliases (`*`) +In your `.gitlab-ci.yml` file, you can use YAML-specific features like anchors (`&`), aliases (`*`), and map merging (`<<`). Use these features to reduce the complexity of the code in the `.gitlab-ci.yml` file. Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/). -In most cases, the [`extends` keyword](#extends) is more user friendly and should -be used over these special YAML features. +In most cases, the [`extends` keyword](#extends) is more user friendly and you should +use it when possible. You can use YAML anchors to merge YAML arrays. @@ -4387,9 +4571,10 @@ Use anchors to duplicate or inherit properties. Use anchors with [hidden jobs](# to provide templates for your jobs. When there are duplicate keys, GitLab performs a reverse deep merge based on the keys. -You can't use YAML anchors across multiple files when leveraging the [`include`](#include) -feature. Anchors are only valid in the file they were defined in. Instead -of using YAML anchors, you can use the [`extends` keyword](#extends). +You can't use YAML anchors across multiple files when using the [`include`](#include) +keyword. Anchors are only valid in the file they were defined in. To reuse configuration +from different YAML files, use [`!reference` tags](#reference-tags) or the +[`extends` keyword](#extends). The following example uses anchors and map merging. It creates two jobs, `test1` and `test2`, that inherit the `.job_template` configuration, each @@ -4414,8 +4599,8 @@ test2: ``` `&` sets up the name of the anchor (`job_configuration`), `<<` means "merge the -given hash into the current one", and `*` includes the named anchor -(`job_configuration` again). The expanded version of the example above is: +given hash into the current one," and `*` includes the named anchor +(`job_configuration` again). The expanded version of this example is: ```yaml .job_template: @@ -4555,7 +4740,7 @@ Use [YAML anchors](#anchors) with `variables` to repeat assignment of variables across multiple jobs. You can also use YAML anchors when a job requires a specific `variables` block that would otherwise override the global variables. -In the example below, we override the `GIT_STRATEGY` variable without affecting +The following example shows how override the `GIT_STRATEGY` variable without affecting the use of the `SAMPLE_VARIABLE` variable: ```yaml @@ -4575,7 +4760,7 @@ job_no_git_strategy: ### Hide jobs -If you want to temporarily 'disable' a job, rather than commenting out all the +If you want to temporarily disable a job, rather than commenting out all the lines where the job is defined: ```yaml @@ -4594,7 +4779,7 @@ GitLab CI/CD. In the following example, `.hidden_job` is ignored: ``` Use this feature to ignore jobs, or use the -[special YAML features](#special-yaml-features) and transform the hidden jobs +[YAML-specific features](#yaml-specific-features) and transform the hidden jobs into templates. ### `!reference` tags @@ -4606,7 +4791,7 @@ sections and reuse it in the current section. Unlike [YAML anchors](#anchors), y use `!reference` tags to reuse configuration from [included](#include) configuration files as well. -In this example, a `script` and an `after_script` from two different locations are +In the following example, a `script` and an `after_script` from two different locations are reused in the `test` job: - `setup.yml`: @@ -4635,7 +4820,7 @@ reused in the `test` job: - !reference [.teardown, after_script] ``` -In this example, `test-vars-1` reuses the all the variables in `.vars`, while `test-vars-2` +In the following example, `test-vars-1` reuses the all the variables in `.vars`, while `test-vars-2` selects a specific variable and reuses it as a new `MY_VAR` variable. ```yaml @@ -4699,7 +4884,7 @@ Defining `image`, `services`, `cache`, `before_script`, and `after_script` globally is deprecated. Support could be removed from a future release. -Use [`default:`](#global-defaults) instead. For example: +Use [`default:`](#custom-default-keyword-values) instead. For example: ```yaml default: diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md index 765b982dbeb..851c9776c45 100644 --- a/doc/ci/yaml/gitlab_ci_yaml.md +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -27,6 +27,7 @@ In the `.gitlab-ci.yml` file, you can define: The scripts are grouped into **jobs**, and jobs run as part of a larger **pipeline**. You can group multiple independent jobs into **stages** that run in a defined order. +The CI/CD configuration needs at least one job that is [not hidden](README.md#hide-jobs). You should organize your jobs in a sequence that suits your application and is in accordance with the tests you wish to perform. To [visualize](../pipeline_editor/index.md#visualize-ci-configuration) the process, imagine @@ -84,7 +85,7 @@ displayed by GitLab:  If anything goes wrong, you can -[roll back](../environments/index.md#retrying-and-rolling-back) the changes: +[roll back](../environments/index.md#retry-or-roll-back-a-deployment) the changes:  diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index 5750fe1ba61..4fc52995fc1 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -121,10 +121,10 @@ job: - echo -e "\e[31mThis text is red,\e[0m but this text isn't\e[31m however this text is red again." ``` -You can define the color codes in Shell variables, or even [custom environment variables](../variables/README.md#custom-cicd-variables), +You can define the color codes in Shell environment variables, or even [custom CI/CD variables](../variables/README.md#custom-cicd-variables), which makes the commands easier to read and reusable. -For example, using the same example as above and variables defined in a `before_script`: +For example, using the same example as above and environment variables defined in a `before_script`: ```yaml job: @@ -145,3 +145,34 @@ job: - Write-Host $TXT_RED"This text is red,"$TXT_CLEAR" but this text isn't"$TXT_RED" however this text is red again." - Write-Host "This text is not colored" ``` + +## Troubleshooting + +### `Syntax is incorrect` in scripts that use `:` + +If you use a colon (`:`) in a script, GitLab might output: + +- `Syntax is incorrect` +- `script config should be a string or a nested array of strings up to 10 levels deep` + +For example, if you use `"PRIVATE-TOKEN: ${PRIVATE_TOKEN}"` as part of a cURL command: + +```yaml +pages-job: + stage: deploy + script: + - curl --header 'PRIVATE-TOKEN: ${PRIVATE_TOKEN}' "https://gitlab.example.com/api/v4/projects" +``` + +The YAML parser thinks the `:` defines a YAML keyword, and outputs the +`Syntax is incorrect` error. + +To use commands that contain a colon, you should wrap the whole command +in single quotes. You might need to change existing single quotes (`'`) into double quotes (`"`): + +```yaml +pages-job: + stage: deploy + script: + - 'curl --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "https://gitlab.example.com/api/v4/projects"' +``` |
