diff options
| author | Robert Speicher <rspeicher@gmail.com> | 2021-01-20 13:34:23 -0600 |
|---|---|---|
| committer | Robert Speicher <rspeicher@gmail.com> | 2021-01-20 13:34:23 -0600 |
| commit | 6438df3a1e0fb944485cebf07976160184697d72 (patch) | |
| tree | 00b09bfd170e77ae9391b1a2f5a93ef6839f2597 /doc/ci | |
| parent | 42bcd54d971da7ef2854b896a7b34f4ef8601067 (diff) | |
| download | gitlab-ce-6438df3a1e0fb944485cebf07976160184697d72.tar.gz | |
Add latest changes from gitlab-org/gitlab@13-8-stable-eev13.8.0-rc42
Diffstat (limited to 'doc/ci')
58 files changed, 1054 insertions, 1080 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 0bd97bcb425..740be7d1dbd 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -73,7 +73,7 @@ to your needs:  -While building your `.gitlab-ci.yml`, you can use the [CI/CD configuration visualization](yaml/visualization.md) to facilitate your writing experience. +While building your `.gitlab-ci.yml`, you can use the [CI/CD configuration visualization](pipeline_editor/index.md#visualize-ci-configuration) to facilitate your writing experience. For a broader overview, see the [CI/CD getting started](quick_start/README.md) guide. @@ -141,8 +141,8 @@ Its feature set is listed on the table below according to DevOps stages. | **Release** | | | [Auto Deploy](../topics/autodevops/stages.md#auto-deploy) | Deploy your application to a production environment in a Kubernetes cluster. | | [Building Docker images](docker/using_docker_build.md) | Maintain Docker-based projects using GitLab CI/CD. | -| [Canary Deployments](../user/project/canary_deployments.md) **(PREMIUM)** | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | -| [Deploy Boards](../user/project/deploy_boards.md) **(PREMIUM)** | Check the current health and status of each CI/CD environment running on Kubernetes. | +| [Canary Deployments](../user/project/canary_deployments.md) | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | +| [Deploy Boards](../user/project/deploy_boards.md) | Check the current health and status of each CI/CD environment running on Kubernetes. | | [Feature Flags](../operations/feature_flags.md) **(PREMIUM)** | Deploy your features behind Feature Flags. | | [GitLab Pages](../user/project/pages/index.md) | Deploy static websites. | | [GitLab Releases](../user/project/releases/index.md) | Add release notes to Git tags. | @@ -221,11 +221,3 @@ been necessary. These are: #### 10.0 - No breaking changes. - -#### 9.0 - -- [CI variables renaming for GitLab 9.0](variables/deprecated_variables.md#gitlab-90-renamed-variables). Read about the - deprecated CI variables and what you should use for GitLab 9.0+. -- [New CI job permissions model](../user/project/new_ci_build_permissions_model.md). - See what changed in GitLab 8.12 and how that affects your jobs. - There's a new way to access your Git submodules and LFS objects in jobs. diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index e8b22a24017..08a45714de3 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -316,6 +316,37 @@ rspec: - rspec spec ``` +If you have jobs that each need a different selection of gems, use the `prefix` +keyword in the global `cache` definition. This configuration generates a different +cache for each job. + +For example, a testing job might not need the same gems as a job that deploys to +production: + +```yaml +cache: + key: + files: + - Gemfile.lock + prefix: ${CI_JOB_NAME} + paths: + - vendor/ruby + +test_job: + stage: test + before_script: + - bundle install --without production --path vendor/ruby + script: + - bundle exec rspec + +deploy_job: + stage: production + before_script: + - bundle install --without test --path vendor/ruby + script: + - bundle exec deploy +``` + ### Caching Go dependencies Assuming your project is using [Go Modules](https://github.com/golang/go/wiki/Modules) to install diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 4706180688a..0be86527cb5 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -11,10 +11,9 @@ Interacting with a major cloud provider may have become a much needed task that' part of your delivery process. With GitLab you can [deploy your application anywhere](https://about.gitlab.com/stages-devops-lifecycle/deploy-targets/). -For some specific deployment targets, GitLab makes this process less painful by providing Docker images -that come with the needed libraries and tools pre-installed. -By referencing them in your CI/CD pipeline, you'll be able to interact with your chosen -cloud provider more easily. +For some specific deployment targets, GitLab makes this process less painful by providing Docker +images with the needed libraries and tools pre-installed. By referencing them in your +CI/CD pipeline, you can interact with your chosen cloud provider more easily. ## AWS @@ -36,7 +35,7 @@ Some credentials are required to be able to run `aws` commands: 1. Select your newly created user to access its details. Navigate to **Security credentials > Create a new access key**. NOTE: - A new **Access key ID** and **Secret access key** pair will be generated. Please take a note of them right away. + A new **Access key ID** and **Secret access key** are generated. Please take a note of them right away. 1. In your GitLab project, go to **Settings > CI / CD**. Set the following as [environment variables](../variables/README.md#gitlab-cicd-environment-variables) @@ -170,14 +169,14 @@ After you have these prerequisites ready, follow these steps: 1. Commit and push your updated `.gitlab-ci.yml` to your project's repository, and you're done! - Your application Docker image will be rebuilt and pushed to the GitLab registry. + Your application Docker image is rebuilt and pushed to the GitLab registry. If your image is located in a private registry, make sure your task definition is [configured with a `repositoryCredentials` attribute](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html). - Then the targeted task definition will be updated with the location of the new - Docker image, and a new revision will be created in ECS as result. + Then the targeted task definition is updated with the location of the new + Docker image, and a new revision is created in ECS as result. - Finally, your AWS ECS service will be updated with the new revision of the + Finally, your AWS ECS service is updated with the new revision of the task definition, making the cluster pull the newest version of your application. @@ -190,7 +189,7 @@ and [`Jobs/Deploy/ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blo used along with the main template. They may move or change unexpectedly causing your pipeline to fail if you didn't include the main template. Also, the job names within these templates may change. Do not override these jobs names in your own pipeline, -as the override will stop working when the name changes. +as the override stops working when the name changes. Alternatively, if you don't wish to use the `AWS/Deploy-ECS.gitlab-ci.yml` template to deploy to AWS ECS, you can always use our diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 8e5ce2fb359..af88a006156 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -7,17 +7,17 @@ type: concepts, howto # Building Docker images with GitLab CI/CD -GitLab CI/CD allows you to use Docker Engine to build and test Docker-based projects. +You can use GitLab CI/CD with Docker Engine to build and test Docker-based projects. -One of the new trends in Continuous Integration/Deployment is to: +For example, you might want to: 1. Create an application image. 1. Run tests against the created image. 1. Push image to a remote registry. 1. Deploy to a server from the pushed image. -It's also useful when your application already has the `Dockerfile` that can be -used to create and test an image: +Or, if your application already has a `Dockerfile`, you can +use it to create and test an image: ```shell docker build -t my-image dockerfiles/ @@ -26,29 +26,37 @@ docker tag my-image my-registry:5000/my-image docker push my-registry:5000/my-image ``` -This requires special configuration of GitLab Runner to enable `docker` support -during jobs. +To run Docker commands in your CI/CD jobs, you must configure +GitLab Runner to enable `docker` support. -## Runner Configuration +## Enable Docker commands in your CI/CD jobs -There are three methods to enable the use of `docker build` and `docker run` -during jobs, each with their own tradeoffs. +There are three ways to enable the use of `docker build` and `docker run` +during jobs, each with their own tradeoffs. You can use: -An alternative to using `docker build` is to [use kaniko](using_kaniko.md). -This avoids having to execute a runner in privileged mode. +- [The shell executor](#use-the-shell-executor) +- [The Docker executor with the Docker image (Docker-in-Docker)](#use-the-docker-executor-with-the-docker-image-docker-in-docker) +- [Docker socket binding](#use-docker-socket-binding) -NOTE: -To see how Docker and GitLab Runner are configured for shared runners on -GitLab.com, see [GitLab.com shared -runners](../../user/gitlab_com/index.md#shared-runners). +If you don't want to execute a runner in privileged mode, +but want to use `docker build`, you can also [use kaniko](using_kaniko.md). + +If you are using shared runners on GitLab.com, see +[GitLab.com shared runners](../../user/gitlab_com/index.md#shared-runners) +to learn more about how these runners are configured. -### Use shell executor +### Use the shell executor -The simplest approach is to install GitLab Runner in `shell` execution mode. -GitLab Runner then executes job scripts as the `gitlab-runner` user. +One way to configure GitLab Runner for `docker` support is to use the +`shell` executor. -1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/#installation). -1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: +After you register a runner and select the `shell` executor, +your job scripts are executed as the `gitlab-runner` user. +This user needs permission to run Docker commands. + +1. [Install](https://gitlab.com/gitlab-org/gitlab-runner/#installation) GitLab Runner. +1. [Register](https://docs.gitlab.com/runner/register/) a runner. + Select the `shell` executor. For example: ```shell sudo gitlab-runner register -n \ @@ -58,12 +66,10 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. --description "My Runner" ``` -1. Install Docker Engine on server. - - For more information how to install Docker Engine on different systems, - check out the [Supported installations](https://docs.docker.com/engine/installation/). +1. On the server where GitLab Runner is installed, install Docker Engine. + View a list of [supported platforms](https://docs.docker.com/engine/installation/). -1. Add `gitlab-runner` user to `docker` group: +1. Add the `gitlab-runner` user to the `docker` group: ```shell sudo usermod -aG docker gitlab-runner @@ -75,7 +81,7 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. sudo -u gitlab-runner -H docker info ``` - You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: +1. In GitLab, to verify that everything works, add `docker info` to `.gitlab-ci.yml`: ```yaml before_script: @@ -87,28 +93,30 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. - docker run my-docker-image /script/to/run/tests ``` -1. You can now use `docker` command (and **install** `docker-compose` if needed). +You can now use `docker` commands (and install `docker-compose` if needed). + +When you add `gitlab-runner` to the `docker` group, you are effectively granting `gitlab-runner` full root permissions. +Learn more about the [security of the `docker` group](https://blog.zopyx.com/on-docker-security-docker-group-considered-harmful/). -By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. -For more information please read [On Docker security: `docker` group considered harmful](https://blog.zopyx.com/on-docker-security-docker-group-considered-harmful/). +### Use the Docker executor with the Docker image (Docker-in-Docker) -### Use Docker-in-Docker workflow with Docker executor +Another way to configure GitLab Runner for `docker` support is to +register a runner with the Docker executor and use the [Docker image](https://hub.docker.com/_/docker/) +to run your job scripts. This configuration is referred to as "Docker-in-Docker." -The second approach is to use the special Docker-in-Docker (dind) -[Docker image](https://hub.docker.com/_/docker/) with all tools installed -(`docker`) and run the job script in context of that -image in privileged mode. +The Docker image has all of the `docker` tools installed +and can run the job script in context of the image in privileged mode. -`docker-compose` is not part of Docker-in-Docker (dind). To use `docker-compose` in your -CI builds, follow the `docker-compose` +The `docker-compose` command is not available in this configuration by default. +To use `docker-compose` in your job scripts, follow the `docker-compose` [installation instructions](https://docs.docker.com/compose/install/). WARNING: -By enabling `--docker-privileged`, you are effectively disabling all of +When you enable `--docker-privileged`, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check out the official Docker documentation on -[Runtime privilege and Linux capabilities](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). +[runtime privilege and Linux capabilities](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges: @@ -363,10 +371,11 @@ build: - docker run my-docker-image /script/to/run/tests ``` -#### Use Docker socket binding +### Use Docker socket binding -The third approach is to bind-mount `/var/run/docker.sock` into the -container so that Docker is available in the context of that image. +Another way to configure GitLab Runner for `docker` support is to +bind-mount `/var/run/docker.sock` into the +container so that Docker is available in the context of the image. NOTE: If you bind the Docker socket and you are @@ -395,7 +404,7 @@ To make Docker available in the context of the image: commands are siblings of the runner rather than children of the runner.** This may have complications and limitations that are unsuitable for your workflow. - Your `config.toml` file should not have an entry like this: + Your `config.toml` file should now have an entry like this: ```toml [[runners]] @@ -854,13 +863,13 @@ After you've built a Docker image, you can push it up to the built-in ### `docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?` This is a common error when you are using -[Docker in Docker](#use-docker-in-docker-workflow-with-docker-executor) +[Docker in Docker](#use-the-docker-executor-with-the-docker-image-docker-in-docker) v19.03 or higher. This occurs because Docker starts on TLS automatically, so you need to do some setup. If: - This is the first time setting it up, carefully read - [using Docker in Docker workflow](#use-docker-in-docker-workflow-with-docker-executor). + [using Docker in Docker workflow](#use-the-docker-executor-with-the-docker-image-docker-in-docker). - You are upgrading from v18.09 or earlier, read our [upgrade guide](https://about.gitlab.com/releases/2019/07/31/docker-in-docker-with-docker-19-dot-03/). diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 7171269cf2d..630e106b72c 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -10,18 +10,18 @@ type: concepts, howto GitLab CI/CD in conjunction with [GitLab Runner](../runners/README.md) can use [Docker Engine](https://www.docker.com/) to test and build any application. -Docker is an open-source project that allows you to use predefined images to -run applications in independent "containers" that are run within a single Linux -instance. [Docker Hub](https://hub.docker.com/) has a rich database of pre-built images that can be -used to test and build your applications. - -When used with GitLab CI/CD, Docker runs each job in a separate and isolated -container using the predefined image that's set up in -[`.gitlab-ci.yml`](../yaml/README.md). - -This makes it easier to have a simple and reproducible build environment that -can also run on your workstation. The added benefit is that you can test all -the commands that we explore later from your shell, rather than having to +Docker is an open-source project that has predefined images you can use to +run applications in independent "containers." These containers run in a single Linux +instance. [Docker Hub](https://hub.docker.com/) is a database of pre-built images you can +use to test and build your applications. + +When you use Docker with GitLab CI/CD, Docker runs each job in a separate and isolated +container. You specify the container image in the project's +[`.gitlab-ci.yml`](../yaml/README.md) file. + +Docker containers provide a reproducible build environment that +can run on your workstation. When a Docker container is running, you can test +commands from your shell, rather than having to test them on a dedicated CI server. ## Register Docker Runner @@ -29,7 +29,7 @@ test them on a dedicated CI server. To use GitLab Runner with Docker you need to [register a new runner](https://docs.gitlab.com/runner/register/) to use the `docker` executor. -An example can be seen below. First we set up a temporary template to supply the services: +In this example, we first set up a temporary template to supply the services: ```shell cat > /tmp/test-config.template.toml << EOF @@ -42,7 +42,7 @@ name = "mysql:latest" EOF ``` -Then we register the runner using the template that was just created: +Then use this template to register the runner: ```shell sudo gitlab-runner register \ @@ -63,25 +63,26 @@ accessible during the build process. The `image` keyword is the name of the Docker image the Docker executor runs to perform the CI tasks. -By default, the executor only pulls images from [Docker Hub](https://hub.docker.com/), -however this can be configured in the `gitlab-runner/config.toml` by setting -the [Docker pull policy](https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work) to allow using local images. +By default, the executor pulls images only from [Docker Hub](https://hub.docker.com/). +However, you can configure the location in the `gitlab-runner/config.toml` file. For example, +you can set the [Docker pull policy](https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work) +to use local images. -For more information about images and Docker Hub, please read +For more information about images and Docker Hub, read the [Docker Fundamentals](https://docs.docker.com/engine/understanding-docker/) documentation. ## What is a service -The `services` keyword defines just another Docker image that's run during -your job and is linked to the Docker image that the `image` keyword defines. -This allows you to access the service image during build time. +The `services` keyword defines another Docker image that's run during +your job. It's linked to the Docker image that the `image` keyword defines, +which allows you to access the service image during build time. The service image can run any application, but the most common use case is to run a database container, for example, `mysql`. It's easier and faster to use an -existing image and run it as an additional container than install `mysql` every +existing image and run it as an additional container than to install `mysql` every time the project is built. -You're not limited to have only database services. You can add as many +You're not limited to only database services. You can add as many services you need to `.gitlab-ci.yml` or manually modify `config.toml`. Any image found at [Docker Hub](https://hub.docker.com/) or your private Container Registry can be used as a service. @@ -94,56 +95,57 @@ You can see some widely used services examples in the relevant documentation of ### How services are linked to the job -To better understand how the container linking works, read +To better understand how container linking works, read [Linking containers together](https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/). -To summarize, if you add `mysql` as service to your application, the image is -then used to create a container that's linked to the job container. +If you add `mysql` as service to your application, the image is +used to create a container that's linked to the job container. The service container for MySQL is accessible under the hostname `mysql`. -So, in order to access your database service you have to connect to the host -named `mysql` instead of a socket or `localhost`. Read more in [accessing the -services](#accessing-the-services). +To access your database service, connect to the host named `mysql` instead of a +socket or `localhost`. Read more in [accessing the services](#accessing-the-services). ### How the health check of services works -Services are designed to provide additional functionality which is **network accessible**. -It may be a database like MySQL, or Redis, and even `docker:stable-dind` which -allows you to use Docker in Docker. It can be practically anything that's -required for the CI/CD job to proceed and is accessed by network. +Services are designed to provide additional features which are **network accessible**. +They may be a database like MySQL, or Redis, and even `docker:stable-dind` which +allows you to use Docker-in-Docker. It can be practically anything that's +required for the CI/CD job to proceed, and is accessed by network. To make sure this works, the runner: 1. Checks which ports are exposed from the container by default. 1. Starts a special container that waits for these ports to be accessible. -When the second stage of the check fails, either because there is no opened port in the -service, or the service was not started properly before the timeout and the port is not -responding, it prints the warning: `*** WARNING: Service XYZ probably didn't start properly`. +If the second stage of the check fails, it prints the warning: `*** WARNING: Service XYZ probably didn't start properly`. +This issue can occur because: + +- There is no opened port in the service. +- The service was not started properly before the timeout, and the port is not + responding. In most cases it affects the job, but there may be situations when the job still succeeds even if that warning was printed. For example: -- The service was started a little after the warning was raised, and the job is +- The service was started shortly after the warning was raised, and the job is not using the linked service from the beginning. In that case, when the job needed to access the service, it may have been already there waiting for connections. - The service container is not providing any networking service, but it's doing something with the job's directory (all services have the job directory mounted as a volume under `/builds`). In that case, the service does its job, and - since the job is not trying to connect to it, it does not fail. + because the job is not trying to connect to it, it does not fail. ### What services are not for -As it was mentioned before, this feature is designed to provide **network accessible** +As mentioned before, this feature is designed to provide **network accessible** services. A database is the simplest example of such a service. -The services feature is not designed to, and does not add any software from the +The services feature is not designed to, and does not, add any software from the defined `services` image(s) to the job's container. For example, if you have the following `services` defined in your job, the `php`, -`node` or `go` commands are **not** available for your script, and thus -the job fails: +`node` or `go` commands are **not** available for your script, and the job fails: ```yaml job: @@ -162,16 +164,15 @@ If you need to have `php`, `node` and `go` available for your script, you should either: - Choose an existing Docker image that contains all required tools. -- Create your own Docker image, with all the required tools included +- Create your own Docker image, with all the required tools included, and use that in your job. ### Accessing the services Let's say that you need a Wordpress instance to test some API integration with -your application. - -You can then use for example the [tutum/wordpress](https://hub.docker.com/r/tutum/wordpress/) image in your -`.gitlab-ci.yml`: +your application. You can then use for example the +[`tutum/wordpress`](https://hub.docker.com/r/tutum/wordpress/) image in your +`.gitlab-ci.yml` file: ```yaml services: @@ -179,13 +180,13 @@ services: ``` If you don't [specify a service alias](#available-settings-for-services), -when the job is run, `tutum/wordpress` is started and you have -access to it from your build container under two hostnames to choose from: +when the job runs, `tutum/wordpress` is started. You have +access to it from your build container under two hostnames: - `tutum-wordpress` - `tutum__wordpress` -Hostnames with underscores are not RFC valid and may cause problems in 3rd party +Hostnames with underscores are not RFC valid and may cause problems in third-party applications. The default aliases for the service's hostname are created from its image name @@ -202,7 +203,7 @@ To override the default behavior, you can ## Define `image` and `services` from `.gitlab-ci.yml` -You can simply define an image that's used for all jobs and a list of +You can define an image that's used for all jobs, and a list of services that you want to use during build time: ```yaml @@ -275,7 +276,7 @@ test: You can also pass custom environment [variables](../variables/README.md) to fine tune your Docker `images` and `services` directly in the `.gitlab-ci.yml` file. -For more information, see [custom environment variables](../variables/README.md#gitlab-ciyml-defined-variables) +For more information, read [custom environment variables](../variables/README.md#gitlab-ciyml-defined-variables) ```yaml # The following variables are automatically passed down to the Postgres container @@ -314,11 +315,11 @@ test: When configuring the `image` or `services` entries, you can use a string or a map as options: -- when using a string as an option, it must be the full name of the image to use +- When using a string as an option, it must be the full name of the image to use (including the Registry part if you want to download the image from a Registry - other than Docker Hub) -- when using a map as an option, then it must contain at least the `name` - option, which is the same name of the image as used for the string setting + other than Docker Hub). +- When using a map as an option, then it must contain at least the `name` + option, which is the same name of the image as used for the string setting. For example, the following two definitions are equal: @@ -350,8 +351,8 @@ For example, the following two definitions are equal: | Setting | Required | GitLab version | Description | |------------|----------|----------------| ----------- | -| `name` | yes, when used with any other option | 9.4 |Full name of the image that should be used. It should contain the Registry part if needed. | -| `entrypoint` | no | 9.4 |Command or script that should be executed as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. | +| `name` | yes, when used with any other option | 9.4 |Full name of the image to use. It should contain the Registry part if needed. | +| `entrypoint` | no | 9.4 |Command or script to execute as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. | ### Available settings for `services` @@ -359,8 +360,8 @@ For example, the following two definitions are equal: | Setting | Required | GitLab version | Description | |------------|----------|----------------| ----------- | -| `name` | yes, when used with any other option | 9.4 | Full name of the image that should be used. It should contain the Registry part if needed. | -| `entrypoint` | no | 9.4 |Command or script that should be executed as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. | +| `name` | yes, when used with any other option | 9.4 | Full name of the image to use. It should contain the Registry part if needed. | +| `entrypoint` | no | 9.4 |Command or script to execute as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. | | `command` | no | 9.4 |Command or script that should be used as the container's command. It's translated to arguments passed to Docker after the image's name. The syntax is similar to [`Dockerfile`'s `CMD`](https://docs.docker.com/engine/reference/builder/#cmd) directive, where each shell token is a separate string in the array. | | `alias` (1) | no | 9.4 |Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. | @@ -379,8 +380,8 @@ services: - mysql:latest ``` -The runner would start two containers using the `mysql:latest` image, but both -of them would be added to the job's container with the `mysql` alias based on +The runner would start two containers, each that uses the `mysql:latest` image. +However, both of them would be added to the job's container with the `mysql` alias, based on the [default hostname naming](#accessing-the-services). This would end with one of the services not being accessible. @@ -404,31 +405,33 @@ in `.gitlab-ci.yml` file. > Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended configuration options](#extended-docker-configuration-options). Let's assume you have a `super/sql:latest` image with some SQL database -inside it and you would like to use it as a service for your job. Let's also +in it. You would like to use it as a service for your job. Let's also assume that this image does not start the database process while starting -the container and the user needs to manually use `/usr/bin/super-sql run` as +the container. The user needs to manually use `/usr/bin/super-sql run` as a command to start the database. -Before the new extended Docker configuration options, you would need to create -your own image based on the `super/sql:latest` image, add the default command, -and then use it in job's configuration, like: +Before the new extended Docker configuration options, you would need to: -```dockerfile -# my-super-sql:latest image's Dockerfile +- Create your own image based on the `super/sql:latest` image. +- Add the default command. +- Use the image in the job's configuration: -FROM super/sql:latest -CMD ["/usr/bin/super-sql", "run"] -``` + ```dockerfile + # my-super-sql:latest image's Dockerfile -```yaml -# .gitlab-ci.yml + FROM super/sql:latest + CMD ["/usr/bin/super-sql", "run"] + ``` -services: - - my-super-sql:latest -``` + ```yaml + # .gitlab-ci.yml + + services: + - my-super-sql:latest + ``` -After the new extended Docker configuration options, you can now simply -set a `command` in `.gitlab-ci.yml`, like: +After the new extended Docker configuration options, you can +set a `command` in the `.gitlab-ci.yml` file instead: ```yaml # .gitlab-ci.yml @@ -438,15 +441,15 @@ services: command: ["/usr/bin/super-sql", "run"] ``` -As you can see, the syntax of `command` is similar to [Dockerfile's `CMD`](https://docs.docker.com/engine/reference/builder/#cmd). +The syntax of `command` is similar to [Dockerfile's `CMD`](https://docs.docker.com/engine/reference/builder/#cmd). ### Overriding the entrypoint of an image > Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended configuration options](#extended-docker-configuration-options). -Before showing the available entrypoint override methods, let's describe shortly -how the runner starts and uses a Docker image for the containers used in the -CI jobs: +Before showing the available entrypoint override methods, let's describe +how the runner starts. It uses a Docker image for the containers used in the +CI/CD jobs: 1. The runner starts a Docker container using the defined entrypoint (default from `Dockerfile` that may be overridden in `.gitlab-ci.yml`) @@ -455,32 +458,35 @@ CI jobs: [`before_script`](../yaml/README.md#before_script), [`script`](../yaml/README.md#script), and [`after_script`](../yaml/README.md#after_script)). -1. The runner sends the script to the container's shell STDIN and receives the +1. The runner sends the script to the container's shell `stdin` and receives the output. -To override the entrypoint of a Docker image, the recommended solution is to +To override the entrypoint of a Docker image, you should define an empty `entrypoint` in `.gitlab-ci.yml`, so the runner does not start a useless shell layer. However, that does not work for all Docker versions, and -you should check which one your runner is using. Specifically: +you should check which one your runner is using: -- If Docker 17.06 or later is used, the `entrypoint` can be set to an empty value. -- If Docker 17.03 or previous versions are used, the `entrypoint` can be set to +- _If Docker 17.06 or later is used,_ the `entrypoint` can be set to an empty value. +- _If Docker 17.03 or previous versions are used,_ the `entrypoint` can be set to `/bin/sh -c`, `/bin/bash -c` or an equivalent shell available in the image. The syntax of `image:entrypoint` is similar to [Dockerfile's `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint). -Let's assume you have a `super/sql:experimental` image with some SQL database -inside it and you would like to use it as a base image for your job because you +Let's assume you have a `super/sql:experimental` image with a SQL database +in it. You want to use it as a base image for your job because you want to execute some tests with this database binary. Let's also assume that -this image is configured with `/usr/bin/super-sql run` as an entrypoint. That -means that when starting the container without additional options, it runs -the database's process, while the runner expects that the image has no +this image is configured with `/usr/bin/super-sql run` as an entrypoint. When +the container starts without additional options, it runs +the database's process. The runner expects that the image has no entrypoint or that the entrypoint is prepared to start a shell command. -With the extended Docker configuration options, instead of creating your -own image based on `super/sql:experimental`, setting the `ENTRYPOINT` -to a shell, and then using the new image in your CI job, you can now simply -define an `entrypoint` in `.gitlab-ci.yml`. +With the extended Docker configuration options, instead of: + +- Creating your own image based on `super/sql:experimental`. +- Setting the `ENTRYPOINT` to a shell. +- Using the new image in your CI job. + +You can now define an `entrypoint` in the `.gitlab-ci.yml` file. **For Docker 17.06+:** @@ -508,7 +514,7 @@ Look for the `[runners.docker]` section: services = ["mysql:latest", "postgres:latest"] ``` -The image and services defined this way are added to all job run by +The image and services defined this way are added to all jobs run by that runner. ## Define an image from a private Container Registry @@ -516,8 +522,8 @@ that runner. To access private container registries, the GitLab Runner process can use: - [Statically defined credentials](#using-statically-defined-credentials). That is, a username and password for a specific registry. -- [Credentials Store](#using-credentials-store). For more information, see [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credentials-store). -- [Credential Helpers](#using-credential-helpers). For more information, see [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers). +- [Credentials Store](#using-credentials-store). For more information, read [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credentials-store). +- [Credential Helpers](#using-credential-helpers). For more information, read [the relevant Docker documentation](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers). To define which should be used, the GitLab Runner process reads the configuration in the following order: @@ -531,7 +537,7 @@ To define which should be used, the GitLab Runner process reads the configuratio GitLab Runner reads this configuration **only** from `config.toml` and ignores it if it's provided as an environment variable. This is because GitLab Runner uses **only** -`config.toml` configuration and does not interpolate **ANY** environment variables at +`config.toml` configuration and does not interpolate **any** environment variables at runtime. ### Requirements and limitations @@ -543,7 +549,7 @@ runtime. at least version **1.8** if you want to use private registries. - Available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html) in GitLab Runner 13.1 and later. -- [Credentials Store](#using-credentials-store) and [Credential Helpers](#using-credential-helpers) require binaries to be added to the GitLab Runner's `$PATH`, and require access to do so. Therefore, these features are not available on shared runners or any other runner where the user does not have access to the environment where the runner is installed. +- [Credentials Store](#using-credentials-store) and [Credential Helpers](#using-credential-helpers) require binaries to be added to the GitLab Runner's `$PATH`, and require access to do so. Therefore, these features are not available on shared runners, or any other runner where the user does not have access to the environment where the runner is installed. ### Using statically-defined credentials @@ -562,7 +568,7 @@ See below for examples of each. #### Determining your `DOCKER_AUTH_CONFIG` data As an example, let's assume you want to use the `registry.example.com:5000/private/image:latest` -image, which is private and requires you to sign in to a private container +image. This image is private and requires you to sign in to a private container registry. Let's also assume that these are the sign-in credentials: @@ -592,7 +598,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`: - In some setups, it's possible that Docker client uses the available system key store to store the result of `docker login`. In that case, it's impossible to - read `~/.docker/config.json`, so you need to prepare the required + read `~/.docker/config.json`, so you must prepare the required base64-encoded version of `${username}:${password}` and create the Docker configuration JSON manually. Open a terminal and execute the following command: @@ -691,7 +697,7 @@ To add `DOCKER_AUTH_CONFIG` to a runner: To configure credentials store, follow these steps: 1. To use a credentials store, you need an external helper program to interact with a specific keychain or external store. - Make sure helper program is available in GitLab Runner `$PATH`. + Make sure the helper program is available in GitLab Runner `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: @@ -710,16 +716,16 @@ To configure credentials store, follow these steps: `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner reads this configuration file and uses the needed helper for this specific repository. -`credsStore` is used to access ALL the registries. -If you want to use both images from private registry and public images from Docker Hub, -pulling from Docker Hub would fail, because Docker daemon tries to use the same credentials for **ALL** the registries. +`credsStore` is used to access **all** the registries. +If you use both images from a private registry and public images from Docker Hub, +pulling from Docker Hub fails. Docker daemon tries to use the same credentials for **all** the registries. ### Using Credential Helpers > Support for using Credential Helpers was added in GitLab Runner 12.0 As an example, let's assume that you want to use the `aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest` -image which is private and requires you to log in into a private container registry. +image. This image is private and requires you to log in into a private container registry. To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow these steps: @@ -750,7 +756,7 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th } ``` - This configures Docker to use the credential helper for all Amazon ECR registries. + This configures Docker to use the credential helper for all Amazon Elastic Container Registry (ECR) registries. - Or, if you're running self-managed runners, add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. @@ -772,13 +778,13 @@ registries to the `"credHelpers"` hash as described above. ## Configuring services -Many services accept environment variables which allow you to easily change -database names or set account names depending on the environment. +Many services accept environment variables, which you can use to change +database names or set account names, depending on the environment. GitLab Runner 0.5.0 and up passes all YAML-defined variables to the created service containers. -For all possible configuration variables check the documentation of each image +For all possible configuration variables, check the documentation of each image provided in their corresponding Docker hub page. All variables are passed to all services containers. It's not @@ -786,12 +792,12 @@ designed to distinguish which variable should go where. ### PostgreSQL service example -See the specific documentation for +Read the specific documentation for [using PostgreSQL as a service](../services/postgres.md). ### MySQL service example -See the specific documentation for +Read the specific documentation for [using MySQL as a service](../services/mysql.md). ## How Docker integration works @@ -800,15 +806,15 @@ Below is a high level overview of the steps performed by Docker during job time. 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. -1. Create cache container to store all volumes as defined in `config.toml` and +1. Create a cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.6` as in above example). -1. Create build container and link any service container to build container. -1. Start build container and send job script to the container. -1. Run job script. +1. Create a build container and link any service container to build container. +1. Start the build container, and send a job script to the container. +1. Run the job script. 1. Checkout code in: `/builds/group-name/project-name/`. 1. Run any step defined in `.gitlab-ci.yml`. -1. Check exit status of build script. -1. Remove build container and all created service containers. +1. Check the exit status of build script. +1. Remove the build container and all created service containers. ## How to debug a job locally @@ -827,8 +833,7 @@ EOF Here we use as an example the GitLab Runner repository which contains a Makefile, so running `make` executes the commands defined in the Makefile. -Your mileage may vary, so instead of `make` you could run the command which -is specific to your project. +Instead of `make`, you could run the command which is specific to your project. Then create some service containers: @@ -850,7 +855,7 @@ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:po The above command creates a container named `build` that's spawned from the `ruby:2.6` image and has two services linked to it. The `build_script` is -piped using STDIN to the bash interpreter which in turn executes the +piped using `stdin` to the bash interpreter which in turn executes the `build_script` in the `build` container. When you finish testing and no longer need the containers, you can remove them @@ -861,5 +866,5 @@ docker rm -f -v build service-mysql service-postgres ``` This forcefully (`-f`) removes the `build` container, the two service -containers as well as all volumes (`-v`) that were created with the container +containers, and all volumes (`-v`) that were created with the container creation. diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 13d3c607f8a..7eb2a8286c7 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -14,7 +14,7 @@ container images from a Dockerfile, inside a container or Kubernetes cluster. kaniko solves two problems with using the [Docker-in-Docker -build](using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor) method: +build](using_docker_build.md#use-the-docker-executor-with-the-docker-image-docker-in-docker) method: - Docker-in-Docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) to function, which is a significant security concern. @@ -70,6 +70,39 @@ build: - if: $CI_COMMIT_TAG ``` +### Building an image with kaniko behind a proxy + +If you use a custom GitLab Runner behind an http(s) proxy, kaniko needs to be set +up accordingly. This means: + +- Adding the proxy to `/kaniko/.docker/config.json` +- Passing the `http_proxy` environment variables as build args so the Dockerfile + instructions can use the proxy when building the image. + +The previous example can be extended as follows: + +```yaml +build: + stage: build + image: + name: gcr.io/kaniko-project/executor:debug + entrypoint: [""] + script: + - mkdir -p /kaniko/.docker + - |- + KANIKOPROXYBUILDARGS="" + KANIKOCFG="{ \"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}" + if [ "x${http_proxy}" != "x" -o "x${https_proxy}" != "x" ]; then + KANIKOCFG="${KANIKOCFG}, \"proxies\": { \"default\": { \"httpProxy\": \"${http_proxy}\", \"httpsProxy\": \"${https_proxy}\", \"noProxy\": \"${no_proxy}\"}}" + KANIKOPROXYBUILDARGS="--build-arg http_proxy=${http_proxy} --build-arg https_proxy=${https_proxy} --build-arg no_proxy=${no_proxy}" + fi + KANIKOCFG="${KANIKOCFG} }" + echo "${KANIKOCFG}" > /kaniko/.docker/config.json + - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile $KANIKOPROXYBUILDARGS --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_TAG + only: + - tags +``` + ## Using a registry with a custom certificate When trying to push to a Docker registry that uses a certificate that is signed diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 95419e58d36..4dac076ffb7 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -14,6 +14,9 @@ You can: - [Restrict write-access to a critical environment](#restrict-write-access-to-a-critical-environment) - [Prevent deployments during deploy freeze windows](#prevent-deployments-during-deploy-freeze-windows) +- [Set appropriate roles to your project](#setting-appropriate-roles-to-your-project) +- [Protect production secrets](#protect-production-secrets) +- [Separate project for deployments](#separate-project-for-deployments) If you are using a continuous deployment workflow and want to ensure that concurrent deployments to the same environment do not happen, you should enable the following options: @@ -38,8 +41,8 @@ For example: ```yaml deploy: - script: deploy-to-prod - resource_group: prod + script: deploy-to-prod + resource_group: prod ``` Example of a problematic pipeline flow **before** using the resource group: @@ -89,6 +92,56 @@ If you want to prevent deployments for a particular period, for example during a vacation period when most employees are out, you can set up a [Deploy Freeze](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). During a deploy freeze period, no deployment can be executed. This is helpful to ensure that deployments do not happen unexpectedly. + +## Setting appropriate roles to your project + +GitLab supports several different roles that can be assigned to your project members. See +[Project members permissions](../../user/permissions.md#project-members-permissions) +for an explanation of these roles and the permissions of each. + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=Mq3C1KveDc0">How to secure your CD pipelines</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/Mq3C1KveDc0" frameborder="0" allowfullscreen="true"> </iframe> +</figure> + +## Protect production secrets + +Production secrets are needed to deploy successfully. For example, when deploying to the cloud, +cloud providers require these secrets to connect to their services. In the project settings, you can +define and protect environment variables for these secrets. [Protected variables](../variables/README.md#protect-a-custom-variable) +are only passed to pipelines running on [protected branches](../../user/project/protected_branches.md) +or [protected tags](../../user/project/protected_tags.md). +The other pipelines don't get the protected variable. You can also +[scope variables to specific environments](../variables/where_variables_can_be_used.md#variables-with-an-environment-scope). +We recommend that you use protected variables on protected environments to make sure that the +secrets aren't exposed unintentionally. You can also define production secrets on the +[runner side](../runners/README.md#prevent-runners-from-revealing-sensitive-information). +This prevents other maintainers from reading the secrets and makes sure that the runner only runs on +protected branches. + +For more information, see [pipeline security](../pipelines/index.md#pipeline-security-on-protected-branches). + +## Separate project for deployments + +All project maintainers have access to production secrets. If you need to limit the number of users +that can deploy to a production environment, you can create a separate project and configure a new +permission model that isolates the CD permissions from the original project and prevents the +original project's maintainers from accessing the production secret and CD configuration. You can +connect the CD project to your development projects by using [multi-project pipelines](../multi_project_pipelines.md). + +## Protect `gitlab-ci.yml` from change + +A `.gitlab-ci.yml` may contain rules to deploy an application to the production server. This +deployment usually runs automatically after pushing a merge request. To prevent developers from +changing the `gitlab-ci.yml`, you can define it in a different repository. The configuration can +reference a file in another project with a completely different set of permissions (similar to +[separating a project for deployments](#separate-project-for-deployments)). +In this scenario, the `gitlab-ci.yml` is publicly accessible, but can only be edited by users with +appropriate permissions in the other project. + +For more information, see [Custom CI configuration path](../pipelines/settings.md#custom-ci-configuration-path). ## Troubleshooting @@ -99,14 +152,13 @@ If you have multiple jobs for the same environment (including non-deployment job ```yaml build:service-a: - environment: - name: production - + environment: + name: production + build:service-b: - environment: - name: production + environment: + name: production ``` -The [Skip outdated deployment jobs](../pipelines/settings.md#skip-outdated-deployment-jobs) might not work well with this configuration, and will need to be disabled. - -There is a [plan to introduce a new annotation for environments](https://gitlab.com/gitlab-org/gitlab/-/issues/208655) to address this issue. +The [Skip outdated deployment jobs](../pipelines/settings.md#skip-outdated-deployment-jobs) might +not work well with this configuration, and must be disabled. diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index 81acc3a36e9..15eb4d2c526 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -44,8 +44,8 @@ allows more control over the this feature. The steps in an incremental rollout d number of pods that are defined for the deployment, which are configured when the Kubernetes cluster is created. -For example, if your application has 10 pods and a 10% rollout job is run, the new instance of the -application will be deployed to a single pod while the remaining 9 will present the previous instance. +For example, if your application has 10 pods and a 10% rollout job runs, the new instance of the +application is deployed to a single pod while the remaining nine are present the previous instance. First we [define the template as manual](https://gitlab.com/gl-release/incremental-rollout-example/blob/master/.gitlab-ci.yml#L100-103): @@ -65,7 +65,7 @@ rollout 10%: ROLLOUT_PERCENTAGE: 10 ``` -When the jobs are built, a **play** button will appear next to the job's name. Click the **play** button +When the jobs are built, a **play** button appears next to the job's name. Click the **play** button to release each stage of pods. You can also rollback by running a lower percentage job. Once 100% is reached, you cannot roll back using this method. It is still possible to roll back by redeploying the old version using the **Rollback** button on the environment page. @@ -79,13 +79,13 @@ available, demonstrating manually triggered incremental rollouts. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7545) in GitLab 11.4. -Timed rollouts behave in the same way as manual rollouts, except that each job is defined with a delay -in minutes before it will deploy. Clicking on the job will reveal the countdown. +Timed rollouts behave in the same way as manual rollouts, except that each job is defined with a +delay in minutes before it deploys. Clicking the job reveals the countdown.  -It is possible to combine this functionality with manual incremental rollouts so that the job will -countdown and then deploy. +It is possible to combine this functionality with manual incremental rollouts so that the job +counts down and then deploys. First we [define the template as timed](https://gitlab.com/gl-release/timed-rollout-example/blob/master/.gitlab-ci.yml#L86-89): diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 10513e0797e..7bf30ef1b95 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -28,7 +28,7 @@ This helps find bugs in your software, and also in the deployment process as wel GitLab CI/CD is capable of not only testing or building your projects, but also deploying them in your infrastructure, with the added benefit of giving you a -way to track your deployments. In other words, you will always know what is +way to track your deployments. In other words, you always know what is currently being deployed or has been deployed on your servers. It's important to know that: @@ -102,12 +102,12 @@ We have defined three [stages](../yaml/README.md#stages): - `build` - `deploy` -The jobs assigned to these stages will run in this order. If any job fails, then -the pipeline fails and jobs that are assigned to the next stage won't run. +The jobs assigned to these stages run in this order. If any job fails, then +the pipeline fails and jobs that are assigned to the next stage don't run. In our case: -- The `test` job will run first. +- The `test` job runs first. - Then the `build` job. - Lastly the `deploy_staging` job. @@ -127,13 +127,13 @@ numbers, spaces, and `-`, `_`, `/`, `{`, `}`, or `.`. Also, it must not start no In summary, with the above `.gitlab-ci.yml` we have achieved the following: -- All branches will run the `test` and `build` jobs. -- The `deploy_staging` job will run [only](../yaml/README.md#onlyexcept-basic) on the `master` +- All branches run the `test` and `build` jobs. +- The `deploy_staging` job runs [only](../yaml/README.md#onlyexcept-basic) on the `master` branch, which means all merge requests that are created from branches don't get deployed to the staging server. -- When a merge request is merged, all jobs will run and the `deploy_staging` - job will deploy our code to a staging server while the deployment - will be recorded in an environment named `staging`. +- When a merge request is merged, all jobs run and the `deploy_staging` + job deploys our code to a staging server while the deployment + is recorded in an environment named `staging`. #### Environment variables and runners @@ -147,8 +147,8 @@ two forms: If you change the name of an existing environment, the: -- `$CI_ENVIRONMENT_NAME` variable will be updated with the new environment name. -- `$CI_ENVIRONMENT_SLUG` variable will remain unchanged to prevent unintended side +- `$CI_ENVIRONMENT_NAME` variable is updated with the new environment name. +- `$CI_ENVIRONMENT_SLUG` variable remains unchanged to prevent unintended side effects. Starting with GitLab 9.3, the environment URL is exposed to the runner via @@ -214,13 +214,13 @@ It parses the `deploy.env` report artifact, registers a list of variables as run uses it for expanding `environment:url: $DYNAMIC_ENVIRONMENT_URL` and sets it to the environment URL. You can also specify a static part of the URL at `environment:url:`, such as `https://$DYNAMIC_ENVIRONMENT_URL`. If the value of `DYNAMIC_ENVIRONMENT_URL` is -`example.com`, the final result will be `https://example.com`. +`example.com`, the final result is `https://example.com`. The assigned URL for the `review/your-branch-name` environment is [visible in the UI](#using-the-environment-url). Note the following: -- `stop_review` doesn't generate a dotenv report artifact, so it won't recognize the +- `stop_review` doesn't generate a dotenv report artifact, so it doesn't recognize the `DYNAMIC_ENVIRONMENT_URL` variable. Therefore you shouldn't set `environment:url:` in the `stop_review` job. - If the environment URL isn't valid (for example, the URL is malformed), the system doesn't update @@ -280,7 +280,7 @@ deploy_prod: The `when: manual` action: - Exposes a "play" button in the GitLab UI for that job. -- Means the `deploy_prod` job will only be triggered when the "play" button is clicked. +- Means the `deploy_prod` job is only triggered when the "play" button is clicked. You can find the "play" button in the pipelines, environments, deployments, and jobs views. @@ -330,7 +330,7 @@ For more information, see [Where variables can be used](../variables/where_varia Runners expose various [environment variables](../variables/README.md) when a job runs, so you can use them as environment names. -In the following example, the job will deploy to all branches except `master`: +In the following example, the job deploys to all branches except `master`: ```yaml deploy_review: @@ -363,7 +363,7 @@ For the value of: may contain a `/` or other characters that would be invalid in a domain name or URL, so we use `$CI_ENVIRONMENT_SLUG` to guarantee that we get a valid URL. - For example, given a `$CI_COMMIT_REF_NAME` of `100-Do-The-Thing`, the URL will be something + For example, given a `$CI_COMMIT_REF_NAME` of `100-Do-The-Thing`, the URL is something like `https://100-do-the-4f99a2.example.com`. Again, the way you set up the web server to serve these requests is based on your setup. @@ -396,7 +396,7 @@ The following configuration options are supported: - [`namespace`](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) -In the following example, the job will deploy your application to the +In the following example, the job deploys your application to the `production` Kubernetes namespace. ```yaml @@ -414,7 +414,7 @@ deploy: ``` When deploying to a Kubernetes cluster using the GitLab Kubernetes integration, -information about the cluster and namespace will be displayed above the job +information about the cluster and namespace is displayed above the job trace on the deployment job page:  @@ -502,7 +502,7 @@ deploy_prod: A more realistic example would also include copying files to a location where a webserver (for example, NGINX) could then access and serve them. -The example below will copy the `public` directory to `/srv/nginx/$CI_COMMIT_REF_SLUG/public`: +The example below copies the `public` directory to `/srv/nginx/$CI_COMMIT_REF_SLUG/public`: ```yaml review_app: @@ -514,7 +514,7 @@ review_app: url: https://$CI_COMMIT_REF_SLUG.example.com ``` -This example requires that NGINX and GitLab Runner are set up on the server this job will run on. +This example requires that NGINX and GitLab Runner are set up on the server this job runs on. See the [limitations](#limitations) section for some edge cases regarding the naming of your branches and Review Apps. @@ -526,10 +526,10 @@ The complete example provides the following workflow to developers: - Push the branch to GitLab. - Create a merge request. -Behind the scenes, the runner will: +Behind the scenes, the runner: -- Pick up the changes and start running the jobs. -- Run the jobs sequentially as defined in `stages`: +- Picks up the changes and starts running the jobs. +- Runs the jobs sequentially as defined in `stages`: - First, run the tests. - If the tests succeed, build the app. - If the build succeeds, the app is deployed to an environment with a name specific to the @@ -561,7 +561,7 @@ A list of environments and deployment statuses is available on each project's ** For example: - + This example shows: @@ -571,10 +571,16 @@ This example shows: - The commit information of the last deployment, such as who committed it, to what branch, and the Git SHA of the commit. - The exact time the last deployment was performed. +- The upcoming deployment, if a deployment for the environment is in progress. +- When the environment stops automatically. - A button that takes you to the URL that you defined under the `environment` keyword in `.gitlab-ci.yml`. -- A button that re-deploys the latest deployment, meaning it runs the job - defined by the environment name for that specific commit. +- A number of deployment actions, including: + - Prevent the environment from [stopping automatically](#automatically-stopping-an-environment). + - [Open the live environment](#using-the-environment-url). + - Trigger [a manual deployment to a different environment](#configuring-manual-deployments). + - [Retry the deployment](#retrying-and-rolling-back). + - [Stop the environment](#stopping-an-environment). The information shown in the **Environments** page is limited to the latest deployments, but an environment can have multiple deployments. @@ -587,7 +593,7 @@ deployments, but an environment can have multiple deployments. > - The environments page can only be viewed by users with [Reporter permission](../../user/permissions.md#project-members-permissions) > and above. For more information on permissions, see the [permissions documentation](../../user/permissions.md). > - Only deploys that happen after your `.gitlab-ci.yml` is properly configured -> will show up in the **Environment** and **Last deployment** lists. +> show up in the **Environment** and **Last deployment** lists. ### Viewing deployment history @@ -619,7 +625,7 @@ To retry or rollback a deployment: #### What to expect with a rollback Pressing the **Rollback** button on a specific commit triggers a _new_ deployment with its own -unique job ID. This means that you will see a new deployment that points to the commit you're +unique job ID. This new deployment points to the commit you're rolling back to. Note that the defined deployment process in the job's `script` determines whether the rollback @@ -633,7 +639,7 @@ places within GitLab: - In a merge request widget as a link:  - In the Environments view as a button: -  +  - In the Deployments view as a button:  @@ -698,20 +704,20 @@ stop_review: If you can't use [Pipelines for merge requests](../merge_request_pipelines/index.md), setting the [`GIT_STRATEGY`](../runners/README.md#git-strategy) to `none` is necessary in the -`stop_review` job so that the [runner](https://docs.gitlab.com/runner/) won't +`stop_review` job so that the [runner](https://docs.gitlab.com/runner/) doesn't try to check out the code after the branch is deleted. When you have an environment that has a stop action defined (typically when -the environment describes a Review App), GitLab will automatically trigger a +the environment describes a Review App), GitLab automatically triggers a stop action when the associated branch is deleted. The `stop_review` job must be in the same `stage` as the `deploy_review` job in order for the environment to automatically stop. Additionally, both jobs should have matching [`rules`](../yaml/README.md#onlyexcept-basic) or [`only/except`](../yaml/README.md#onlyexcept-basic) configuration. In the example -above, if the configuration is not identical, the `stop_review` job might not be -included in all pipelines that include the `deploy_review` job, and it will not be -possible to trigger the `action: stop` to stop the environment automatically. +above, if the configuration isn't identical, the `stop_review` job might not be +included in all pipelines that include the `deploy_review` job, and it isn't +possible to trigger `action: stop` to stop the environment automatically. You can read more in the [`.gitlab-ci.yml` reference](../yaml/README.md#environmenton_stop). @@ -767,7 +773,7 @@ stop_review_app: ``` As long as a merge request is active and keeps getting new commits, -the review app will not stop, so developers don't need to worry about +the review app doesn't stop, so developers don't need to worry about re-initiating review app. On the other hand, since `stop_review_app` is set to `auto_stop_in: 1 week`, @@ -777,8 +783,8 @@ GitLab automatically triggers the `stop_review_app` job to stop the environment. You can also check the expiration date of environments through the GitLab UI. To do so, go to **Operations > Environments > Environment**. You can see the auto-stop period at the left-top section and a pin-mark button at the right-top section. This pin-mark -button can be used to prevent auto-stopping the environment. By clicking this button, the `auto_stop_in` setting is over-written -and the environment will be active until it's stopped manually. +button can be used to prevent auto-stopping the environment. By clicking this button, the +`auto_stop_in` setting is overwritten and the environment is active until it's stopped manually.  @@ -820,8 +826,8 @@ build with the specified environment runs. Newer deployments can also You may want to specify an environment keyword to [protect builds from unauthorized access](protected_environments.md), or to get access to [scoped variables](#scoping-environments-with-specs). In these cases, -you can use the `action: prepare` keyword to ensure deployments won't be created, -and no builds would be canceled: +you can use the `action: prepare` keyword to ensure deployments aren't created, +and no builds are canceled: ```yaml build: @@ -929,13 +935,13 @@ dashboard to appear, you need to Configure Prometheus to collect at least one In GitLab 9.2 and later, all deployments to an environment are shown directly on the monitoring dashboard. -Once configured, GitLab will attempt to retrieve [supported performance metrics](../../user/project/integrations/prometheus_library/index.md) +Once configured, GitLab attempts to retrieve [supported performance metrics](../../user/project/integrations/prometheus_library/index.md) for any environment that has had a successful deployment. If monitoring data was -successfully retrieved, a **Monitoring** button will appear for each environment. +successfully retrieved, a **Monitoring** button appears for each environment.  -Clicking on the **Monitoring** button will display a new page showing up to the last +Clicking the **Monitoring** button displays a new page showing up to the last 8 hours of performance data. It may take a minute or two for data to appear after initial deployment. @@ -962,10 +968,10 @@ of your web browser. To enable it, follow the instructions given in the service documentation. Note that container-based deployments often lack basic tools (like an editor), and may -be stopped or restarted at any time. If this happens, you will lose all your +be stopped or restarted at any time. If this happens, you lose all your changes. Treat this as a debugging tool, not a comprehensive online IDE. -Once enabled, your environments will gain a "terminal" button: +Once enabled, your environments gain a **Terminal** button:  @@ -973,12 +979,12 @@ You can also access the terminal button from the page for a specific environment  -Wherever you find it, clicking the button will take you to a separate page to +Wherever you find it, clicking the button takes you to a separate page to establish the terminal session:  -This works just like any other terminal. You'll be in the container created +This works like any other terminal. You're in the container created by your deployment so you can: - Run shell commands and get responses in real time. @@ -1008,9 +1014,8 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/* You can limit the environment scope of a variable by defining which environments it can be available for. -Wildcards can be used, and the default environment scope is `*`, which means -any jobs will have this variable, not matter if an environment is defined or -not. +Wildcards can be used and the default environment scope is `*`. This means that +any jobs can have this variable regardless of whether an environment is defined. For example, if the environment scope is `production`, then only the jobs having the environment `production` defined would have this specific variable. @@ -1057,7 +1062,7 @@ environment's operational health. ## Limitations In the `environment: name`, you are limited to only the [predefined environment variables](../variables/predefined_variables.md). -Re-using variables defined inside `script` as part of the environment name will not work. +Re-using variables defined inside `script` as part of the environment name doesn't work. ## Further reading @@ -1066,7 +1071,7 @@ Below are some links you may find interesting: - [The `.gitlab-ci.yml` definition of environments](../yaml/README.md#environment) - [A blog post on Deployments & Environments](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/) - [Review Apps - Use dynamic environments to deploy your code for every branch](../review_apps/index.md) -- [Deploy Boards for your applications running on Kubernetes](../../user/project/deploy_boards.md) **(PREMIUM)** +- [Deploy Boards for your applications running on Kubernetes](../../user/project/deploy_boards.md) <!-- ## Troubleshooting diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 94f9aac51d6..0e4ad1df65f 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -36,14 +36,14 @@ To protect an environment: 1. In the **Allowed to Deploy** dropdown menu, select the role, users, or groups you want to give deploy access to. Keep in mind that: - There are two roles to choose from: - - **Maintainers**: will allow access to all maintainers in the project. - - **Developers**: will allow access to all maintainers and all developers in the project. + - **Maintainers**: Allows access to all maintainers in the project. + - **Developers**: Allows access to all maintainers and all developers in the project. - You can only select groups that are already associated with the project. - - Only users that have at least Developer permission level will appear in + - Only users that have at least the Developer permission level appear in the **Allowed to Deploy** dropdown menu. 1. Click the **Protect** button. -The protected environment will now appear in the list of protected environments. +The protected environment now appears in the list of protected environments. ### Use the API to protect an environment @@ -79,7 +79,7 @@ Alternatively, you can use the API to protect an environment: 1. Use the API to add a user to the group as a reporter: ```shell - $ curl --request POST --header "PRIVATE-TOKEN: xxxxxxxxxxxx" --data "user_id=3222377&access_level=20" "https://gitlab.com/api/v4/groups/9899826/members" + $ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --data "user_id=3222377&access_level=20" "https://gitlab.com/api/v4/groups/9899826/members" {"id":3222377,"name":"Sean Carroll","username":"sfcarroll","state":"active","avatar_url":"https://assets.gitlab-static.net/uploads/-/system/user/avatar/3222377/avatar.png","web_url":"https://gitlab.com/sfcarroll","access_level":20,"created_at":"2020-10-26T17:37:50.309Z","expires_at":null} ``` @@ -87,7 +87,7 @@ Alternatively, you can use the API to protect an environment: 1. Use the API to add the group to the project as a reporter: ```shell - $ curl --request POST --header "PRIVATE-TOKEN: xxxxxxxxxxxx" --request POST "https://gitlab.com/api/v4/projects/22034114/share?group_id=9899826&group_access=20" + $ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --request POST "https://gitlab.com/api/v4/projects/22034114/share?group_id=9899826&group_access=20" {"id":1233335,"project_id":22034114,"group_id":9899826,"group_access":20,"expires_at":null} ``` @@ -95,7 +95,7 @@ Alternatively, you can use the API to protect an environment: 1. Use the API to add the group with protected environment access: ```shell - curl --header 'Content-Type: application/json' --request POST --data '{"name": "production", "deploy_access_levels": [{"group_id": 9899826}]}' --header "PRIVATE-TOKEN: xxxxxxxxxxx" "https://gitlab.com/api/v4/projects/22034114/protected_environments" + curl --header 'Content-Type: application/json' --request POST --data '{"name": "production", "deploy_access_levels": [{"group_id": 9899826}]}' --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/22034114/protected_environments" ``` The group now has access and can be seen in the UI. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index e728941dd9d..9fa0bb080ac 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -20,32 +20,31 @@ Examples are available in several forms. As a collection of: ## CI/CD examples -The following table lists examples with step-by-step tutorials that are contained in this section. +The following table lists examples with step-by-step tutorials that are contained in this section: -| Use case | Resource | -|:------------------------------|:---------------------------------------------------------------------------------------------------------------------------| +| Use case | Resource | +|:------------------------------|:---------| | Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | -| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | -| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | -| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | -| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | -| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | +| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | +| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | | GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | -| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | -| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | -| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | -| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | -| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | -| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | -| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | +| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | +| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | +| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | +| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | +| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | +| NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | +| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | +| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | | Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | -| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | -| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | -| NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | +| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | +| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | +| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | -### Contributing examples +### How to contributing examples Contributions are welcome! You can help your favorite programming language users and GitLab by sending a merge request with a guide for that language. @@ -72,6 +71,7 @@ choose one of these templates: - [dotNET (`dotNET.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET.gitlab-ci.yml) - [dotNET Core (`dotNET-Core.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET-Core.yml) - [Elixir (`Elixir.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Elixir.gitlab-ci.yml) +- [Flutter (`Flutter.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Flutter.gitlab-ci.yml) - [goLang (`Go.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Go.gitlab-ci.yml) - [Gradle (`Gradle.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Gradle.gitlab-ci.yml) - [Grails (`Grails.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Grails.gitlab-ci.yml) diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index e37bdcc9407..c1df21297e3 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -3,9 +3,14 @@ stage: Verify group: Continuous Integration info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' +author: Fabio Busatto +author_gitlab: bikebilly type: tutorial +date: 2017-08-15 --- +<!-- vale off --> + # How to deploy Maven projects to Artifactory with GitLab CI/CD ## Introduction diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index b7f59761889..fccc62a4ca0 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -19,14 +19,14 @@ To learn more, read [Using external secrets in CI](../../secrets/index.md). This tutorial assumes you are familiar with GitLab CI/CD and Vault. -To follow along, you will need: +To follow along, you must have: - An account on GitLab. - A running Vault server and access to it is required to configure authentication and create roles and policies. For HashiCorp Vaults, this can be the Open Source or Enterprise version. NOTE: -You will need to replace the `vault.example.com` URL below with the URL of your Vault server and `gitlab.example.com` with the URL of your GitLab instance. +You must replace the `vault.example.com` URL below with the URL of your Vault server, and `gitlab.example.com` with the URL of your GitLab instance. ## How it works @@ -47,6 +47,7 @@ The JWT's payload looks like this: "project_id": "22", # "project_path": "mygroup/myproject", # "user_id": "42", # Id of the user executing the job + "user_login": "myuser" # GitLab @username "user_email": "myuser@example.com", # Email of the user executing the job "pipeline_id": "1212", # "job_id": "1212", # @@ -56,7 +57,7 @@ The JWT's payload looks like this: } ``` -The JWT is encoded by using RS256 and signed with a dedicated private key. The expire time for the token will be set to job's timeout, if specified, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job will generate new JWT using the current signing key. +The JWT is encoded by using RS256 and signed with a dedicated private key. The expire time for the token is set to job's timeout, if specified, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job generates new JWT using the current signing key. You can use this JWT and your instance's JWKS endpoint (`https://gitlab.example.com/-/jwks`) to authenticate with a Vault server that is configured to allow the JWT Authentication method for authentication. @@ -110,7 +111,7 @@ EOF Success! Uploaded policy: myproject-production ``` -You'll also need roles that will link the JWT with these policies. +You also need roles that link the JWT with these policies. One for staging named `myproject-staging`: @@ -150,7 +151,7 @@ $ vault write auth/jwt/role/myproject-production - <<EOF EOF ``` -This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims will be allowed to authenticate. +This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims is allowed to authenticate. Combined with [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets. @@ -158,7 +159,7 @@ Combined with [protected branches](../../../user/project/protected_branches.md), [user_claim](https://www.vaultproject.io/api/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login. -[bound_claims_type](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values will be interpreted as globs, with `*` matching any number of characters. +[bound_claims_type](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values are interpreted as globs, with `*` matching any number of characters. For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). @@ -177,7 +178,7 @@ $ vault write auth/jwt/config \ For the full list of available configuration options, see Vault's [API documentation](https://www.vaultproject.io/api/auth/jwt#configure). -The following job, when run for the `master` branch, will be able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: +The following job, when run for the `master` branch, is able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: ```yaml read_secrets: @@ -201,7 +202,7 @@ read_secrets:  -The following job will be able to authenticate using the `myproject-production` role and read secrets under `/secret/myproject/production/`: +The following job is able to authenticate using the `myproject-production` role and read secrets under `/secret/myproject/production/`: ```yaml read_secrets: diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md index f4f3bf306ef..9c145677f6e 100644 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -2,18 +2,23 @@ stage: Release group: Release info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +author: Dylan Griffith +author_gitlab: DylanGriffith type: tutorial +date: 2018-06-07 +description: "Continuous Deployment of a Spring Boot application to Cloud Foundry with GitLab CI/CD" --- +<!-- vale off --> + # Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD ## Introduction -In this article, we'll demonstrate how to deploy a [Spring -Boot](https://projects.spring.io/spring-boot/) application to [Cloud -Foundry (CF)](https://www.cloudfoundry.org/) with GitLab CI/CD using the [Continuous -Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment) -method. +This article demonstrates how to use the [Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment) +method to deploy a [Spring Boot](https://projects.spring.io/spring-boot/) application to +[Cloud Foundry (CF)](https://www.cloudfoundry.org/) +with GitLab CI/CD. All the code for this project can be found in this [GitLab repository](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). @@ -25,17 +30,16 @@ using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring This tutorial assumes you are familiar with Java, GitLab, Cloud Foundry, and GitLab CI/CD. -To follow along, you will need: +To follow along, you need: - An account on [Pivotal Web Services (PWS)](https://run.pivotal.io/) or any other Cloud Foundry (CF) instance. - An account on GitLab. NOTE: -You will need to replace the `api.run.pivotal.io` URL in the all below -commands with the [API -URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) of your CF -instance if you're not deploying to PWS. +If you're not deploying to PWS, you must replace the `api.run.pivotal.io` URL in all the below +commands with the [API URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) +of your CF instance. ## Create your project @@ -46,9 +50,9 @@ GitLab when creating a new project: ## Configure the deployment to Cloud Foundry -To deploy to Cloud Foundry we need to add a `manifest.yml` file. This -is the configuration for the CF CLI we will use to deploy the application. We -will create this in the root directory of our project with the following +To deploy to Cloud Foundry you must add a `manifest.yml` file. This +is the configuration for the CF CLI you must use to deploy the application. +Create this in the root directory of your project with the following content: ```yaml @@ -62,12 +66,12 @@ applications: ## Configure GitLab CI/CD to deploy your application -Now we need to add the GitLab CI/CD configuration file -([`.gitlab-ci.yml`](../../yaml/README.md)) to our -project's root. This is how GitLab figures out what commands need to be run whenever -code is pushed to our repository. We will add the following `.gitlab-ci.yml` -file to the root directory of the repository, GitLab will detect it -automatically and run the steps defined once we push our code: +Now you must add the GitLab CI/CD configuration file +([`.gitlab-ci.yml`](../../yaml/README.md)) +to your project's root. This is how GitLab figures out what commands must run whenever +code is pushed to your repository. Add the following `.gitlab-ci.yml` +file to the root directory of the repository. GitLab detects it +automatically and runs the defined steps once you push your code: ```yaml image: java:8 @@ -96,15 +100,13 @@ production: - master ``` -We've used the `java:8` [Docker -image](../../docker/using_docker_images.md) to build -our application as it provides the up-to-date Java 8 JDK on [Docker -Hub](https://hub.docker.com/). We've also added the [`only` -clause](../../yaml/README.md#onlyexcept-basic) -to ensure our deployments only happen when we push to the master branch. +This uses the `java:8` [Docker image](../../docker/using_docker_images.md) +to build your application, as it provides the up-to-date Java 8 JDK on [Docker Hub](https://hub.docker.com/). +You also added the [`only` clause](../../yaml/README.md#onlyexcept-basic) +to ensure your deployments only happen when you push to the master branch. Because the steps defined in `.gitlab-ci.yml` require credentials to sign in to -CF, you'll need to add your CF credentials as +CF, you must add your CF credentials as [environment variables](../../variables/README.md#predefined-environment-variables) in GitLab CI/CD. To set the environment variables, navigate to your project's **Settings > CI/CD**, and then expand **Variables**. Name the variables @@ -122,8 +124,8 @@ your application and add its credentials to GitLab instead of using a developer's credentials. To start a manual deployment in GitLab go to **CI/CD > Pipelines** then click -on **Run Pipeline**. After the app is finished deploying, it will display the -URL of your application in the logs for the `production` job like: +**Run Pipeline**. After the app is finished deploying, it displays the +URL of your application in the logs for the `production` job: ```shell requested state: started diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 386512af38b..958093364af 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -55,10 +55,10 @@ To use different provider take a look at long list of [Supported Providers](http ## Using Dpl with Docker -In most cases, you will have configured [GitLab Runner](https://docs.gitlab.com/runner/) to use your server's shell commands. +In most cases, you configured [GitLab Runner](https://docs.gitlab.com/runner/) to use your server's shell commands. This means that all commands are run in the context of local user (e.g. `gitlab_runner` or `gitlab_ci_multi_runner`). It also means that most probably in your Docker container you don't have the Ruby runtime installed. -You will have to install it: +You must install it: ```yaml staging: @@ -115,7 +115,7 @@ We also use two secure variables: ## Storing API keys -Secure Variables can added by going to your project's +To add secure variables, navigate to your project's **Settings > CI / CD > Variables**. The variables that are defined in the project settings are sent along with the build script to the runner. The secure variables are stored out of the repository. Never store secrets in diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 24990264f19..6bc96ae6c30 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -9,13 +9,13 @@ type: tutorial This guide covers the building of dependencies of a PHP project while compiling assets via an NPM script using [GitLab CI/CD](../../README.md). -While it is possible to create your own image with custom PHP and Node.js versions, for brevity, we will use an existing [Docker image](https://hub.docker.com/r/tetraweb/php/) that contains both PHP and Node.js installed. +While it is possible to create your own image with custom PHP and Node.js versions, for brevity we use an existing [Docker image](https://hub.docker.com/r/tetraweb/php/) that contains both PHP and Node.js installed. ```yaml image: tetraweb/php ``` -The next step is to install zip/unzip packages and make composer available. We will place these in the `before_script` section: +The next step is to install zip/unzip packages and make composer available. We place these in the `before_script` section: ```yaml before_script: @@ -26,7 +26,7 @@ before_script: - php -r "unlink('composer-setup.php');" ``` -This will make sure we have all requirements ready. Next, we want to run `composer install` to fetch all PHP dependencies and `npm install` to load Node.js packages, then run the `npm` script. We need to append them into `before_script` section: +This makes sure we have all requirements ready. Next, run `composer install` to fetch all PHP dependencies and `npm install` to load Node.js packages. Then run the `npm` script. We need to append them into `before_script` section: ```yaml before_script: @@ -43,19 +43,19 @@ In this particular case, the `npm deploy` script is a Gulp script that does the 1. Copy various assets (images, fonts) around 1. Replace some strings -All these operations will put all files into a `build` folder, which is ready to be deployed to a live server. +All these operations put all files into a `build` folder, which is ready to be deployed to a live server. ## How to transfer files to a live server -You have multiple options: rsync, SCP, SFTP, and so on. For now, we will use SCP. +You have multiple options: rsync, SCP, SFTP, and so on. For now, use SCP. -To make this work, you need to add a GitLab CI/CD Variable (accessible on `gitlab.example/your-project-name/variables`). That variable will be called `STAGING_PRIVATE_KEY` and it's the **private** SSH key of your server. +To make this work, you must add a GitLab CI/CD Variable (accessible on `gitlab.example/your-project-name/variables`). Name this variable `STAGING_PRIVATE_KEY` and set it to the **private** SSH key of your server. ### Security tip Create a user that has access **only** to the folder that needs to be updated. -After you create that variable, you need to make sure that key will be added to the Docker container on run: +After you create that variable, make sure that key is added to the Docker container on run: ```yaml before_script: @@ -71,7 +71,7 @@ In order, this means that: 1. We check if the `ssh-agent` is available and we install it if it's not. 1. We create the `~/.ssh` folder. 1. We make sure we're running bash. -1. We disable host checking (we don't ask for user accept when we first connect to a server and since every job will equal a first connect, we kind of need this). +1. We disable host checking (we don't ask for user accept when we first connect to a server, and since every job equals a first connect, we need this). And this is basically all you need in the `before_script` section. @@ -96,14 +96,14 @@ stage_deploy: Here's the breakdown: -1. `only:dev` means that this build will run only when something is pushed to the `dev` branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want) -1. `ssh-add ...` we will add that private key you added on the web UI to the Docker container -1. We will connect via `ssh` and create a new `_tmp` folder -1. We will connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder -1. We will connect again via `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`. -1. We connect to SSH and remove the `_old` folder +1. `only:dev` means that this build runs only when something is pushed to the `dev` branch. You can remove this block completely and have everything run on every push (but probably this is something you don't want). +1. `ssh-add ...` we add that private key you added on the web UI to the Docker container. +1. We connect via `ssh` and create a new `_tmp` folder. +1. We connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder. +1. We connect again via `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`. +1. We connect to SSH and remove the `_old` folder. -What's the deal with the artifacts? We just tell GitLab CI/CD to keep the `build` directory (later on, you can download that as needed). +What's the deal with the artifacts? We tell GitLab CI/CD to keep the `build` directory (later on, you can download that as needed). ### Why we do it this way @@ -114,7 +114,7 @@ If you're using this only for stage server, you could do this in two steps: - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/live ``` -The problem is that there will be a small period of time when you won't have the app on your server. +The problem is that there's a small period of time when you don't have the app on your server. Therefore, for a production environment we use additional steps to ensure that at any given time, a functional app is in place. @@ -122,13 +122,13 @@ Therefore, for a production environment we use additional steps to ensure that a Since this was a WordPress project, I gave real life code snippets. Some further ideas you can pursue: -- Having a slightly different script for `master` branch will allow you to deploy to a production server from that branch and to a stage server from any other branches. +- Having a slightly different script for `master` branch allows you to deploy to a production server from that branch and to a stage server from any other branches. - Instead of pushing it live, you can push it to WordPress official repository (with creating a SVN commit, etc.). - You could generate i18n text domains on the fly. --- -Our final `.gitlab-ci.yml` will look like this: +Our final `.gitlab-ci.yml` looks like this: ```yaml image: tetraweb/php diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 7abdcf1f9be..298ffff568a 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -2,9 +2,14 @@ stage: Verify group: Continuous Integration info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +author: Ryan Hall +author_gitlab: blitzgren type: tutorial +date: 2018-03-07 --- +<!-- vale off --> + # DevOps and Game Dev with GitLab CI/CD With advances in WebGL and WebSockets, browsers are extremely viable as game development diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 96183b040a2..4521c2ed52e 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -2,9 +2,15 @@ stage: Verify group: Testing info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +author: Vincent Tunru +author_gitlab: Vinnl type: tutorial +date: 2019-02-18 +description: 'Confidence checking your entire app every time a new feature is added can quickly become repetitive. Learn how to automate it with GitLab CI/CD.' --- +<!-- vale off --> + # End-to-end testing with GitLab CI/CD and WebdriverIO [Review Apps](../../review_apps/index.md) are great: for every merge request diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 490fb857942..c6ddeefb916 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -2,9 +2,15 @@ stage: Verify group: Continuous Integration info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +disqus_identifier: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' +author: Mehran Rasulian +author_gitlab: mehranrasulian type: tutorial +date: 2017-08-31 --- +<!-- vale off --> + # Test and deploy Laravel applications with GitLab CI/CD and Envoy ## Introduction diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 70d29b739b1..037faaf66a2 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -35,7 +35,7 @@ You can also view or fork the complete [example source](https://gitlab.com/gitla } ``` -1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in NPM's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json#files). +1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in NPM's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#files). 1. Add a `.gitignore` file to the project to avoid committing `node_modules`: diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 089d72852bb..1204a1ae837 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -28,16 +28,16 @@ test: staging: stage: deploy script: - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-ruby-test-staging --api-key=$HEROKU_STAGING_API_KEY + - gem install dpl --pre + - dpl heroku api --app=gitlab-ci-ruby-test-staging --api-key=$HEROKU_STAGING_API_KEY only: - master production: stage: deploy script: - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-ruby-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY + - gem install dpl --pre + - dpl heroku api --app=gitlab-ci-ruby-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY only: - tags ``` @@ -50,7 +50,7 @@ This project has three jobs: ## Store API keys -You'll need to create two variables in your project's **Settings > CI/CD > Environment variables**: +You'll need to create two variables in your project's **Settings > CI/CD > Environment variables** and do not check **Protect variable** and **Mask variable**: - `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. - `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-new.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-new.png Binary files differdeleted file mode 100644 index 04d3dc40fa5..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-new.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-server.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-server.png Binary files differdeleted file mode 100644 index 63812b41c2c..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-server.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/pipelines.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/pipelines.png Binary files differdeleted file mode 100644 index c0daa1a6a91..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/pipelines.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/select_template_v12_6.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/select_template_v12_6.png Binary files differdeleted file mode 100644 index c8c5e152a13..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/select_template_v12_6.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/set_up_ci_v12_6.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/set_up_ci_v12_6.png Binary files differdeleted file mode 100644 index fafabb27bac..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/set_up_ci_v12_6.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md index 90f04fb3615..057b6ec126f 100644 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md @@ -1,397 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: '../README.md' --- -# Testing a Phoenix application with GitLab CI/CD +This example is no longer available. [View other examples](../README.md). -[Phoenix](https://www.phoenixframework.org/) is a web development framework written in [Elixir](https://elixir-lang.org), which is a -functional language designed for productivity and maintainability that runs on the -[Erlang VM](https://www.erlang.org). Erlang VM is really fast and can handle very large numbers of -simultaneous users. - -That's why we're hearing so much about Phoenix today. - -In this tutorial, we'll teach you how to set up [GitLab CI/CD](../../README.md) to build and test a Phoenix -application. - -The tutorial assumes that you know how to create a Phoenix app, run tests locally, and how to work with Git -and the GitLab UI. - -## Introduction - -### What is Phoenix? - -[Phoenix](https://www.phoenixframework.org/) is a web development framework written in [Elixir](https://elixir-lang.org). It's useful - for building fast, reliable, and high-performance applications, as it uses [Erlang VM](https://www.erlang.org). - -Many components and concepts are similar to Ruby on Rails or Python's Django. High developer -productivity and high application performance are only a few advantages on learning how to use it. -Working on the MVC pattern, it's was designed to be modular and flexible. Easy to maintain a growing -app is a plus. - -Phoenix can run in any OS where Erlang is supported: - -- Ubuntu -- CentOS -- Mac OS X -- Debian -- Windows -- Fedora -- Raspberry Pi OS - -Check the [Phoenix learning guide](https://hexdocs.pm/phoenix/overview.html) for more information. - -### What is Elixir? - -[Elixir](https://elixir-lang.org) is a dynamic, functional language created to use all the maturity of Erlang -(30 years old!) in these days, in an easy way. It has similarities with Ruby, specially on syntax, -so Ruby developers are quite excited with the rapid growing of Elixir. A full-stack Ruby developer -can learn how to use Elixir and Phoenix in just a few weeks! - -In Elixir we have a command called `mix`, which is a helper to create projects, testing, run -migrations and [much more](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix). We'll use it later on in this tutorial. - -Check the [Elixir documentation](https://elixir-lang.org/getting-started/introduction) for more information. - -## Requirements - -To follow this tutorial, you'll need to have installed: - -- Elixir [installation instructions](https://elixir-lang.org/install) -- Phoenix Framework [installation instructions](https://hexdocs.pm/phoenix/installation.html) -- PostgreSQL (if you need to use MySQL server, check [Phoenix instructions](https://hexdocs.pm/phoenix/ecto.html#using-mysql)) - -### Create a new Phoenix project - -Open your terminal and go to the directory you wish to create your project. -You don't need to create an empty directory for the project's files, because the `mix` command will -do it for us. - -When we call `mix` command, we'll pass two arguments: - -- The task we want it to run: `phoenix.new` -- And the parameter `phoenix.new` requires, which is the name of the new project. In this case, - we're calling it `hello_gitlab_ci`, but you're free to set your own name: - -```shell -mix phoenix.new hello_gitlab_ci -``` - -When asked, answer `Y` to fetch and install dependencies. - -If everything went fine, you'll get an output like this: - - - -Now, our project is located inside the directory with the same name we pass to `mix` command, for -example, `~/GitLab/hello_gitlab_ci`. -If we take a look at the directory, we'll see the Phoenix files and the dependencies needed to run. - -### Initialize the PostgreSQL database - -By default, Phoenix requires a PostgreSQL database to store whatever we need to store in our app. In -this case, we'll only create an empty database. - -First, we need to navigate to our recently created project's directory, and then execute again -`mix`. This time, `mix` will receive the parameter `ecto.create`, which is the task to create our -new database. [Ecto](https://hexdocs.pm/ecto/Ecto.html) is the database wrapper for Elixir. - -When we do run `mix` the first time after creating our project, it will compile our files to -bytecode, which will be interpreted by Erlang VM. In the next times, it will only compile our -changes. - -Run the commands below to create our empty database: - -```shell -cd hello_gitlab_ci -mix ecto.create -``` - -We expect to see this output at the end of the command: - -```plaintext -Generated hello_gitlab_ci app -The database for HelloGitlabCi.Repo has been created -``` - -Phoenix assumes that our PostgreSQL database will have a `postgres` user account with the correct -permissions and a password of `postgres`. If it's not your case, check -[Ecto's instructions](https://hexdocs.pm/ecto/Ecto.html#module-repositories). - -### Start Phoenix server - -Now, it's time to see if everything we did until now went well. We'll call `mix` again, this time -with `phoenix.server` parameter, which will start Phoenix's HTTP Server. - -```shell -mix phoenix.server -``` - -This will be the output to this command: - -```plaintext -[info] Running HelloGitlabCi.Endpoint with Cowboy using http://localhost:4000 -23 May 11:44:35 - info: compiling -23 May 11:44:37 - info: compiled 6 files into 2 files, copied 3 in 9.8 sec -``` - -Now, we have our app running locally. We can preview it directly on our browser. Let's open -[`localhost:4000`](http://localhost:4000) to see our Phoenix Framework welcome page. If the link do -not work, open [`127.0.0.1:4000`](http://127.0.0.1:4000) instead and later, configure your OS to -point `localhost` to `127.0.0.1`. - - - -Great, now we have a local Phoenix Server running our app. - -Locally, our application is running in an [`iex`](https://elixir-lang.org/getting-started/introduction.html#interactive-mode) session, which stands for Interactive Elixir. -In this interactive mode, we can type any Elixir expression and get its result. To exit `iex`, we -need to press `Ctrl+C` twice. So, when we need to stop the Phoenix server, we have to hit `Ctrl+C` -twice. - -## Introducing GitLab CI/CD - -With GitLab, we can manage our development workflow, improve our productivity, track issues, -perform code review, and much more from a single platform. With GitLab CI/CD, we can be much more -productive, because every time we, or our co-workers push any code, GitLab CI/CD will build and -test the changes, telling us in real time if anything goes wrong. - -Certainly, when our application starts to grow, we'll need more developers working on the same -project and this process of building and testing can easily become a mess without proper management. -That's also why GitLab CI/CD is so important to our application. Every time someone pushes its code to -GitLab, we'll quickly know if their changes broke something or not. We don't need to stop everything -we're doing to test manually and locally every change our team does. - -Let's see this in practice. - -## Adjusting Phoenix configuration - -Now, we need to adjust our Phoenix configuration before configuring GitLab CI/CD. -There is a directory (`config`) in your Phoenix project that contains a configuration file for every -environment it can run. Since we will work with a single environment, we'll edit just the test -configuration file (`test.exs`). - -But, why do we need to adjust our configuration? Well, GitLab CI/CD builds and tests our code in one -isolated virtual machine, called a [runner](../../runners/README.md), using Docker technology. In this runner, -GitLab CI/CD has access to everything our Phoenix application need to run, exactly as we have in our -`localhost`, but we have to tell GitLab CI/CD where to create and find this database using system -variables. This way, GitLab CI/CD will create our test database inside the runner, just like we do -when running our Phoenix in our `localhost`. - -- Open `hello_gitlab_ci/config/test.exs` on your favorite code editor -- Go to **Configure your database** session and edit the block to include `System.get_env`: - - ```elixir - # Configure your database - config :hello_gitlab_ci, HelloGitlabCi.Repo, - adapter: Ecto.Adapters.Postgres, - username: System.get_env("POSTGRES_USER") || "postgres", - password: System.get_env("POSTGRES_PASSWORD") || "postgres", - database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test", - hostname: System.get_env("POSTGRES_HOST") || "localhost", - pool: Ecto.Adapters.SQL.Sandbox - ``` - - We'll need these system variables later on. - -- Create an empty file named `.gitkeep` into `hello_gitlab_ci/priv/repo/migrations` - - As our project is still fresh, we don't have any data on our database, so, the `migrations` - directory will be empty. - Without `.gitkeep`, Git will not upload this empty directory and we'll got an error when running our - test on GitLab. - - If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir. - -Now, let's run a local test and see if everything we did didn't break anything. - -## Testing - -Earlier, when we created our project, we ran `mix phoenix.new`. -This task created everything a Phoenix application needed, including some unit tests into -`hello_gitlab_ci/test` directory. - -Let's run a new task with `mix` to run those tests for us. This time, the parameter expected is -`test`. We can add `--trace` parameter for debugging purposes. - -In your terminal, navigate to the directory `hello_gitlab_ci` and run: - -```shell -mix test -``` - -Our expected result is this: - -```plaintext -.... - -Finished in 0.7 seconds -4 tests, 0 failures - -Randomized with seed 610000 -``` - -Our test was successful. It's time to push our files to GitLab. - -## Configuring CI/CD Pipeline - -The first step is to create a new file called `.gitlab-ci.yml` in `hello_gitlab_ci` directory of our -project. - -- The easiest way to do this is to click on **Set up CI/CD** on project's main page: - -  - -- On the next screen, we can use a template with Elixir tests already included. Click on **Apply a template** and select **Elixir**: - -  - - This template file tells GitLab CI/CD about what we wish to do every time a new commit is made. - However, we have to adapt it slightly to run a Phoenix app. - -- The first line tells GitLab what Docker image will be used. - - Remember when we learned about runners, the isolated virtual machine where GitLab CI/CD builds and tests - our application? This virtual machine must have all dependencies to run our application. This is - where a Docker image is needed. The correct image will provide the entire system for us. - - As we are focusing on testing (not deploying), you can use the [elixir:latest](https://hub.docker.com/_/elixir) Docker image, which already has the - dependencies for running Phoenix tests installed, such as Elixir and Erlang: - - ```yaml - image: elixir:latest - ``` - -- We'll only use `postgres`, so we can delete the `mysql` and `redis` lines from the `services` section: - - ```yaml - services: - - postgres:latest - ``` - -- Now, we'll create a new section called `variables`, before the `before_script` section: - - ```yaml - variables: - POSTGRES_DB: hello_gitlab_ci_test - POSTGRES_HOST: postgres - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "postgres" - MIX_ENV: "test" - ``` - - Above, we set up the values for GitLab CI/CD to authenticate into PostgreSQL, like we did in - `config/test.exs` earlier. The `POSTGRES_USER` and `POSTGRES_PASSWORD` values - are used by the `postgres` service to create a user with those credentials. - -- In the `before_script` section, we'll add some commands to prepare everything for the test: - - ```yaml - before_script: - - mix local.rebar --force - - mix local.hex --force - - mix deps.get --only test - - mix ecto.create - - mix ecto.migrate - ``` - - This ensures that [rebar3](https://rebar3.org) and [hex](https://hex.pm) are both installed - before attempting to fetch the dependencies that are required to run the tests. Next, the `postgres` db - is created and migrated with `ecto`, to ensure it's up-to-date. - -- Finally, we'll leave the `mix` section unchanged. - -Let's take a look at the updated file after the changes: - -```yaml -image: elixir:latest - -services: - - postgres:latest - -variables: - POSTGRES_DB: hello_gitlab_ci_test - POSTGRES_HOST: postgres - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "postgres" - MIX_ENV: "test" - -before_script: - - mix local.rebar --force - - mix local.hex --force - - mix deps.get --only test - - mix ecto.create - - mix ecto.migrate - -mix: - script: - - mix test -``` - -For safety, we can check if we get any syntax errors before submitting this file to GitLab. Copy the -contents of `.gitlab-ci.yml` and paste it on [GitLab CI/CD Lint tool](https://gitlab.com/ci/lint). Please note that -this link will only work for logged in users. - -## Watching the build - -I don't know about you, but I love to watch that black screen being filled with compilation output. -With this, I can feel the happiness of something I made working correctly. On `localhost` it's easy -to watch our build, but on GitLab, is it possible? Yes! - -Let's go to **Pipelines** and see GitLab doing the job. Just click on **Pipelines** to find the -actual running build job. - - - -Click on build's ID to watch the entire process. If everything went as expected, we can wait for the -**Build succeeded** at the end of the process! :) - -```shell -$ mix test -.... - -Finished in 0.3 seconds -4 tests, 0 failures - -Randomized with seed 206909 -Build succeeded -``` - -If we take a look at the project's main page on the GitLab UI, we can see the status of the last -build made by GitLab CI/CD. - -Time to show the world our green build badge! Navigate to your project's **Settings > CI/CD** and -expand **General pipelines settings**. Scroll down to **Pipeline status** and copy the Markdown code -for your badge. Paste it on the top of your `README.md` file, to let people outside of our project -see if our latest code is running without errors. - -When we finish this edition, GitLab will start another build and show a **build running** badge. It -is expected, after all we just configured GitLab CI/CD to do this for every push! But you may think -"Why run build and tests for simple things like editing README.md?" and it is a good question. -For changes that don't affect your application, you can add the keyword [`[ci skip]`](../../yaml/README.md#skip-pipeline) -to commit message and the build related to that commit will be skipped. - -In the end, we finally got our pretty green build succeeded badge! By outputting the result on the -README file, it shows to whoever lands on your project's page that your code is up-to-date and -working properly. - -## Conclusion - -When we have a growing application with many developers working on it, or when we have an open -source project being watched and contributed by the community, it is really important to have our -code permanently working. GitLab CI/CD is a time saving powerful tool to help us maintain our code -organized and working. - -As we could see in this post, GitLab CI/CD is really easy to configure and use. We have [many -other reasons](https://about.gitlab.com/blog/2015/02/03/7-reasons-why-you-should-be-using-ci/) to keep -using GitLab CI/CD. The benefits to our teams will be huge! - -## References - -- [GitLab CI/CD introductory guide](https://about.gitlab.com/blog/2015/12/14/getting-started-with-gitlab-and-gitlab-ci/) -- [GitLab CI/CD full Documentation](../../README.md) -- [GitLab Runner documentation](../../runners/README.md) -- [Using Docker images documentation](../../docker/using_docker_images.md) +<!-- This redirect file can be deleted after <2021-04-05>. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/img/environments_available.png b/doc/ci/img/environments_available.png Binary files differdeleted file mode 100644 index 6c64e9398f7..00000000000 --- a/doc/ci/img/environments_available.png +++ /dev/null diff --git a/doc/ci/img/environments_available_13_7.png b/doc/ci/img/environments_available_13_7.png Binary files differnew file mode 100644 index 00000000000..2e1f56c5894 --- /dev/null +++ b/doc/ci/img/environments_available_13_7.png diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index 3fa427bc875..c04bcd6f549 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -147,10 +147,10 @@ according to each stage (Verify, Package, Release). - Continuous Deployment, automatically deploying your app to production. - Continuous Delivery, manually click to deploy your app to production. - Deploy static websites with [GitLab Pages](../../user/project/pages/index.md). - - Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature with [Canary Deployments](../../user/project/canary_deployments.md). **(PREMIUM)** + - Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature with [Canary Deployments](../../user/project/canary_deployments.md). - Deploy your features behind [Feature Flags](../../operations/feature_flags.md). - Add release notes to any Git tag with [GitLab Releases](../../user/project/releases/index.md). - - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). **(PREMIUM)** + - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). - Deploy your application to a production environment in a Kubernetes cluster with [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy). With GitLab CI/CD you can also: diff --git a/doc/ci/merge_request_pipelines/img/pipeline-fork_v13_7.png b/doc/ci/merge_request_pipelines/img/pipeline-fork_v13_7.png Binary files differnew file mode 100644 index 00000000000..eb44290aa66 --- /dev/null +++ b/doc/ci/merge_request_pipelines/img/pipeline-fork_v13_7.png diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index 220032eeab1..999d15eac24 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -177,6 +177,10 @@ coming from a fork: - It's created and runs in the fork (source) project, not the parent (target) project. - It uses the fork project's CI/CD configuration and resources. +If a pipeline runs in a fork, the **fork** icon appears for the pipeline in the merge request. + + + Sometimes parent project members want the pipeline to run in the parent project. This could be to ensure that the post-merge pipeline passes in the parent project. For example, a fork project could try to use a corrupted runner that doesn't execute diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index 1b9bade3b76..e83789efdbf 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -50,6 +50,8 @@ To enable pipelines for merge results: - You must not be using [fast forward merges](../../../user/project/merge_requests/fast_forward_merge.md) yet. To follow progress, see [#58226](https://gitlab.com/gitlab-org/gitlab/-/issues/26996). +- Your repository must be a GitLab repository, not an + [external repository](../../ci_cd_for_external_repos/index.md). ## Enable pipelines for merged results @@ -58,7 +60,7 @@ To enable pipelines for merged results for your project: 1. [Configure your CI/CD configuration file](../index.md#configuring-pipelines-for-merge-requests) so that the pipeline or individual jobs run for merge requests. 1. Visit your project's **Settings > General** and expand **Merge requests**. -1. Check **Enable merged results pipelines.**. +1. Check **Enable merged results pipelines**. 1. Click **Save changes**. WARNING: diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index 0b25b32b2a5..e5b9ad030d0 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -74,6 +74,8 @@ To enable merge trains: - You must have maintainer [permissions](../../../../user/permissions.md). - You must be using [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or later. - In GitLab 12.0 and later, you need [Redis](https://redis.io/) 3.2 or later. +- Your repository must be a GitLab repository, not an + [external repository](../../../ci_cd_for_external_repos/index.md). ## Enable merge trains @@ -177,9 +179,13 @@ for more information. ### Merge Train Pipeline cannot be retried -A Merge Train pipeline cannot be retried because the merge request is dropped from the merge train upon failure. For this reason, the retry button does not appear next to the pipeline icon. +When a pipeline for merge trains fails the merge request is dropped from the train and the pipeline can't be retried. +Pipelines for merge trains run on the merged result of the changes in the merge request and +the changes from other merge requests already on the train. If the merge request is dropped from the train, +the merged result is out of date and the pipeline can't be retried. -In the case of pipeline failure, you should [re-enqueue](#add-a-merge-request-to-a-merge-train) the merge request to the merge train, which then initiates a new pipeline. +Instead, you should [add the merge request to the train](#add-a-merge-request-to-a-merge-train) +again, which triggers a new pipeline. ### Unable to add to merge train with message "The pipeline for this merge request failed." diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 1df196182c0..006d6bda0e0 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -13,6 +13,9 @@ type: reference You can set up [GitLab CI/CD](README.md) across multiple projects, so that a pipeline in one project can trigger a pipeline in another project. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview see the [Multi-project pipelines demo](https://www.youtube.com/watch?v=g_PIwBM1J84). + GitLab CI/CD is a powerful continuous integration tool that works not only per project, but also across projects with multi-project pipelines. diff --git a/doc/ci/yaml/img/ci_config_visualization_hover_v13_7.png b/doc/ci/pipeline_editor/img/ci_config_visualization_hover_v13_7.png Binary files differindex 9387fc6ccf4..9387fc6ccf4 100644 --- a/doc/ci/yaml/img/ci_config_visualization_hover_v13_7.png +++ b/doc/ci/pipeline_editor/img/ci_config_visualization_hover_v13_7.png diff --git a/doc/ci/yaml/img/ci_config_visualization_v13_7.png b/doc/ci/pipeline_editor/img/ci_config_visualization_v13_7.png Binary files differindex ef2aa6fe9e9..ef2aa6fe9e9 100644 --- a/doc/ci/yaml/img/ci_config_visualization_v13_7.png +++ b/doc/ci/pipeline_editor/img/ci_config_visualization_v13_7.png diff --git a/doc/ci/pipeline_editor/img/pipeline_editor_commit_v13_8.png b/doc/ci/pipeline_editor/img/pipeline_editor_commit_v13_8.png Binary files differnew file mode 100644 index 00000000000..cc1f666f319 --- /dev/null +++ b/doc/ci/pipeline_editor/img/pipeline_editor_commit_v13_8.png diff --git a/doc/ci/pipeline_editor/img/pipeline_editor_lint_v13_8.png b/doc/ci/pipeline_editor/img/pipeline_editor_lint_v13_8.png Binary files differnew file mode 100644 index 00000000000..28d21f71378 --- /dev/null +++ b/doc/ci/pipeline_editor/img/pipeline_editor_lint_v13_8.png diff --git a/doc/ci/pipeline_editor/img/pipeline_editor_validate_v13_8.png b/doc/ci/pipeline_editor/img/pipeline_editor_validate_v13_8.png Binary files differnew file mode 100644 index 00000000000..a4140d5220a --- /dev/null +++ b/doc/ci/pipeline_editor/img/pipeline_editor_validate_v13_8.png diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md new file mode 100644 index 00000000000..61b8e289509 --- /dev/null +++ b/doc/ci/pipeline_editor/index.md @@ -0,0 +1,133 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# Pipeline Editor **(CORE)** + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4540) in GitLab 13.8. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-pipeline-editor). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +The pipeline editor is the primary place to edit the GitLab CI/CD configuration in +your `.gitlab-ci.yml` file. To access it, go to **CI/CD > Editor**. + +From the pipeline editor page you can: + +- [Validate](#validate-ci-configuration) your configuration syntax while editing the file. +- Do a deeper [lint](#lint-ci-configuration) of your configuration, that verifies it with any configuration + added with the [`include`](../yaml/README.md#include) keyword. +- See a [visualization](#visualize-ci-configuration) of the current configuration. +- [Commit](#commit-changes-to-ci-configuration) the changes to a specific branch. + +NOTE: +You must have already [created a CI/CD configuration file](../quick_start/README.md#create-a-gitlab-ciyml-file) +to use the editor. + +## Validate CI configuration + +As you edit your pipeline configuration, it is continually validated against the GitLab CI/CD +pipeline schema. It checks the syntax of your CI YAML configuration, and also runs +some basic logical validations. + +The result of this validation is shown at the top of the editor page. If your configuration +is invalid, a tip is shown to help you fix the problem: + + + +## Lint CI configuration + +To test the validity of your GitLab CI/CD configuration before committing the changes, +you can use the CI lint tool. To access it, go to **CI/CD > Editor** and select the **Lint** tab. + +This tool checks for syntax and logical errors but goes into more detail than the +automatic [validation](#validate-ci-configuration) in the editor. + +The results are updated in real-time. Any changes you make to the configuration are +reflected in the CI lint. It displays the same results as the existing [CI Lint tool](../lint.md). + + + +## Visualize CI configuration + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241722) in GitLab 13.5. +> - [Moved to **CI/CD > Editor**](https://gitlab.com/gitlab-org/gitlab/-/issues/263141) in GitLab 13.7. +> - It was [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/290117) in GitLab 13.8. +> - It's enabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-cicd-configuration-visualization). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +To see a visualization of your `gitlab-ci.yml` configuration, navigate to **CI/CD > Editor** +and select the `visualization` tab. The visualization shows all stages and jobs. +[`needs`](../yaml/README.md#needs) relationships are displayed as lines connecting jobs together, showing the hierarchy of execution: + + + +Hovering on a job highlights its `needs` relationships: + + + +If the configuration does not have any `needs` relationships, then no lines are drawn because +each job depends only on the previous stage being completed successfully. + +### Enable or disable CI/CD configuration visualization **(CORE ONLY)** + +CI/CD configuration visualization is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it. + +To disable it: + +```ruby +Feature.disable(:ci_config_visualization_tab) +``` + +To enable it: + +```ruby +Feature.enable(:ci_config_visualization_tab) +``` + +## Commit changes to CI configuration + +The commit form appears at the bottom of each tab in the editor so you can commit +your changes at any time. + +When you are satisfied with your changes, add a descriptive commit message and enter +a branch. The branch field defaults to your project's default branch. + +If you enter a new branch name, the **Start a new merge request with these changes** +checkbox appears. Select it to start a new merge request after you commit the changes. + + + +## Enable or disable pipeline editor **(CORE ONLY)** + +The pipeline editor is under development and not ready for production use. It is +deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can disable it. + +To disable it: + +```ruby +Feature.disable(:ci_pipeline_editor_page) +``` + +To enable it: + +```ruby +Feature.enable(:ci_pipeline_editor_page) +``` diff --git a/doc/ci/pipelines/img/job_artifacts_merge_request.png b/doc/ci/pipelines/img/job_artifacts_merge_request.png Binary files differnew file mode 100644 index 00000000000..fa1ed9acbf8 --- /dev/null +++ b/doc/ci/pipelines/img/job_artifacts_merge_request.png diff --git a/doc/ci/pipelines/img/pipelines_duration_chart.png b/doc/ci/pipelines/img/pipelines_duration_chart.png Binary files differdeleted file mode 100644 index 12ec262dadb..00000000000 --- a/doc/ci/pipelines/img/pipelines_duration_chart.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipelines_success_chart.png b/doc/ci/pipelines/img/pipelines_success_chart.png Binary files differdeleted file mode 100644 index f44dc25ff1c..00000000000 --- a/doc/ci/pipelines/img/pipelines_success_chart.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 22e331f2de0..7920a3bf7f3 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -89,9 +89,9 @@ This table lists the refspecs injected for each pipeline type: | Pipeline type | Refspecs | |--------------- |---------------------------------------- | -| Pipeline for Branches | `+refs/pipelines/<id>:refs/pipelines/<id>` and `+refs/heads/<name>:refs/remotes/origin/<name>` | -| pipeline for Tags | `+refs/pipelines/<id>:refs/pipelines/<id>` and `+refs/tags/<name>:refs/tags/<name>` | -| [Pipeline for Merge Requests](../merge_request_pipelines/index.md) | `+refs/pipelines/<id>:refs/pipelines/<id>` | +| Pipeline for Branches | `+<sha>:refs/pipelines/<id>` and `+refs/heads/<name>:refs/remotes/origin/<name>` | +| pipeline for Tags | `+<sha>:refs/pipelines/<id>` and `+refs/tags/<name>:refs/tags/<name>` | +| [Pipeline for Merge Requests](../merge_request_pipelines/index.md) | `+<sha>:refs/pipelines/<id>` | The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a @@ -132,6 +132,10 @@ Pipelines can be manually executed, with predefined or manually-specified [varia You might do this if the results of a pipeline (for example, a code build) are required outside the normal operation of the pipeline. +[In GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/30101), +all global variables with descriptions defined in the `.gitlab-ci.yml` file are +displayed in the variable fields. + To execute a pipeline manually: 1. Navigate to your project's **CI/CD > Pipelines**. @@ -345,18 +349,7 @@ Stages in pipeline mini graphs are collapsible. Hover your mouse over them and c ### Pipeline success and duration charts -> - Introduced in GitLab 3.1.1 as Commit Stats, and later renamed to Pipeline Charts. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/38318) to CI / CD Analytics in GitLab 12.8. - -GitLab tracks the history of your pipeline successes and failures, as well as how long each pipeline ran. To view this information, go to **Analytics > CI / CD Analytics**. - -View successful pipelines: - - - -View pipeline duration history: - - +Pipeline analytics are available on the [**CI/CD Analytics** page](../../user/analytics/ci_cd_analytics.md#pipeline-success-and-duration-charts). ### Pipeline badges diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 787ee8f8573..4c77a578aa4 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -155,9 +155,10 @@ as artifacts. The collected Code Quality report uploads to GitLab as an artifact and is summarized in merge requests. -#### `artifacts:reports:sast` **(ULTIMATE)** +#### `artifacts:reports:sast` > - Introduced in GitLab 11.5. +> - Made [available in all tiers](https://gitlab.com/groups/gitlab-org/-/epics/2098) in GitLab 13.3. > - Requires GitLab Runner 11.5 and above. The `sast` report collects [SAST vulnerabilities](../../user/application_security/sast/index.md) @@ -349,6 +350,11 @@ in the GitLab UI to do this:  +1. While on the details page of a merge request, you can see the download + icon for each job's artifacts on the right side of the merge request widget: + +  + 1. And finally, when browsing an archive you can see the download button at the top right corner: @@ -459,6 +465,23 @@ To retrieve a job artifact from a different project, you might need to use a private token to [authenticate and download](../../api/job_artifacts.md#get-job-artifacts) the artifact. +## Keep artifacts from most recent successful jobs + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16267) in GitLab 13.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) in GitLab 13.4. +> - [Made optional with a CI/CD setting](https://gitlab.com/gitlab-org/gitlab/-/issues/241026) in GitLab 13.8. + +By default, the latest artifacts from the most recent successful jobs are never deleted. +If a job is configured with [`expire_in`](../yaml/README.md#artifactsexpire_in), +its artifacts only expire if a more recent artifact exists. + +Keeping the latest artifacts can use a large amount of storage space in projects +with a lot of jobs or large artifacts. If the latest artifacts are not needed in +a project, you can disable this behavior to save space: + +1. Navigate to **Settings > CI/CD > Artifacts**. +1. Uncheck **Keep artifacts from most recent successful jobs**. + ## Troubleshooting ### Error message `No files to upload` diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 35a8888381f..cddfcb754ec 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -67,7 +67,7 @@ To configure a job to be executed only when the pipeline has been scheduled (or the opposite), use [only and except](../yaml/README.md#onlyexcept-basic) configuration keywords. -For example: +In the example below `make world` runs in scheduled pipelines, and `make build` runs in pipelines that are not scheduled: ```yaml job:on-schedule: diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 7fd88d011b3..e9c85353db3 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -129,7 +129,7 @@ The pipeline starts when the commit is committed. - To validate your `.gitlab-ci.yml` file, use the [CI Lint tool](../lint.md), which is available in every project. -- You can also use [CI/CD configuration visualization](../yaml/visualization.md) to +- You can also use [CI/CD configuration visualization](../pipeline_editor/index.md#visualize-ci-configuration) to view a graphical representation of your `.gitlab-ci.yml` file. - For the complete `.gitlab-ci.yml` syntax, see [the `.gitlab-ci.yml` reference topic](../yaml/README.md). diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 31fcfe9d6e8..3512b77e4e2 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -69,7 +69,7 @@ The process of configuring Review Apps is as follows: When configuring Review Apps for a project, you need to add a new job to `.gitlab-ci.yml`, as mentioned above. To facilitate this and if you are using Kubernetes, you can click -the **Enable Review Apps** button and GitLab will prompt you with a template code block that +the **Enable Review Apps** button and GitLab prompts you with a template code block that you can copy and paste into `.gitlab-ci.yml` as a starting point. To do so: 1. Go to the project your want to create a Review App job for. @@ -115,7 +115,7 @@ and faster to preview proposed modifications. Configuring Route Maps involves telling GitLab how the paths of files in your repository map to paths of pages on your website using a Route Map. -Once set, GitLab will display **View on ...** buttons, which will take you +Once set, GitLab displays **View on ...** buttons, which take you to the pages changed directly from merge requests. To set up a route map, add a file inside the repository at `.gitlab/route-map.yml`, @@ -165,15 +165,15 @@ The public path for a source path is determined by finding the first In the example above, the fact that mappings are evaluated in order of their definition is used to ensure that `source/index.html.haml` -will match `/source\/(.+?\.html).*/` instead of `/source\/(.*)/`, -and will result in a public path of `index.html`, instead of +matches `/source\/(.+?\.html).*/` instead of `/source\/(.*)/`, +and results in a public path of `index.html`, instead of `index.html.haml`. -After you have the route mapping set up, it will take effect in the following locations: +After you have the route mapping set up, it takes effect in the following locations: - In the merge request widget. The: - - **View app** button will take you to the environment URL set in `.gitlab-ci.yml`. - - Dropdown will list the first 5 matched items from the route map, but you can filter them if more + - **View app** button takes you to the environment URL set in `.gitlab-ci.yml`. + - Dropdown lists the first 5 matched items from the route map, but you can filter them if more than 5 are available.  @@ -221,7 +221,7 @@ To see Visual reviews in action, see the [Visual Reviews Walk through](https://y The feedback form is served through a script you add to pages in your Review App. If you have [Developer permissions](../../user/permissions.md) to the project, you can access it by clicking the **Review** button in the **Pipeline** section -of the merge request. The form modal will also show a dropdown for changed pages +of the merge request. The form modal also shows a dropdown for changed pages if [route maps](#route-maps) are configured in the project.  @@ -251,13 +251,13 @@ to replace those values at runtime when each review app is created: `CI_MERGE_REQUEST_IID` variable. `CI_MERGE_REQUEST_IID` is available only if [`only: [merge_requests]`](../merge_request_pipelines/index.md) is used and the merge request is created. -- `data-mr-url` is the URL of the GitLab instance and will be the same for all +- `data-mr-url` is the URL of the GitLab instance and is the same for all review apps. - `data-project-path` is the project's path, which can be found by `CI_PROJECT_PATH`. -- `data-require-auth` is optional for public projects but required for [private and internal ones](#authentication-for-visual-reviews). If this is set to `true`, the user will be required to enter their [personal access token](../../user/profile/personal_access_tokens.md) instead of their name and email. +- `data-require-auth` is optional for public projects but required for [private and internal ones](#authentication-for-visual-reviews). If this is set to `true`, the user is required to enter their [personal access token](../../user/profile/personal_access_tokens.md) instead of their name and email. - `id` is always `review-app-toolbar-script`, you don't need to change that. - `src` is the source of the review toolbar script, which resides in the - respective GitLab instance and will be the same for all review apps. + respective GitLab instance and is the same for all review apps. For example, in a Ruby application with code hosted on in a project GitLab.com, you would need to have this script: diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index c9a5f2f3899..d6ea4d83825 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -116,7 +116,7 @@ You can also enable shared runners for individual projects. To enable shared runners: 1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. -1. Click **Allow shared runners**. +1. Select **Enable shared runners for this project**. #### Disable shared runners @@ -126,7 +126,12 @@ You must have Owner permissions for the project or group. To disable shared runners for a project: 1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. -1. In the **Shared runners** area, click **Disable shared runners**. +1. In the **Shared runners** area, select **Enable shared runners for this project** so the toggle is grayed-out. + +Shared runners are automatically disabled for a project: + +- If the shared runners setting for the parent group is disabled, and +- If overriding this setting is not permitted at the project level. To disable shared runners for a group: @@ -264,6 +269,15 @@ by a project that has jobs with a long timeout (for example, one week). When not configured, runners do not override the project timeout. +On GitLab.com, you cannot override the job timeout for shared runners and must use the [project defined timeout](../pipelines/settings.md#timeout). + +To set the maximum job timeout: + +1. In a project, go to **Settings > CI/CD > Runners**. +1. Select your specific runner to edit the settings. +1. Enter a value under **Maximum job timeout**. +1. Select **Save changes**. + How this feature works: **Example 1 - Runner timeout bigger than project timeout** diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index 2505e56356d..1fec1f77bc3 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -68,36 +68,11 @@ execution time and the error output. ### Number of recent failures > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in GitLab 13.7. -> - It's [deployed behind a feature flag](../user/feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-the-number-of-recent-failures). **(CORE ONLY)** - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268249) in GitLab 13.8. If a test failed in the project's default branch in the last 14 days, a message like `Failed {n} time(s) in {default_branch} in the last 14 days` is displayed for that test. -#### Enable or disable the number of recent failures **(CORE ONLY)** - -Displaying the number of failures in the last 14 days is under development and not -ready for production use. It is deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) -can enable it. - -To enable it: - -```ruby -Feature.enable(:test_failure_history) -``` - -To disable it: - -```ruby -Feature.disable(:test_failure_history) -``` - ## How to set it up To enable the Unit test reports in merge requests, you need to add diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index f4ca51be151..5fca8e8c2b7 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -594,7 +594,35 @@ WARNING: Variables with multi-line values are not supported due to limitations with the Auto DevOps scripting environment. -### Override a variable by manually running a pipeline +### When you can override variables + +You can override the value of a variable when: + +1. [Manually running](#override-a-variable-by-manually-running-a-pipeline) pipelines in the UI. +1. Manually creating pipelines [via API](../../api/pipelines.md#create-a-new-pipeline). +1. Manually playing a job via the UI. +1. Using [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). +1. Manually triggering pipelines with [the API](../triggers/README.md#making-use-of-trigger-variables). +1. Passing variables to a [downstream pipeline](../multi_project_pipelines.md#passing-variables-to-a-downstream-pipeline). + +These pipeline variables declared in these events take [priority over other variables](#priority-of-environment-variables). + +#### Restrict who can override variables + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/295234) in GitLab 13.8. + +To allow only users with Maintainer role to set these variables, you can use +[the API](../../api/projects.md#edit-project) to enable the project setting `restrict_user_defined_variables`. +When a user without Maintainer role tries to run a pipeline with overridden +variables, an `Insufficient permissions to set pipeline variables` error occurs. + +The setting is `disabled` by default. + +If you [store your CI configurations in a different repository](../../ci/pipelines/settings.md#custom-ci-configuration-path), +use this setting for strict control over all aspects of the environment +the pipeline runs in. + +#### Override a variable by manually running a pipeline > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44059) in GitLab 10.8. @@ -785,7 +813,9 @@ Feature.enable(:ci_if_parenthesis_enabled) ### Storing regular expressions in variables -It is possible to store a regular expression in a variable, to be used for pattern matching: +It is possible to store a regular expression in a variable, to be used for pattern matching. +The following example tests whether `$RELEASE` contains either the +string `staging0` or the string `staging1`: ```yaml variables: @@ -847,13 +877,7 @@ before making them visible again. ### Restricted access to debug logging > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213159) in GitLab 13.7. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-restricted-access-to-debug-logging). **(CORE ONLY)** - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292661) in GitLab 13.8. With restricted access to debug logging, only users with [developer or higher permissions](../../user/permissions.md#project-members-permissions) @@ -867,25 +891,6 @@ If you add `CI_DEBUG_TRACE` as a local variable to your runners, debug logs are to all users with access to job logs. The permission levels are not checked by Runner, so you should make use of the variable in GitLab only. -#### Enable or disable Restricted access to debug logging **(CORE ONLY)** - -Restricted Access to Debug logging is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:restrict_access_to_build_debug_mode) -``` - -To disable it: - -```ruby -Feature.disable(:restrict_access_to_build_debug_mode) -``` - ### Enable Debug logging To enable debug logs (traces), set the `CI_DEBUG_TRACE` variable to `true`: diff --git a/doc/ci/variables/deprecated_variables.md b/doc/ci/variables/deprecated_variables.md index 755e34fa5ca..8d23ec1fd97 100644 --- a/doc/ci/variables/deprecated_variables.md +++ b/doc/ci/variables/deprecated_variables.md @@ -1,36 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'README.md' --- -# Deprecated GitLab CI/CD variables +This documentation page was removed. For information about variables, see [GitLab CI/CD environment variables](README.md) -Read through this document to learn what predefined variables -were deprecated and their new references. - -## GitLab 9.0 renamed variables - -To follow conventions of naming across GitLab, and to further move away from the -`build` term and toward `job`, some [CI/CD environment variables](README.md#predefined-environment-variables) were renamed for GitLab 9.0 -release. - -Starting with GitLab 9.0, we have deprecated the `$CI_BUILD_*` variables. **You are -strongly advised to use the new variables as we might remove the old ones in -future GitLab releases.** - -| 8.x name | 9.0+ name | -| --------------------- |------------------------ | -| `CI_BUILD_BEFORE_SHA` | `CI_COMMIT_BEFORE_SHA` | -| `CI_BUILD_ID` | `CI_JOB_ID` | -| `CI_BUILD_MANUAL` | `CI_JOB_MANUAL` | -| `CI_BUILD_NAME` | `CI_JOB_NAME` | -| `CI_BUILD_REF` | `CI_COMMIT_SHA` | -| `CI_BUILD_REF_NAME` | `CI_COMMIT_REF_NAME` | -| `CI_BUILD_REF_SLUG` | `CI_COMMIT_REF_SLUG` | -| `CI_BUILD_REPO` | `CI_REPOSITORY_URL` | -| `CI_BUILD_STAGE` | `CI_JOB_STAGE` | -| `CI_BUILD_TAG` | `CI_COMMIT_TAG` | -| `CI_BUILD_TOKEN` | `CI_JOB_TOKEN` | -| `CI_BUILD_TRIGGERED` | `CI_PIPELINE_TRIGGERED` | +<!-- This redirect file can be deleted after 2021-04-14. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index ba0a5e8f461..701fe33b53f 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -14,138 +14,134 @@ Some of the predefined environment variables are available only if a minimum version of [GitLab Runner](https://docs.gitlab.com/runner/) is used. Consult the table below to find the version of GitLab Runner that's required. -NOTE: -Starting with GitLab 9.0, we have deprecated some variables. Read the -[9.0 Renaming](deprecated_variables.md#gitlab-90-renamed-variables) section to find out their replacements. -**To avoid problems with deprecated and removed variables in future releases, you are strongly advised to use the new variables.** - You can add a command to your `.gitlab-ci.yml` file to [output the values of all variables available for a job](README.md#list-all-environment-variables). Kubernetes-specific environment variables are detailed in the [Kubernetes deployment variables](../../user/project/clusters/index.md#deployment-variables) section. -| Variable | GitLab | Runner | Description | -|-----------------------------------------------|--------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `CHAT_CHANNEL` | 10.6 | all | Source chat channel which triggered the [ChatOps](../chatops/README.md) command | -| `CHAT_INPUT` | 10.6 | all | Additional arguments passed in the [ChatOps](../chatops/README.md) command | -| `CI` | all | 0.4 | Mark that job is executed in CI environment | -| `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL | -| `CI_BUILDS_DIR` | all | 11.10 | Top-level directory where builds are executed. | -| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch. Is always `0000000000000000000000000000000000000000` in pipelines for merge requests. | -| `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. | -| `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | -| `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built | -| `CI_COMMIT_REF_PROTECTED` | 11.11 | all | `true` if the job is running on a protected reference, `false` if not | -| `CI_COMMIT_REF_SLUG` | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | -| `CI_COMMIT_SHA` | 9.0 | all | The commit revision for which project is built | -| `CI_COMMIT_SHORT_SHA` | 11.7 | all | The first eight characters of `CI_COMMIT_SHA` | -| `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Present in branch pipelines, including pipelines for the default branch. Not present in merge request pipelines or tag pipelines. | -| `CI_COMMIT_TAG` | 9.0 | 0.5 | The commit tag name. Present only when building tags. | -| `CI_COMMIT_TITLE` | 10.8 | all | The title of the commit - the full first line of the message | -| `CI_COMMIT_TIMESTAMP` | 13.4 | all | The timestamp of the commit in the ISO 8601 format. | -| `CI_CONCURRENT_ID` | all | 11.10 | Unique ID of build execution within a single executor. | -| `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | Unique ID of build execution within a single executor and project. | -| `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to CI configuration file. Defaults to `.gitlab-ci.yml` | -| `CI_DEBUG_TRACE` | all | 1.7 | Whether [debug logging (tracing)](README.md#debug-logging) is enabled | -| `CI_DEFAULT_BRANCH` | 12.4 | all | The name of the default branch for the project. | -| `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | 13.7 | all | The image prefix for pulling images through the Dependency Proxy. | -| `CI_DEPENDENCY_PROXY_SERVER` | 13.7 | all | The server for logging in to the Dependency Proxy. This is equivelant to `$CI_SERVER_HOST:$CI_SERVER_PORT`. | -| `CI_DEPENDENCY_PROXY_PASSWORD` | 13.7 | all | The password to use to pull images through the Dependency Proxy. | -| `CI_DEPENDENCY_PROXY_USER` | 13.7 | all | The username to use to pull images through the Dependency Proxy. | -| `CI_DEPLOY_FREEZE` | 13.2 | all | Included with the value `true` if the pipeline runs during a [deploy freeze window](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). | -| `CI_DEPLOY_PASSWORD` | 10.8 | all | Authentication password of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | -| `CI_DEPLOY_USER` | 10.8 | all | Authentication username of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | -| `CI_DISPOSABLE_ENVIRONMENT` | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. | -| `CI_ENVIRONMENT_NAME` | 8.15 | all | The name of the environment for this job. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. | -| `CI_ENVIRONMENT_SLUG` | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. | -| `CI_ENVIRONMENT_URL` | 9.3 | all | The URL of the environment for this job. Only present if [`environment:url`](../yaml/README.md#environmenturl) is set. | -| `CI_EXTERNAL_PULL_REQUEST_IID` | 12.3 | all | Pull Request ID from GitHub if the [pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_SOURCE_REPOSITORY` | 13.3 | all | The source repository name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_TARGET_REPOSITORY` | 13.3 | all | The target repository name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_NAME` | 12.3 | all | The source branch name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the source branch of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_NAME` | 12.3 | all | The target branch name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the target branch of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | -| `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Included with the value `true` only if the pipeline's project has any open [requirements](../../user/project/requirements/index.md). Not included if there are no open requirements for the pipeline's project. | -| `CI_OPEN_MERGE_REQUESTS` | 13.7 | all | Contains a comma-delimited list of up to 4 Merge Requests from the current source project and branch in the form `gitlab-org/gitlab!333,gitlab-org/gitlab-foss!11` | -| `CI_JOB_ID` | 9.0 | all | The unique ID of the current job that GitLab CI/CD uses internally | -| `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the image running the CI job | -| `CI_JOB_MANUAL` | 8.12 | all | The flag to indicate that job was manually started | -| `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml` | -| `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | -| `CI_JOB_STATUS` | all | 13.5 | The state of the job as each runner stage is executed. Use with [`after_script`](../yaml/README.md#after_script) where `CI_JOB_STATUS` can be either: `success`, `failed` or `canceled`. | +| Variable | GitLab | Runner | Description | +|-----------------------------------------------|--------|--------|-------------| +| `CHAT_CHANNEL` | 10.6 | all | Source chat channel which triggered the [ChatOps](../chatops/README.md) command. | +| `CHAT_INPUT` | 10.6 | all | Additional arguments passed in the [ChatOps](../chatops/README.md) command. | +| `CI` | all | 0.4 | Mark that job is executed in CI environment. | +| `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL. | +| `CI_BUILDS_DIR` | all | 11.10 | Top-level directory where builds are executed. | +| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch. Is always `0000000000000000000000000000000000000000` in pipelines for merge requests. | +| `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. | +| `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | +| `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built. | +| `CI_COMMIT_REF_PROTECTED` | 11.11 | all | `true` if the job is running on a protected reference, `false` if not. | +| `CI_COMMIT_REF_SLUG` | 9.0 | all | `$CI_COMMIT_REF_NAME` in lowercase, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | +| `CI_COMMIT_SHA` | 9.0 | all | The commit revision for which project is built. | +| `CI_COMMIT_SHORT_SHA` | 11.7 | all | The first eight characters of `CI_COMMIT_SHA`. | +| `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Present in branch pipelines, including pipelines for the default branch. Not present in merge request pipelines or tag pipelines. | +| `CI_COMMIT_TAG` | 9.0 | 0.5 | The commit tag name. Present only when building tags. | +| `CI_COMMIT_TITLE` | 10.8 | all | The title of the commit - the full first line of the message. | +| `CI_COMMIT_TIMESTAMP` | 13.4 | all | The timestamp of the commit in the ISO 8601 format. | +| `CI_CONCURRENT_ID` | all | 11.10 | Unique ID of build execution in a single executor. | +| `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | Unique ID of build execution in a single executor and project. | +| `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to CI configuration file. Defaults to `.gitlab-ci.yml`. | +| `CI_DEBUG_TRACE` | all | 1.7 | Whether [debug logging (tracing)](README.md#debug-logging) is enabled. | +| `CI_DEFAULT_BRANCH` | 12.4 | all | The name of the default branch for the project. | +| `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | 13.7 | all | The image prefix for pulling images through the Dependency Proxy. | +| `CI_DEPENDENCY_PROXY_SERVER` | 13.7 | all | The server for logging in to the Dependency Proxy. This is equivalent to `$CI_SERVER_HOST:$CI_SERVER_PORT`. | +| `CI_DEPENDENCY_PROXY_PASSWORD` | 13.7 | all | The password to use to pull images through the Dependency Proxy. | +| `CI_DEPENDENCY_PROXY_USER` | 13.7 | all | The username to use to pull images through the Dependency Proxy. | +| `CI_DEPLOY_FREEZE` | 13.2 | all | Included with the value `true` if the pipeline runs during a [deploy freeze window](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). | +| `CI_DEPLOY_PASSWORD` | 10.8 | all | Authentication password of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | +| `CI_DEPLOY_USER` | 10.8 | all | Authentication username of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | +| `CI_DISPOSABLE_ENVIRONMENT` | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. | +| `CI_ENVIRONMENT_NAME` | 8.15 | all | The name of the environment for this job. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. | +| `CI_ENVIRONMENT_SLUG` | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, and so on. Only present if [`environment:name`](../yaml/README.md#environmentname) is set. | +| `CI_ENVIRONMENT_URL` | 9.3 | all | The URL of the environment for this job. Only present if [`environment:url`](../yaml/README.md#environmenturl) is set. | +| `CI_EXTERNAL_PULL_REQUEST_IID` | 12.3 | all | Pull Request ID from GitHub if the [pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | +| `CI_EXTERNAL_PULL_REQUEST_SOURCE_REPOSITORY` | 13.3 | all | The source repository name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | +| `CI_EXTERNAL_PULL_REQUEST_TARGET_REPOSITORY` | 13.3 | all | The target repository name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | +| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_NAME` | 12.3 | all | The source branch name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | +| `CI_EXTERNAL_PULL_REQUEST_SOURCE_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the source branch of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | +| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_NAME` | 12.3 | all | The target branch name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | +| `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the target branch of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | +| `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Included with the value `true` only if the pipeline's project has any open [requirements](../../user/project/requirements/index.md). Not included if there are no open requirements for the pipeline's project. | +| `CI_OPEN_MERGE_REQUESTS` | 13.8 | all | Available in branch and merge request pipelines. Contains a comma-separated list of up to four merge requests that use the current branch and project as the merge request source. For example `gitlab-org/gitlab!333,gitlab-org/gitlab-foss!11`. | +| `CI_JOB_ID` | 9.0 | all | The unique ID of the current job that GitLab CI/CD uses internally. | +| `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the image running the CI job. | +| `CI_JOB_MANUAL` | 8.12 | all | The flag to indicate that job was manually started. | +| `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml`. | +| `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml`. | +| `CI_JOB_STATUS` | all | 13.5 | The state of the job as each runner stage is executed. Use with [`after_script`](../yaml/README.md#after_script) where `CI_JOB_STATUS` can be either: `success`, `failed` or `canceled`. | | `CI_JOB_TOKEN` | 9.0 | 1.2 | Token used for authenticating with [a few API endpoints](../../api/README.md#gitlab-ci-job-token) and downloading [dependent repositories](../../user/project/new_ci_build_permissions_model.md#dependent-repositories). The token is valid as long as the job is running. | | `CI_JOB_JWT` | 12.10 | all | RS256 JSON web token that can be used for authenticating with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | -| `CI_JOB_URL` | 11.1 | 0.5 | Job details URL | -| `CI_KUBERNETES_ACTIVE` | 13.0 | all | Included with the value `true` only if the pipeline has a Kubernetes cluster available for deployments. Not included if no cluster is available. Can be used as an alternative to [`only:kubernetes`/`except:kubernetes`](../yaml/README.md#onlykubernetesexceptkubernetes) with [`rules:if`](../yaml/README.md#rulesif) | -| `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of username(s) of assignee(s) for the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_ID` | 11.6 | all | The instance-level ID of the merge request. Only available if [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. This is a unique ID across all projects on GitLab. | -| `CI_MERGE_REQUEST_IID` | 11.6 | all | The project-level IID (internal ID) of the merge request. Only available If [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. This ID is unique for the current project. | -| `CI_MERGE_REQUEST_LABELS` | 11.9 | all | Comma-separated label names of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_MILESTONE` | 11.9 | all | The milestone title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_PROJECT_ID` | 11.6 | all | The ID of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_PROJECT_PATH` | 11.6 | all | The path of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) (e.g. `namespace/awesome-project`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_PROJECT_URL` | 11.6 | all | The URL of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) (e.g. `http://192.168.10.15:3000/namespace/awesome-project`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_REF_PATH` | 11.6 | all | The ref path of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). (e.g. `refs/merge-requests/1/head`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME` | 11.6 | all | The source branch name of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the source branch of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used, the merge request is created, and the pipeline is a [merged result pipeline](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | -| `CI_MERGE_REQUEST_SOURCE_PROJECT_ID` | 11.6 | all | The ID of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_PROJECT_PATH` | 11.6 | all | The path of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_SOURCE_PROJECT_URL` | 11.6 | all | The URL of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_TARGET_BRANCH_NAME` | 11.6 | all | The target branch name of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the target branch of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used, the merge request is created, and the pipeline is a [merged result pipeline](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | -| `CI_MERGE_REQUEST_TITLE` | 11.9 | all | The title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_JOB_URL` | 11.1 | 0.5 | Job details URL. | +| `CI_KUBERNETES_ACTIVE` | 13.0 | all | Included with the value `true` only if the pipeline has a Kubernetes cluster available for deployments. Not included if no cluster is available. Can be used as an alternative to [`only:kubernetes`/`except:kubernetes`](../yaml/README.md#onlykubernetesexceptkubernetes) with [`rules:if`](../yaml/README.md#rulesif). | +| `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of username(s) of assignee(s) for the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_ID` | 11.6 | all | The instance-level ID of the merge request. Only available if [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. This is a unique ID across all projects on GitLab. | +| `CI_MERGE_REQUEST_IID` | 11.6 | all | The project-level IID (internal ID) of the merge request. Only available If [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. This ID is unique for the current project. | +| `CI_MERGE_REQUEST_LABELS` | 11.9 | all | Comma-separated label names of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_MILESTONE` | 11.9 | all | The milestone title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_PROJECT_ID` | 11.6 | all | The ID of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_PROJECT_PATH` | 11.6 | all | The path of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) (for example `namespace/awesome-project`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_PROJECT_URL` | 11.6 | all | The URL of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) (for example `http://192.168.10.15:3000/namespace/awesome-project`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_REF_PATH` | 11.6 | all | The ref path of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). (for example `refs/merge-requests/1/head`). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME` | 11.6 | all | The source branch name of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the source branch of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used, the merge request is created, and the pipeline is a [merged result pipeline](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | +| `CI_MERGE_REQUEST_SOURCE_PROJECT_ID` | 11.6 | all | The ID of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_SOURCE_PROJECT_PATH` | 11.6 | all | The path of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_SOURCE_PROJECT_URL` | 11.6 | all | The URL of the source project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_TARGET_BRANCH_NAME` | 11.6 | all | The target branch name of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the target branch of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used, the merge request is created, and the pipeline is a [merged result pipeline](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | +| `CI_MERGE_REQUEST_TITLE` | 11.9 | all | The title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | | `CI_MERGE_REQUEST_EVENT_TYPE` | 12.3 | all | The event type of the merge request, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Can be `detached`, `merged_result` or `merge_train`. | | `CI_MERGE_REQUEST_DIFF_ID` | 13.7 | all | The version of the merge request diff, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). | | `CI_MERGE_REQUEST_DIFF_BASE_SHA` | 13.7 | all | The base SHA of the merge request diff, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). | -| `CI_NODE_INDEX` | 11.5 | all | Index of the job in the job set. If the job is not parallelized, this variable is not set. | -| `CI_NODE_TOTAL` | 11.5 | all | Total number of instances of this job running in parallel. If the job is not parallelized, this variable is set to `1`. | -| `CI_PAGES_DOMAIN` | 11.8 | all | The configured domain that hosts GitLab Pages. | -| `CI_PAGES_URL` | 11.8 | all | URL to GitLab Pages-built pages. Always belongs to a subdomain of `CI_PAGES_DOMAIN`. | -| `CI_PIPELINE_ID` | 8.10 | all | The instance-level ID of the current pipeline. This is a unique ID across all projects on GitLab. | -| `CI_PIPELINE_IID` | 11.0 | all | The project-level IID (internal ID) of the current pipeline. This ID is unique for the current project. | +| `CI_NODE_INDEX` | 11.5 | all | Index of the job in the job set. If the job is not parallelized, this variable is not set. | +| `CI_NODE_TOTAL` | 11.5 | all | Total number of instances of this job running in parallel. If the job is not parallelized, this variable is set to `1`. | +| `CI_PAGES_DOMAIN` | 11.8 | all | The configured domain that hosts GitLab Pages. | +| `CI_PAGES_URL` | 11.8 | all | URL to GitLab Pages-built pages. Always belongs to a subdomain of `CI_PAGES_DOMAIN`. | +| `CI_PIPELINE_ID` | 8.10 | all | The instance-level ID of the current pipeline. This is a unique ID across all projects on GitLab. | +| `CI_PIPELINE_IID` | 11.0 | all | The project-level IID (internal ID) of the current pipeline. This ID is unique for the current project. | | `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/README.md#authentication-tokens). For pipelines created before GitLab 9.5, this is displayed as `unknown`. | -| `CI_PIPELINE_TRIGGERED` | all | all | The flag to indicate that job was [triggered](../triggers/README.md) | -| `CI_PIPELINE_URL` | 11.1 | 0.5 | Pipeline details URL | +| `CI_PIPELINE_TRIGGERED` | all | all | The flag to indicate that job was [triggered](../triggers/README.md). | +| `CI_PIPELINE_URL` | 11.1 | 0.5 | Pipeline details URL. | +| `CI_PROJECT_CONFIG_PATH` | 13.8 | all | The CI configuration path for the project. | | `CI_PROJECT_DIR` | all | all | The full path where the repository is cloned and where the job is run. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see [Advanced configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) for GitLab Runner. | -| `CI_PROJECT_ID` | all | all | The unique ID of the current project that GitLab CI/CD uses internally | -| `CI_PROJECT_NAME` | 8.10 | 0.5 | The name of the directory for the project that is currently being built. For example, if the project URL is `gitlab.example.com/group-name/project-1`, the `CI_PROJECT_NAME` would be `project-1`. | -| `CI_PROJECT_NAMESPACE` | 8.10 | 0.5 | The project namespace (username or group name) that is currently being built | -| `CI_PROJECT_ROOT_NAMESPACE` | 13.2 | 0.5 | The **root** project namespace (username or group name) that is currently being built. For example, if `CI_PROJECT_NAMESPACE` is `root-group/child-group/grandchild-group`, `CI_PROJECT_ROOT_NAMESPACE` would be `root-group`. | -| `CI_PROJECT_PATH` | 8.10 | 0.5 | The namespace with project name | -| `CI_PROJECT_PATH_SLUG` | 9.3 | all | `$CI_PROJECT_PATH` lowercased and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | -| `CI_PROJECT_REPOSITORY_LANGUAGES` | 12.3 | all | Comma-separated, lowercased list of the languages used in the repository (e.g. `ruby,javascript,html,css`) | -| `CI_PROJECT_TITLE` | 12.4 | all | The human-readable project name as displayed in the GitLab web interface. | -| `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address to access project | -| `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility (internal, private, public) | -| `CI_REGISTRY` | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of the GitLab Container Registry. This variable includes a `:port` value if one has been specified in the registry configuration. | -| `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | -| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to use to push containers to the GitLab Container Registry, for the current project. | -| `CI_REGISTRY_USER` | 9.0 | all | The username to use to push containers to the GitLab Container Registry, for the current project. | -| `CI_REPOSITORY_URL` | 9.0 | all | The URL to clone the Git repository | -| `CI_RUNNER_DESCRIPTION` | 8.10 | 0.5 | The description of the runner as saved in GitLab | -| `CI_RUNNER_EXECUTABLE_ARCH` | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor) | -| `CI_RUNNER_ID` | 8.10 | 0.5 | The unique ID of runner being used | -| `CI_RUNNER_REVISION` | all | 10.6 | GitLab Runner revision that is executing the current job | -| `CI_RUNNER_SHORT_TOKEN` | all | 12.3 | First eight characters of the runner's token used to authenticate new job requests. Used as the runner's unique ID | -| `CI_RUNNER_TAGS` | 8.10 | 0.5 | The defined runner tags | -| `CI_RUNNER_VERSION` | all | 10.6 | GitLab Runner version that is executing the current job | -| `CI_SERVER` | all | all | Mark that job is executed in CI environment | -| `CI_SERVER_URL` | 12.7 | all | The base URL of the GitLab instance, including protocol and port (like `https://gitlab.example.com:8080`) | -| `CI_SERVER_HOST` | 12.1 | all | Host component of the GitLab instance URL, without protocol and port (like `gitlab.example.com`) | -| `CI_SERVER_PORT` | 12.8 | all | Port component of the GitLab instance URL, without host and protocol (like `3000`) | -| `CI_SERVER_PROTOCOL` | 12.8 | all | Protocol component of the GitLab instance URL, without host and port (like `https`) | -| `CI_SERVER_NAME` | all | all | The name of CI server that is used to coordinate jobs | -| `CI_SERVER_REVISION` | all | all | GitLab revision that is used to schedule jobs | -| `CI_SERVER_VERSION` | all | all | GitLab version that is used to schedule jobs | -| `CI_SERVER_VERSION_MAJOR` | 11.4 | all | GitLab version major component | -| `CI_SERVER_VERSION_MINOR` | 11.4 | all | GitLab version minor component | -| `CI_SERVER_VERSION_PATCH` | 11.4 | all | GitLab version patch component | -| `CI_SHARED_ENVIRONMENT` | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | -| `GITLAB_CI` | all | all | Mark that job is executed in GitLab CI/CD environment | -| `GITLAB_FEATURES` | 10.6 | all | The comma separated list of licensed features available for your instance and plan | -| `GITLAB_USER_EMAIL` | 8.12 | all | The email of the user who started the job | -| `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the job | -| `GITLAB_USER_LOGIN` | 10.0 | all | The login username of the user who started the job | -| `GITLAB_USER_NAME` | 10.0 | all | The real name of the user who started the job | +| `CI_PROJECT_ID` | all | all | The unique ID of the current project that GitLab CI/CD uses internally. | +| `CI_PROJECT_NAME` | 8.10 | 0.5 | The name of the directory for the project that is being built. For example, if the project URL is `gitlab.example.com/group-name/project-1`, the `CI_PROJECT_NAME` would be `project-1`. | +| `CI_PROJECT_NAMESPACE` | 8.10 | 0.5 | The project namespace (username or group name) that is being built. | +| `CI_PROJECT_ROOT_NAMESPACE` | 13.2 | 0.5 | The **root** project namespace (username or group name) that is being built. For example, if `CI_PROJECT_NAMESPACE` is `root-group/child-group/grandchild-group`, `CI_PROJECT_ROOT_NAMESPACE` would be `root-group`. | +| `CI_PROJECT_PATH` | 8.10 | 0.5 | The namespace with project name. | +| `CI_PROJECT_PATH_SLUG` | 9.3 | all | `$CI_PROJECT_PATH` in lowercase and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | +| `CI_PROJECT_REPOSITORY_LANGUAGES` | 12.3 | all | Comma-separated, lowercase list of the languages used in the repository (for example `ruby,javascript,html,css`). | +| `CI_PROJECT_TITLE` | 12.4 | all | The human-readable project name as displayed in the GitLab web interface. | +| `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address to access project. | +| `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility (internal, private, public). | +| `CI_REGISTRY` | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of the GitLab Container Registry. This variable includes a `:port` value if one has been specified in the registry configuration. | +| `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project. | +| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to use to push containers to the GitLab Container Registry, for the current project. | +| `CI_REGISTRY_USER` | 9.0 | all | The username to use to push containers to the GitLab Container Registry, for the current project. | +| `CI_REPOSITORY_URL` | 9.0 | all | The URL to clone the Git repository. | +| `CI_RUNNER_DESCRIPTION` | 8.10 | 0.5 | The description of the runner as saved in GitLab. | +| `CI_RUNNER_EXECUTABLE_ARCH` | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor). | +| `CI_RUNNER_ID` | 8.10 | 0.5 | The unique ID of runner being used. | +| `CI_RUNNER_REVISION` | all | 10.6 | GitLab Runner revision that is executing the current job. | +| `CI_RUNNER_SHORT_TOKEN` | all | 12.3 | First eight characters of the runner's token used to authenticate new job requests. Used as the runner's unique ID. | +| `CI_RUNNER_TAGS` | 8.10 | 0.5 | The defined runner tags. | +| `CI_RUNNER_VERSION` | all | 10.6 | GitLab Runner version that is executing the current job. | +| `CI_SERVER` | all | all | Mark that job is executed in CI environment. | +| `CI_SERVER_URL` | 12.7 | all | The base URL of the GitLab instance, including protocol and port (like `https://gitlab.example.com:8080`). | +| `CI_SERVER_HOST` | 12.1 | all | Host component of the GitLab instance URL, without protocol and port (like `gitlab.example.com`). | +| `CI_SERVER_PORT` | 12.8 | all | Port component of the GitLab instance URL, without host and protocol (like `3000`). | +| `CI_SERVER_PROTOCOL` | 12.8 | all | Protocol component of the GitLab instance URL, without host and port (like `https`). | +| `CI_SERVER_NAME` | all | all | The name of CI server that is used to coordinate jobs. | +| `CI_SERVER_REVISION` | all | all | GitLab revision that is used to schedule jobs. | +| `CI_SERVER_VERSION` | all | all | GitLab version that is used to schedule jobs. | +| `CI_SERVER_VERSION_MAJOR` | 11.4 | all | GitLab version major component. | +| `CI_SERVER_VERSION_MINOR` | 11.4 | all | GitLab version minor component. | +| `CI_SERVER_VERSION_PATCH` | 11.4 | all | GitLab version patch component. | +| `CI_SHARED_ENVIRONMENT` | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | +| `GITLAB_CI` | all | all | Mark that job is executed in GitLab CI/CD environment. | +| `GITLAB_FEATURES` | 10.6 | all | The comma separated list of licensed features available for your instance and plan. | +| `GITLAB_USER_EMAIL` | 8.12 | all | The email of the user who started the job. | +| `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the job. | +| `GITLAB_USER_LOGIN` | 10.0 | all | The login username of the user who started the job. | +| `GITLAB_USER_NAME` | 10.0 | all | The real name of the user who started the job. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index f2dc58bc144..e84714f2a46 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -27,6 +27,7 @@ There are two places defined variables can be used. On the: | `environment:url` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | | `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | `resource_group` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| `include` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>Predefined project variables are supported: `GITLAB_FEATURES`, `CI_DEFAULT_BRANCH`, and all variables that start with `CI_PROJECT_` (for example `CI_PROJECT_NAME`). | | `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | | `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | | `services:[]` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 7ca74cdf2a2..19b8f0f1c89 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -363,8 +363,6 @@ use the [`extends` keyword](#extends). | [`remote`](#includeremote) | Include a file from a remote URL. Must be publicly accessible. | | [`template`](#includetemplate) | Include templates that are provided by GitLab. | -The `include` methods do not support [variable expansion](../variables/where_variables_can_be_used.md#variables-usage). - `.gitlab-ci.yml` configuration included by all methods is evaluated at pipeline creation. The configuration is a snapshot in time and persisted in the database. Any changes to referenced `.gitlab-ci.yml` configuration is not reflected in GitLab until the next pipeline is created. @@ -379,6 +377,48 @@ NOTE: Use merging to customize and override included CI/CD configurations with local definitions. Local definitions in `.gitlab-ci.yml` override included definitions. +#### Variables with `include` + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/284883) in GitLab 13.8. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-includepredefined-project-variables). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +You can [use some predefined variables in `include` sections](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) +in your `.gitlab-ci.yml`: + +```yaml +include: + project: '$CI_PROJECT_PATH' + file: '.compliance-gitlab-ci.yml' +``` + +For an example of how you can include these predefined variables, and their impact on CI jobs, +see the following [CI variable demo](https://youtu.be/4XR8gw3Pkos). + +##### Enable or disable include:predefined-project-variables **(CORE ONLY)** + +Use of predefined project variables in `include` section of `.gitlab-ci.yml` is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:variables_in_include_section_ci) +``` + +To disable it: + +```ruby +Feature.disable(:variables_in_include_section_ci) +``` + #### `include:local` `include:local` includes a file from the same repository as `.gitlab-ci.yml`. @@ -412,9 +452,10 @@ include: '.gitlab-ci-production.yml' > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53903) in GitLab 11.7. -To include files from another private project under the same GitLab instance, -use `include:file`. This file is referenced with full paths relative to the -root directory (`/`). For example: +To include files from another private project on the same GitLab instance, +use `include:file`. You can use `include:file` in combination with `include:project` only. + +The included file is referenced with a full path, relative to the root directory (`/`). For example: ```yaml include: @@ -445,10 +486,7 @@ You can use local (relative to target project), project, remote, or template inc ##### Multiple files from a project > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26793) in GitLab 13.6. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to disable it. **(CORE ONLY)** +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/271560) in GitLab 13.8. You can include multiple files from the same project: @@ -461,23 +499,6 @@ include: - '/templates/.tests.yml' ``` -Including multiple files from the same project is under development but ready for production use. It is -deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:ci_include_multiple_files_from_project) -``` - -To disable it: - -```ruby -Feature.disable(:ci_include_multiple_files_from_project) -``` - #### `include:remote` `include:remote` can be used to include a file from a different location, @@ -615,10 +636,33 @@ job: ``` Sometimes, `script` commands must be wrapped in single or double quotes. -For example, commands that contain a colon (`:`) must be wrapped in quotes. +For example, commands that contain a colon (`:`) must be wrapped in single quotes (`'`). The YAML parser needs to interpret the text as a string rather than -a "key: value" pair. Be careful when using special characters: -`:`, `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. +a "key: value" pair. + +For example, this script uses a colon: + +```yaml +job: + script: + - curl --request POST --header 'Content-Type: application/json' "https://gitlab/api/v4/projects" +``` + +To be considered valid YAML, you must wrap the entire command in single quotes. If +the command already uses single quotes, you should change them to double quotes (`"`) +if possible: + +```yaml +job: + script: + - 'curl --request POST --header "Content-Type: application/json" "https://gitlab/api/v4/projects"' +``` + +You can verify the syntax is valid with the [CI Lint](../lint.md) tool. + +Be careful when using these special characters as well: + +- `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. If any of the script commands return an exit code other than zero, the job fails and further commands are not executed. Store the exit code in a variable to @@ -1414,10 +1458,11 @@ In this example, if the first rule matches, then the job has `when: manual` and #### `rules:variables` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/209864) in GitLab 13.7. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-rulesvariables). **(CORE ONLY)** +> - It was [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/289803) on GitLab 13.8. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-rulesvariables). **(CORE ONLY)** WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -1444,10 +1489,10 @@ job: ##### Enable or disable rules:variables **(CORE ONLY)** -rules:variables is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. +rules:variables is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can enable it. +can opt to disable it. To enable it: @@ -2270,6 +2315,58 @@ job3: - deploy_to_staging ``` +#### `allow_failure:exit_codes` + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/273157) in GitLab 13.8. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-allow_failureexit_codes). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +Use `allow_failure:exit_codes` to dynamically control if a job should be allowed +to fail. You can list which exit codes are not considered failures. The job fails +for any other exit code: + +```yaml +test_job_1: + script: + - echo "Run a script that results in exit code 1. This job fails." + - exit 1 + allow_failure: + exit_codes: 137 + +test_job_2: + script: + - echo "Run a script that results in exit code 137. This job is allowed to fail." + - exit 137 + allow_failure: + exit_codes: + - 137 + - 255 +``` + +##### Enable or disable `allow_failure:exit_codes` **(CORE ONLY)** + +`allow_failure:exit_codes` is under development but ready for production use. It is +deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can disable it. + +To disable it: + +```ruby +Feature.disable(:ci_allow_failure_with_exit_codes) +``` + +To enable it: + +```ruby +Feature.enable(:ci_allow_failure_with_exit_codes) +``` + ### `when` `when` is used to implement jobs that are run in case of failure or despite the @@ -2961,8 +3058,6 @@ larger than the [maximum artifact size](../../user/gitlab_com/index.md#gitlab-ci Job artifacts are only collected for successful jobs by default, and artifacts are restored after [caches](#cache). -[Not all executors can use caches](https://docs.gitlab.com/runner/executors/#compatibility-chart). - [Read more about artifacts](../pipelines/job_artifacts.md). #### `artifacts:paths` @@ -3266,7 +3361,7 @@ job: The latest artifacts for refs are locked against deletion, and kept regardless of the expiry time. [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/16267) GitLab 13.0 behind a disabled feature flag, and [made the default behavior](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) -in GitLab 13.4. +in GitLab 13.4. In [GitLab 13.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/241026), you can [disable this behavior in the CI/CD settings](../pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). #### `artifacts:reports` @@ -3386,6 +3481,10 @@ If there is more than one matched line in the job output, the last line is used. For the matched line, the first occurence of `\d+(\.\d+)?` is the code coverage. Leading zeros are removed. +Coverage output from [child pipelines](../parent_child_pipelines.md) is not recorded +or displayed. Check [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/280818) +for more details. + ### `retry` > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/3442) in GitLab 9.5. @@ -3536,15 +3635,13 @@ job split into three separate jobs. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15356) in GitLab 13.3. -Use `matrix:` to configure different variables for jobs that are running in parallel. +Use `matrix:` to run a job multiple times in parallel in a single pipeline, +but with different variable values for each instance of the job. There can be from 2 to 50 jobs. Jobs can only run in parallel if there are multiple runners, or a single runner is [configured to run multiple jobs concurrently](#using-your-own-runners). -[In GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/26362) and later, -you can have one-dimensional matrices with a single job. - Every job gets the same `CI_NODE_TOTAL` [environment variable](../variables/README.md#predefined-environment-variables) value, and a unique `CI_NODE_INDEX` value. ```yaml @@ -3583,6 +3680,22 @@ deploystacks: [vultr, processing] The job naming style was [improved in GitLab 13.4](https://gitlab.com/gitlab-org/gitlab/-/issues/230452). +##### One-dimensional `matrix` jobs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26362) in GitLab 13.5. + +You can also have one-dimensional matrices with a single job: + +```yaml +deploystacks: + stage: deploy + script: + - bin/deploy + parallel: + matrix: + - PROVIDER: [aws, ovh, gcp, vultr] +``` + ### `trigger` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. @@ -3933,7 +4046,23 @@ The Release name. If omitted, it is populated with the value of `release: tag_na #### `release:description` -Specifies the longer description of the Release. +Specifies the long description of the Release. You can also specify a file that contains the +description. + +##### Read description from a file + +> [Introduced](https://gitlab.com/gitlab-org/release-cli/-/merge_requests/67) in GitLab 13.7. + +You can specify a file in `$CI_PROJECT_DIR` that contains the description. The file must be relative +to the project directory (`$CI_PROJECT_DIR`), and if the file is a symbolic link it can't reside +outside of `$CI_PROJECT_DIR`. The `./path/to/file` and file name can't contain spaces. + +```yaml +job: + release: + tag_name: ${MAJOR}_${MINOR}_${REVISION} + description: './path/to/CHANGELOG.md' +``` #### `release:ref` diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md index 602e02dbe6b..e4ede9cf699 100644 --- a/doc/ci/yaml/gitlab_ci_yaml.md +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -27,7 +27,7 @@ The scripts are grouped into **jobs**, and jobs run as part of a larger **pipeline**. You can group multiple independent jobs into **stages** that run in a defined order. You should organize your jobs in a sequence that suits your application and is in accordance with -the tests you wish to perform. To [visualize](visualization.md) the process, imagine +the tests you wish to perform. To [visualize](../pipeline_editor/index.md#visualize-ci-configuration) the process, imagine the scripts you add to jobs are the same as CLI commands you run on your computer. When you add a `.gitlab-ci.yml` file to your diff --git a/doc/ci/yaml/visualization.md b/doc/ci/yaml/visualization.md index 59a92370c70..ff3b0456eca 100644 --- a/doc/ci/yaml/visualization.md +++ b/doc/ci/yaml/visualization.md @@ -1,52 +1,8 @@ --- -stage: Verify -group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: '../pipeline_editor/index.md#visualize-ci-configuration' --- -# Visualize your CI/CD configuration +This document was moved to [another location](../pipeline_editor/index.md#visualize-ci-configuration). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241722) in GitLab 13.5. -> - [Moved to **CI/CD > Editor**](https://gitlab.com/gitlab-org/gitlab/-/issues/263141) in GitLab 13.7. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-cicd-configuration-visualization). **(CORE ONLY)** - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - -To see a visualization of your `gitlab-ci.yml` configuration, navigate to **CI/CD > Editor** -and select the `Visualization` tab. The visualization shows all stages and jobs. -[`needs`](README.md#needs) relationships are displayed as lines connecting jobs together, showing the hierarchy of execution: - - - -Hovering on a job highlights its `needs` relationships: - - - -If the configuration does not have any `needs` relationships, then no lines are drawn because -each job depends only on the previous stage being completed successfully. - -You can only preview one `gitlab-ci.yml` file at a time. Configuration imported with -[`includes`](README.md#include) is ignored and not included in the visualization. - -## Enable or disable CI/CD configuration visualization **(CORE ONLY)** - -CI/CD configuration visualization is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can enable it. - -To enable it: - -```ruby -Feature.enable(:ci_config_visualization_tab) -``` - -To disable it: - -```ruby -Feature.disable(:ci_config_visualization_tab) -``` +<!-- This redirect file can be deleted after 2021-04-13. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> |
