diff options
Diffstat (limited to 'doc/ci')
42 files changed, 1286 insertions, 605 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 150f160b762..43e66adbd35 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -16,9 +16,9 @@ through the [continuous methodologies](introduction/index.md#introduction-to-cic - Continuous Delivery (CD) - Continuous Deployment (CD) -NOTE: **Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more.** -Watch our -["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +NOTE: **Note:** +Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more. +Watch our ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. ## Overview @@ -58,7 +58,7 @@ the following documents: - [How GitLab CI/CD works](introduction/index.md#how-gitlab-cicd-works). - [Fundamental pipeline architectures](pipelines/pipeline_architectures.md). - [GitLab CI/CD basic workflow](introduction/index.md#basic-cicd-workflow). -- [Step-by-step guide for writing `.gitlab-ci.yml` for the first time](../user/project/pages/getting_started_part_four.md). +- [Step-by-step guide for writing `.gitlab-ci.yml` for the first time](../user/project/pages/getting_started/pages_from_scratch.md). If you're migrating from another CI/CD tool, check out our handy references: @@ -126,9 +126,10 @@ Its feature set is listed on the table below according to DevOps stages. | [ChatOps](chatops/README.md) | Trigger CI jobs from chat, with results sent back to the channel. | |---+---| | **Verify** || -| [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the performance impact of pending code changes. | +| [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | +| [Load Performance Testing](../user/project/merge_requests/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | | [CI services](services/README.md) | Link Docker containers with your base image.| -| [Code Quality](../user/project/merge_requests/code_quality.md) **(STARTER)** | Analyze your source code quality. | +| [Code Quality](../user/project/merge_requests/code_quality.md) | Analyze your source code quality. | | [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) **(PREMIUM)** | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | | [Interactive Web Terminals](interactive_web_terminal/index.md) **(CORE ONLY)** | Open an interactive web terminal to debug the running jobs. | | [JUnit tests](junit_test_reports.md) | Identify script failures directly on merge requests. | @@ -139,7 +140,7 @@ Its feature set is listed on the table below according to DevOps stages. | [Building Docker images](docker/using_docker_build.md) | Maintain Docker-based projects using GitLab CI/CD. | | [Canary Deployments](../user/project/canary_deployments.md) **(PREMIUM)** | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | | [Deploy Boards](../user/project/deploy_boards.md) **(PREMIUM)** | Check the current health and status of each CI/CD environment running on Kubernetes. | -| [Feature Flags](../user/project/operations/feature_flags.md) **(PREMIUM)** | Deploy your features behind Feature Flags. | +| [Feature Flags](../operations/feature_flags.md) **(PREMIUM)** | Deploy your features behind Feature Flags. | | [GitLab Pages](../user/project/pages/index.md) | Deploy static websites. | | [GitLab Releases](../user/project/releases/index.md) | Add release notes to Git tags. | | [Review Apps](review_apps/index.md) | Configure GitLab CI/CD to preview code changes. | diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 392dd4fdeba..b6bd01ecf58 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -226,14 +226,14 @@ image: node:latest cache: key: ${CI_COMMIT_REF_SLUG} paths: - - .npm/ + - .npm/ before_script: - npm ci --cache .npm --prefer-offline test_async: script: - - node ./specs/start.js ./specs/async.spec.js + - node ./specs/start.js ./specs/async.spec.js ``` ### Caching PHP dependencies @@ -253,16 +253,16 @@ image: php:7.2 cache: key: ${CI_COMMIT_REF_SLUG} paths: - - vendor/ + - vendor/ before_script: -# Install and run Composer -- curl --show-error --silent https://getcomposer.org/installer | php -- php composer.phar install + # Install and run Composer + - curl --show-error --silent https://getcomposer.org/installer | php + - php composer.phar install test: script: - - vendor/bin/phpunit --configuration phpunit.xml --coverage-text --colors=never + - vendor/bin/phpunit --configuration phpunit.xml --coverage-text --colors=never ``` ### Caching Python dependencies @@ -301,9 +301,9 @@ before_script: test: script: - - python setup.py test - - pip install flake8 - - flake8 . + - python setup.py test + - pip install flake8 + - flake8 . ``` ### Caching Ruby dependencies @@ -330,7 +330,7 @@ before_script: rspec: script: - - rspec spec + - rspec spec ``` ### Caching Go dependencies @@ -354,7 +354,7 @@ test: image: golang:1.13 extends: .go-cache script: - - go test ./... -v -short + - go test ./... -v -short ``` ## Availability of the cache @@ -391,28 +391,28 @@ stages: ```yaml stages: -- build -- test + - build + - test before_script: -- echo "Hello" + - echo "Hello" job A: stage: build script: - - mkdir vendor/ - - echo "build" > vendor/hello.txt + - mkdir vendor/ + - echo "build" > vendor/hello.txt cache: key: build-cache paths: - - vendor/ + - vendor/ after_script: - - echo "World" + - echo "World" job B: stage: test script: - - cat vendor/hello.txt + - cat vendor/hello.txt cache: key: build-cache ``` @@ -483,8 +483,8 @@ cache when the pipeline is run for a second time. ```yaml stages: -- build -- test + - build + - test job A: stage: build @@ -492,7 +492,7 @@ job A: cache: key: same-key paths: - - public/ + - public/ job B: stage: test @@ -500,7 +500,7 @@ job B: cache: key: same-key paths: - - vendor/ + - vendor/ ``` 1. `job A` runs. @@ -520,8 +520,8 @@ will be different): ```yaml stages: -- build -- test + - build + - test job A: stage: build @@ -529,7 +529,7 @@ job A: cache: key: keyA paths: - - vendor/ + - vendor/ job B: stage: test @@ -537,7 +537,7 @@ job B: cache: key: keyB paths: - - vendor/ + - vendor/ ``` In that case, even if the `key` is different (no fear of overwriting), you diff --git a/doc/ci/chatops/README.md b/doc/ci/chatops/README.md index a8fea0836c1..ec0c41032ca 100644 --- a/doc/ci/chatops/README.md +++ b/doc/ci/chatops/README.md @@ -10,47 +10,75 @@ type: index, concepts, howto > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4466) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. > - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24780) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.9. -GitLab ChatOps provides a method to interact with CI/CD jobs through chat services like Slack. Many organizations' discussion, collaboration, and troubleshooting is taking place in chat services these days, and having a method to run CI/CD jobs with output posted back to the channel can significantly augment a team's workflow. +GitLab ChatOps provides a method to interact with CI/CD jobs through chat services +like Slack. Many organizations' discussion, collaboration, and troubleshooting takes +place in chat services. Having a method to run CI/CD jobs with output +posted back to the channel can significantly augment your team's workflow. -NOTE: **Note:** -ChatOps is currently in alpha with some important features missing, like access control. +## How GitLab ChatOps works -## How it works +GitLab ChatOps is built upon [GitLab CI/CD](../README.md) and +[Slack Slash Commands](../../user/project/integrations/slack_slash_commands.md). +ChatOps provides a `run` action for [slash commands](../../integration/slash_commands.md) +with the following arguments: -GitLab ChatOps is built upon two existing features: +- A `<job name>` to execute. +- The `<job arguments>`. -- [GitLab CI/CD](../README.md). -- [Slack Slash Commands](../../user/project/integrations/slack_slash_commands.md). +ChatOps passes the following [CI/CD variables](../variables/README.md#predefined-environment-variables) +to the job: -A new `run` action has been added to the [slash commands](../../integration/slash_commands.md), which takes two arguments: a `<job name>` to execute and the `<job arguments>`. When executed, ChatOps will look up the specified job name and attempt to match it to a corresponding job in [`.gitlab-ci.yml`](../yaml/README.md). If a matching job is found on `master`, a pipeline containing just that job is scheduled. Two additional [CI/CD variables](../variables/README.md#predefined-environment-variables) are passed to the job: `CHAT_INPUT` contains any additional arguments, and `CHAT_CHANNEL` is set to the name of channel the action was triggered in. +- `CHAT_INPUT` contains any additional arguments. +- `CHAT_CHANNEL` is set to the name of channel the action was triggered in. -After the job has finished, its output is sent back to Slack provided it has completed within 30 minutes. If a job takes more than 30 minutes to run it must use the Slack API to manually send data back to a channel. +When executed, ChatOps looks up the specified job name and attempts to match it +to a corresponding job in [`.gitlab-ci.yml`](../yaml/README.md). If a matching job +is found on `master`, a pipeline containing only that job is scheduled. After the +job completes: -[Developer access and above](../../user/permissions.md#project-members-permissions) is required to use the `run` command. If a job should not be able to be triggered from chat, it can be set to `except: [chat]`. +- If the job completes in *less than 30 minutes*, the ChatOps sends the job's output to Slack. +- If the job completes in *more than 30 minutes*, the job must use the + [Slack API](https://api.slack.com/) to send data to the channel. -## Creating a ChatOps CI job +To use the `run` command, you must have +[Developer access or above](../../user/permissions.md#project-members-permissions). +If a job shouldn't be able to be triggered from chat, you can set the job to `except: [chat]`. -Since ChatOps is built upon GitLab CI/CD, the job has all the same features and functions available. There a few best practices to consider however when creating ChatOps jobs: +## Best practices for ChatOps CI jobs -- It is strongly recommended to set `only: [chat]` so the job does not run as part of the standard CI pipeline. -- If the job is set to `when: manual`, the pipeline will be created however the job will wait to be started. -- It is important to keep in mind that there is limited support for access control. If the user who triggered the slash command is a developer in the project, the job will run. The job itself can utilize existing [CI/CD variables](../variables/README.md#predefined-environment-variables) like `GITLAB_USER_ID` to perform additional rights validation, however these variables can be [overridden](../variables/README.md#priority-of-environment-variables). +Since ChatOps is built upon GitLab CI/CD, the job has all the same features and +functions available. Consider these best practices when creating ChatOps jobs: + +- GitLab strongly recommends you set `only: [chat]` so the job does not run as part + of the standard CI pipeline. +- If the job is set to `when: manual`, ChatOps creates the pipeline, but the job waits to be started. +- ChatOps provides limited support for access control. If the user triggering the + slash command has [Developer access or above](../../user/permissions.md#project-members-permissions) + in the project, the job runs. The job itself can use existing + [CI/CD variables](../variables/README.md#predefined-environment-variables) like + `GITLAB_USER_ID` to perform additional rights validation, but + these variables can be [overridden](../variables/README.md#priority-of-environment-variables). ### Controlling the ChatOps reply -For jobs with a single command, its output is automatically sent back to the channel as a reply. For example the chat reply of the following job is simply `Hello World.` +The output for jobs with a single command is sent to the channel as a reply. For +example, the chat reply of the following job is `Hello World` in the channel: ```yaml hello-world: stage: chatops only: [chat] script: - - echo "Hello World." + - echo "Hello World" ``` -Jobs that contain multiple commands, or have a `before_script`, include additional content in the chat reply. In these cases both the commands and their output are included, with the commands wrapped in ANSI colors codes. +Jobs that contain multiple commands (or `before_script`) return additional +content in the chat reply. In these cases, both the commands and their output are +included, with the commands wrapped in ANSI color codes. -To selectively reply with the output of one command, its output must be bounded by the `chat_reply` section. For example, the following job will list the files in the current directory. +To selectively reply with the output of one command, its output must be bounded by +the `chat_reply` section. For example, the following job lists the files in the +current directory: ```yaml ls: @@ -61,20 +89,20 @@ ls: - echo -e "section_start:$( date +%s ):chat_reply\r\033[0K\n$( ls -la )\nsection_end:$( date +%s ):chat_reply\r\033[0K" ``` -## GitLab ChatOps Examples +## GitLab ChatOps examples -The GitLab.com team created a repository of [common ChatOps scripts they use to interact with our Production instance of GitLab](https://gitlab.com/gitlab-com/chatops). They are likely useful -to other administrators of GitLab instances and can serve as inspiration for ChatOps scripts you can write to interact with your own applications. +The GitLab.com team created a repository of [common ChatOps scripts](https://gitlab.com/gitlab-com/chatops) +they use to interact with our Production instance of GitLab. Administrators of +other GitLab instances may find them useful. They can serve as inspiration for ChatOps +scripts you can write to interact with your own applications. ## GitLab ChatOps icon -Say Hi to our ChatOps bot. +The [official GitLab ChatOps icon](img/gitlab-chatops-icon.png) is available for download. You can find and download the official GitLab ChatOps icon here.  -[Download bigger image](img/gitlab-chatops-icon.png) - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 26eb99bd55a..29ce8bdf625 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -15,7 +15,7 @@ cloud provider more easily. ## AWS -GitLab provides Docker images to simplify working with AWS, and a template to make +GitLab provides Docker images that you can use to [run AWS commands from GitLab CI/CD](#run-aws-commands-from-gitlab-cicd), and a template to make it easier to [deploy to AWS](#deploy-your-application-to-the-aws-elastic-container-service-ecs). ### Run AWS commands from GitLab CI/CD @@ -87,6 +87,11 @@ GitLab provides a series of [CI templates that you can include in your project]( To automate deployments of your application to your [Amazon Elastic Container Service](https://aws.amazon.com/ecs/) (AWS ECS) cluster, you can `include` the `Deploy-ECS.gitlab-ci.yml` template in your `.gitlab-ci.yml` file. +GitLab also provides [Docker images](https://gitlab.com/gitlab-org/cloud-deploy/-/tree/master/aws) that can be used in your `gitlab-ci.yml` file to simplify working with AWS: + +- Use `registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest` to use AWS CLI commands. +- Use `registry.gitlab.com/gitlab-org/cloud-deploy/aws-ecs:latest` to deploy your application to AWS ECS. + Before getting started with this process, you need a cluster on AWS ECS, as well as related components, like an ECS service, ECS task definition, a database on AWS RDS, etc. [Read more about AWS ECS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html). diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index fff0fda0ab4..b7dd74a0230 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -82,10 +82,10 @@ are certain use cases that you may need to work around. For more information: ## DAG Visualization > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](https://about.gitlab.com/handbook/product/#beta). -> - It's deployed behind a feature flag, disabled by default. +> - It was deployed behind a feature flag, disabled by default. +> - It became [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36802) in 13.2. > - It's enabled on GitLab.com. -> - It's not recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [enable it](#enable-or-disable-dag-visualization-core-only) +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-dag-visualization-core-only). The DAG visualization makes it easier to visualize the relationships between dependent jobs in a DAG. This graph will display all the jobs in a pipeline that need or are needed by other jobs. Jobs with no relationships are not displayed in this view. @@ -97,15 +97,15 @@ Clicking a node will highlight all the job paths it depends on. ### Enable or disable DAG Visualization **(CORE ONLY)** -DAG Visualization is under development and requires more testing, but is being made available as a beta features so users can check its limitations and uses. +DAG Visualization is under development, but is being made available as a beta feature so users can check its limitations and uses. -It is deployed behind a feature flag that is **disabled by default**. +It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to enable it for your instance: +can opt to disable it for your instance: ```ruby # Instance-wide -Feature.enable(:dag_pipeline_tab) +Feature.disable(:dag_pipeline_tab) # or by project -Feature.enable(:dag_pipeline_tab, Project.find(<project id>)) +Feature.disable(:dag_pipeline_tab, Project.find(<project id>)) ``` diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 65b9c03186b..4bed6d9e323 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -123,7 +123,7 @@ not without its own challenges: - By default, Docker 17.09 and higher uses `--storage-driver overlay2` which is the recommended storage driver. See [Using the overlayfs driver](#use-the-overlayfs-driver) for details. -- Since the `docker:19.03.11-dind` container and the Runner container don't share their +- Since the `docker:19.03.12-dind` container and the Runner container don't share their root filesystem, the job's working directory can be used as a mount point for child containers. For example, if you have files you want to share with a child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH` @@ -142,14 +142,14 @@ not without its own challenges: An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. In the examples below, we are using Docker images tags to specify a -specific version, such as `docker:19.03.11`. If tags like `docker:stable` +specific version, such as `docker:19.03.12`. If tags like `docker:stable` are used, you have no control over what version is going to be used and this can lead to unpredictable behavior, especially when new versions are released. #### TLS enabled -NOTE: **Note** +NOTE: **Note:** Requires GitLab Runner 11.11 or later, but is not supported if GitLab Runner is installed using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). See the @@ -158,7 +158,7 @@ issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/83) for details. The Docker daemon supports connection over TLS and it's done by default -for Docker 19.03.11 or higher. This is the **suggested** way to use the +for Docker 19.03.12 or higher. This is the **suggested** way to use the Docker-in-Docker service and [GitLab.com Shared Runners](../../user/gitlab_com/index.md#shared-runners) support this. @@ -174,13 +174,13 @@ support this. --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:19.03.11" \ + --docker-image "docker:19.03.12" \ --docker-privileged \ --docker-volumes "/certs/client" ``` The above command will register a new Runner to use the special - `docker:19.03.11` image, which is provided by Docker. **Notice that it's + `docker:19.03.12` image, which is provided by Docker. **Notice that it's using the `privileged` mode to start the build and service containers.** If you want to use [Docker-in-Docker](https://www.docker.com/blog/docker-can-now-run-within-docker/) mode, you always have to use `privileged = true` in your Docker containers. @@ -199,7 +199,7 @@ support this. executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.11" + image = "docker:19.03.12" privileged = true disable_cache = false volumes = ["/certs/client", "/cache"] @@ -209,10 +209,10 @@ support this. ``` 1. You can now use `docker` in the build script (note the inclusion of the - `docker:19.03.11-dind` service): + `docker:19.03.12-dind` service): ```yaml - image: docker:19.03.11 + image: docker:19.03.12 variables: # When using dind service, we need to instruct docker, to talk with @@ -237,7 +237,7 @@ support this. DOCKER_TLS_CERTDIR: "/certs" services: - - docker:19.03.11-dind + - docker:19.03.12-dind before_script: - docker info @@ -264,7 +264,7 @@ Assuming that the Runner `config.toml` is similar to: executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.11" + image = "docker:19.03.12" privileged = true disable_cache = false volumes = ["/cache"] @@ -274,10 +274,10 @@ Assuming that the Runner `config.toml` is similar to: ``` You can now use `docker` in the build script (note the inclusion of the -`docker:19.03.11-dind` service): +`docker:19.03.12-dind` service): ```yaml -image: docker:19.03.11 +image: docker:19.03.12 variables: # When using dind service we need to instruct docker, to talk with the @@ -298,7 +298,7 @@ variables: DOCKER_TLS_CERTDIR: "" services: - - docker:19.03.11-dind + - docker:19.03.12-dind before_script: - docker info @@ -318,7 +318,7 @@ container so that Docker is available in the context of that image. NOTE: **Note:** If you bind the Docker socket [when using GitLab Runner 11.11 or newer](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -you can no longer use `docker:19.03.11-dind` as a service because volume bindings +you can no longer use `docker:19.03.12-dind` as a service because volume bindings are done to the services as well, making these incompatible. In order to do that, follow the steps: @@ -333,12 +333,12 @@ In order to do that, follow the steps: --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:19.03.11" \ + --docker-image "docker:19.03.12" \ --docker-volumes /var/run/docker.sock:/var/run/docker.sock ``` The above command will register a new Runner to use the special - `docker:19.03.11` image which is provided by Docker. **Notice that it's using + `docker:19.03.12` image which is provided by Docker. **Notice that it's using the Docker daemon of the Runner itself, and any containers spawned by Docker commands will be siblings of the Runner rather than children of the Runner.** This may have complications and limitations that are unsuitable for your workflow. @@ -352,7 +352,7 @@ In order to do that, follow the steps: executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.11" + image = "docker:19.03.12" privileged = false disable_cache = false volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] @@ -361,11 +361,11 @@ In order to do that, follow the steps: ``` 1. You can now use `docker` in the build script (note that you don't need to - include the `docker:19.03.11-dind` service as when using the Docker in Docker + include the `docker:19.03.12-dind` service as when using the Docker in Docker executor): ```yaml - image: docker:19.03.11 + image: docker:19.03.12 before_script: - docker info @@ -419,10 +419,10 @@ any image that's used with the `--cache-from` argument must first be pulled Here's a `.gitlab-ci.yml` file showing how Docker caching can be used: ```yaml -image: docker:19.03.11 +image: docker:19.03.12 services: - - docker:19.03.11-dind + - docker:19.03.12-dind variables: # Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 2448bb536ab..735cf35584f 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -149,14 +149,14 @@ the job will fail: ```yaml job: services: - - php:7 - - node:latest - - golang:1.10 + - php:7 + - node:latest + - golang:1.10 image: alpine:3.7 script: - - php -v - - node -v - - go version + - php -v + - node -v + - go version ``` If you need to have `php`, `node` and `go` available for your script, you should @@ -176,7 +176,7 @@ You can then use for example the [tutum/wordpress](https://hub.docker.com/r/tutu ```yaml services: -- tutum/wordpress:latest + - tutum/wordpress:latest ``` If you don't [specify a service alias](#available-settings-for-services), @@ -219,7 +219,7 @@ default: test: script: - - bundle exec rake spec + - bundle exec rake spec ``` The image name must be in one of the following formats: @@ -238,16 +238,16 @@ default: test:2.6: image: ruby:2.6 services: - - postgres:11.7 + - postgres:11.7 script: - - bundle exec rake spec + - bundle exec rake spec test:2.7: image: ruby:2.7 services: - - postgres:12.2 + - postgres:12.2 script: - - bundle exec rake spec + - bundle exec rake spec ``` Or you can pass some [extended configuration options](#extended-docker-configuration-options) @@ -260,17 +260,17 @@ default: entrypoint: ["/bin/bash"] services: - - name: my-postgres:11.7 - alias: db-postgres - entrypoint: ["/usr/local/bin/db-postgres"] - command: ["start"] + - name: my-postgres:11.7 + alias: db-postgres + entrypoint: ["/usr/local/bin/db-postgres"] + command: ["start"] before_script: - - bundle install + - bundle install test: script: - - bundle exec rake spec + - bundle exec rake spec ``` ## Passing environment variables to services @@ -292,21 +292,21 @@ variables: POSTGRES_INITDB_ARGS: "--encoding=UTF8 --data-checksums" services: -- name: postgres:11.7 - alias: db - entrypoint: ["docker-entrypoint.sh"] - command: ["postgres"] + - name: postgres:11.7 + alias: db + entrypoint: ["docker-entrypoint.sh"] + command: ["postgres"] image: name: ruby:2.6 entrypoint: ["/bin/bash"] before_script: -- bundle install + - bundle install test: script: - - bundle exec rake spec + - bundle exec rake spec ``` ## Extended Docker configuration options @@ -330,8 +330,8 @@ For example, the following two definitions are equal: image: "registry.example.com/my/image:latest" services: - - postgresql:9.4 - - redis:latest + - postgresql:9.4 + - redis:latest ``` 1. Using a map as an option to `image` and `services`. The use of `image:name` is @@ -342,8 +342,8 @@ For example, the following two definitions are equal: name: "registry.example.com/my/image:latest" services: - - name: postgresql:9.4 - - name: redis:latest + - name: postgresql:9.4 + - name: redis:latest ``` ### Available settings for `image` @@ -378,8 +378,8 @@ would not work properly: ```yaml services: -- mysql:latest -- mysql:latest + - mysql:latest + - mysql:latest ``` The Runner would start two containers using the `mysql:latest` image, but both @@ -392,10 +392,10 @@ look like: ```yaml services: -- name: mysql:latest - alias: mysql-1 -- name: mysql:latest - alias: mysql-2 + - name: mysql:latest + alias: mysql-1 + - name: mysql:latest + alias: mysql-2 ``` The Runner will still start two containers using the `mysql:latest` image, @@ -427,7 +427,7 @@ CMD ["/usr/bin/super-sql", "run"] # .gitlab-ci.yml services: -- my-super-sql:latest + - my-super-sql:latest ``` After the new extended Docker configuration options, you can now simply @@ -437,8 +437,8 @@ set a `command` in `.gitlab-ci.yml`, like: # .gitlab-ci.yml services: -- name: super/sql:latest - command: ["/usr/bin/super-sql", "run"] + - name: super/sql:latest + command: ["/usr/bin/super-sql", "run"] ``` As you can see, the syntax of `command` is similar to [Dockerfile's `CMD`](https://docs.docker.com/engine/reference/builder/#cmd). @@ -545,9 +545,8 @@ runtime. support for using private registries, which required manual configuration of credentials on runner's host. We recommend to upgrade your Runner to at least version **1.8** if you want to use private registries. -- Not available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html), - follow <https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2673> for - details. +- Available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html) + in GitLab Runner 13.1 and later. ### Using statically-defined credentials @@ -601,6 +600,7 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: Open a terminal and execute the following command: ```shell + # Note the use of "-n" - it prevents encoding a newline in the password. echo -n "my_username:my_password" | base64 # Example output to copy @@ -681,11 +681,13 @@ To add `DOCKER_AUTH_CONFIG` to a Runner: 1. Restart the Runner service. -NOTE: **Note:** The double quotes included in the `DOCKER_AUTH_CONFIG` +NOTE: **Note:** +The double quotes included in the `DOCKER_AUTH_CONFIG` data must be escaped with backslashes. This prevents them from being interpreted as TOML. -NOTE: **Note:** The `environment` option is a list. So your Runner may +NOTE: **Note:** +The `environment` option is a list. So your Runner may have existing entries and you should add this to the list, not replace it. @@ -715,7 +717,8 @@ To configure credentials store, follow these steps: `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this configuration file and will use the needed helper for this specific repository. -NOTE: **Note:** `credsStore` is used to access ALL the registries. +NOTE: **Note:** +`credsStore` is used to access ALL the registries. If you will want to use both images from private registry and public images from DockerHub, pulling from DockerHub will fail, because Docker daemon will try to use the same credentials for **ALL** the registries. diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index d53430400ec..1580080ac6e 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -90,7 +90,7 @@ store: - | echo "-----BEGIN CERTIFICATE----- ... - -----END CERTIFICATE-----" >> /kaniko/ssl/certs/ca-certificates.crt + -----END CERTIFICATE-----" >> /kaniko/ssl/certs/additional-ca-cert-bundle.crt ``` ## Video walkthrough of a working example diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 055baa78743..a4ff164a5ba 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -7,7 +7,7 @@ that help maintain deployment security and stability. You can: - [Restrict write-access to a critical environment](#restrict-write-access-to-a-critical-environment) -- [Restrict deployments for a particular period](#restrict-deployments-for-a-particular-period) +- [Prevent deployments during deploy freeze windows](#prevent-deployments-during-deploy-freeze-windows) If you are using a continuous deployment workflow and want to ensure that concurrent deployments to the same environment do not happen, you should enable the following options: @@ -77,10 +77,10 @@ The improved pipeline flow **after** enabling Skip outdated deployment jobs: 1. The `deploy` job in Pipeline-B finishes first, and deploys the newer code. 1. The `deploy` job in Pipeline-A is automatically cancelled, so that it doesn't overwrite the deployment from the newer pipeline. -## Restrict deployments for a particular period +## Prevent deployments during deploy freeze windows If you want to prevent deployments for a particular period, for example during a planned -vacation period when most employees are out, you can set up a [Deploy Freeze](../../user/project/releases/index.md#set-a-deploy-freeze). +vacation period when most employees are out, you can set up a [Deploy Freeze](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). During a deploy freeze period, no deployment can be executed. This is helpful to ensure that deployments do not happen unexpectedly. diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 84480b836f8..0273ab290a6 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -94,7 +94,7 @@ deploy_staging: name: staging url: https://staging.example.com only: - - master + - master ``` We have defined three [stages](../yaml/README.md#stages): @@ -124,7 +124,7 @@ The environment `name` and `url` is exposed in various places within GitLab. Each time a job that has an environment specified succeeds, a deployment is recorded, along with the Git SHA, and environment name. -CAUTION: **Caution**: +CAUTION: **Caution:** Some characters are not allowed in environment names. Use only letters, numbers, spaces, and `-`, `_`, `/`, `{`, `}`, or `.`. Also, it must not start nor end with `/`. @@ -173,8 +173,8 @@ If you want to use the environment URL in GitLab, you would have to update it ma To address this problem, you can configure a deployment job to report back a set of variables, including the URL that was dynamically-generated by the external service. -GitLab supports [dotenv](https://github.com/bkeepers/dotenv) file as the format, -and expands the `environment:url` value with variables defined in the dotenv file. +GitLab supports the [dotenv (`.env`)](https://github.com/bkeepers/dotenv) file format, +and expands the `environment:url` value with variables defined in the `.env` file. To use this feature, specify the [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) keyword in `.gitlab-ci.yml`. @@ -213,14 +213,13 @@ stop_review: As soon as the `review` job finishes, GitLab updates the `review/your-branch-name` environment's URL. -It parses the report artifact `deploy.env`, registers a list of variables as runtime-created, +It parses the `deploy.env` report artifact, registers a list of variables as runtime-created, uses it for expanding `environment:url: $DYNAMIC_ENVIRONMENT_URL` and sets it to the environment URL. You can also specify a static part of the URL at `environment:url:`, such as `https://$DYNAMIC_ENVIRONMENT_URL`. If the value of `DYNAMIC_ENVIRONMENT_URL` is -`123.awesome.com`, the final result will be `https://123.awesome.com`. +`example.com`, the final result will be `https://example.com`. -The assigned URL for the `review/your-branch-name` environment is visible in the UI. -[See where the environment URL is displayed](#using-the-environment-url). +The assigned URL for the `review/your-branch-name` environment is [visible in the UI](#using-the-environment-url). > **Notes:** > @@ -260,7 +259,7 @@ deploy_staging: name: staging url: https://staging.example.com only: - - master + - master deploy_prod: stage: deploy @@ -271,7 +270,7 @@ deploy_prod: url: https://example.com when: manual only: - - master + - master ``` The `when: manual` action: @@ -306,11 +305,6 @@ declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments are a fundamental part of [Review apps](../review_apps/index.md). -### Configuring incremental rollouts - -Learn how to release production changes to only a portion of your Kubernetes pods with -[incremental rollouts](../environments/incremental_rollouts.md). - #### Allowed variables The `name` and `url` parameters for dynamic environments can use most available CI/CD variables, @@ -408,7 +402,7 @@ deploy: kubernetes: namespace: production only: - - master + - master ``` When deploying to a Kubernetes cluster using GitLab's Kubernetes integration, @@ -423,6 +417,11 @@ that are [managed by GitLab](../../user/project/clusters/index.md#gitlab-managed To follow progress on support for GitLab-managed clusters, see the [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/38054). +#### Configuring incremental rollouts + +Learn how to release production changes to only a portion of your Kubernetes pods with +[incremental rollouts](../environments/incremental_rollouts.md). + ### Deployment safety Deployment jobs can be more sensitive than other jobs in a pipeline, @@ -432,7 +431,7 @@ in GitLab that helps maintain deployment security and stability. - [Restrict write-access to a critical environment](deployment_safety.md#restrict-write-access-to-a-critical-environment) - [Limit the job-concurrency for deployment jobs](deployment_safety.md#ensure-only-one-deployment-job-runs-at-a-time) - [Skip outdated deployment jobs](deployment_safety.md#skip-outdated-deployment-jobs) -- [Restrict deployments for a particular period](deployment_safety.md#restrict-deployments-for-a-particular-period) +- [Prevent deployments during deploy freeze windows](deployment_safety.md#prevent-deployments-during-deploy-freeze-windows) ### Complete example @@ -484,7 +483,7 @@ deploy_staging: name: staging url: https://staging.example.com only: - - master + - master deploy_prod: stage: deploy @@ -495,7 +494,7 @@ deploy_prod: url: https://example.com when: manual only: - - master + - master ``` A more realistic example would also include copying files to a location where a @@ -740,6 +739,12 @@ To enable this feature, you need to specify the [`environment:auto_stop_in`](../ You can specify a human-friendly date as the value, such as `1 hour and 30 minutes` or `1 day`. `auto_stop_in` uses the same format of [`artifacts:expire_in` docs](../yaml/README.md#artifactsexpire_in). +NOTE: **Note:** +Due to the resource limitation, a background worker for stopping environments only +runs once every hour. This means environments will not be stopped at the exact +timestamp as the specified period, but will be stopped when the hourly cron worker +detects expired environments. + ##### Auto-stop example In the following example, there is a basic review app setup that creates a new environment @@ -779,15 +784,9 @@ and the environment will be active until it's stopped manually.  -NOTE: **NOTE** -Due to the resource limitation, a background worker for stopping environments only -runs once every hour. This means environments will not be stopped at the exact -timestamp as the specified period, but will be stopped when the hourly cron worker -detects expired environments. - #### Delete a stopped environment -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/22629) in GitLab 12.9. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20620) in GitLab 12.10. You can delete [stopped environments](#stopping-an-environment) in one of two ways: through the GitLab UI or through the API. @@ -812,6 +811,31 @@ stopped environment: Environments can also be deleted by using the [Environments API](../../api/environments.md#delete-an-environment). +### Prepare an environment + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208655) in GitLab 13.2. + +By default, GitLab creates a [deployment](#viewing-deployment-history) every time a +build with the specified environment runs. Newer deployments can also +[cancel older ones](deployment_safety.md#skip-outdated-deployment-jobs). + +You may want to specify an environment keyword to +[protect builds from unauthorized access](protected_environments.md), or to get +access to [scoped variables](#scoping-environments-with-specs). In these cases, +you can use the `action: prepare` keyword to ensure deployments won't be created, +and no builds would be canceled: + +```yaml +build: + stage: build + script: + - echo "Building the app" + environment: + name: staging + action: prepare + url: https://staging.example.com +``` + ### Grouping similar environments > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7015) in GitLab 8.14. @@ -868,7 +892,7 @@ versions of the app, all without leaving GitLab. #### Embedding metrics in GitLab Flavored Markdown -Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab Flavored Markdown](../../user/project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) for more details. +Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab Flavored Markdown](../../operations/metrics/embed.md) for more details. ### Web terminals diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 3a44fcfb182..b6141ddc7ac 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -22,7 +22,7 @@ specific environments are "protected" to prevent unauthorized people from affect By default, a protected environment does one thing: it ensures that only people with the right privileges can deploy to it, thus keeping it safe. -NOTE: **Note**: +NOTE: **Note:** A GitLab admin is always allowed to use environments, even if they are protected. To protect, update, or unprotect an environment, you need to have at least diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index c54e4b96332..37ccef10567 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -25,6 +25,7 @@ The following table lists examples with step-by-step tutorials that are containe | Use case | Resource | |:------------------------------|:---------------------------------------------------------------------------------------------------------------------------| | Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | +| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | | Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | | Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | | Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | @@ -46,8 +47,6 @@ The following table lists examples with step-by-step tutorials that are containe Contributions are welcome! You can help your favorite programming language users and GitLab by sending a merge request with a guide for that language. -You may want to apply for the [GitLab Community Writers Program](https://about.gitlab.com/community/writers/) -to get paid for writing complete articles for GitLab. ## CI/CD templates @@ -63,6 +62,7 @@ choose one of these templates: - [C++ (`C++.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/C++.gitlab-ci.yml) - [Chef (`Chef.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Chef.gitlab-ci.yml) - [Clojure (`Clojure.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Clojure.gitlab-ci.yml) +- [Composer `Composer.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Composer.gitlab-ci.yml) - [Crystal (`Crystal.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Crystal.gitlab-ci.yml) - [Django (`Django.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Django.gitlab-ci.yml) - [Docker (`Docker.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Docker.gitlab-ci.yml) @@ -78,6 +78,7 @@ choose one of these templates: - [LaTeX (`LaTeX.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/LaTeX.gitlab-ci.yml) - [Maven (`Maven.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Maven.gitlab-ci.yml) - [Mono (`Mono.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Mono.gitlab-ci.yml) +- [NPM (`npm.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/npm.gitlab-ci.yml) - [Node.js (`Nodejs.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Nodejs.gitlab-ci.yml) - [OpenShift (`OpenShift.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/OpenShift.gitlab-ci.yml) - [Packer (`Packer.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Packer.gitlab-ci.yml) diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index 3b8be54ae59..9a87786ec70 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -50,7 +50,7 @@ The JWT's payload looks like this: } ``` -The JWT is encoded by using RS256 and signed with your GitLab instance's OpenID Connect private key. The expire time for the token will be set to job's timeout, if specifed, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job will generate new JWT using the current signing key. +The JWT is encoded by using RS256 and signed with your GitLab instance's OpenID Connect private key. The expire time for the token will be set to job's timeout, if specified, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job will generate new JWT using the current signing key. You can use this JWT and your instance's JWKS endpoint (`https://gitlab.example.com/-/jwks`) to authenticate with a Vault server that is configured to allow the JWT Authentication method for authentication. @@ -60,7 +60,7 @@ To communicate with Vault, you can use either its CLI client or perform API requ ## Example -CAUTION: **Caution**: +CAUTION: **Caution:** JWTs are credentials, which can grant access to resources. Be careful where you paste them! Let's say you have the passwords for your staging and production databases stored in a Vault server that is running on `http://vault.example.com:8200`. Your staging password is `pa$$w0rd` and your production password is `real-pa$$w0rd`. @@ -156,7 +156,7 @@ Combined with GitLab's [protected branches](../../../user/project/protected_bran For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). -CAUTION: **Caution**: +CAUTION: **Caution:** Always restrict your roles to project or namespace by using one of the provided claims (e.g. `project_id` or `namespace_id`). Otherwise any JWT generated by this instance may be allowed to authenticate using this role. Now, configure the JWT Authentication method: diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md index 87cee1820bc..e0d4f3f2402 100644 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -61,10 +61,10 @@ content: ```yaml --- applications: -- name: gitlab-hello-world - random-route: true - memory: 1G - path: target/demo-0.0.1-SNAPSHOT.jar + - name: gitlab-hello-world + random-route: true + memory: 1G + path: target/demo-0.0.1-SNAPSHOT.jar ``` ## Configure GitLab CI/CD to deploy your application @@ -96,11 +96,11 @@ build: production: stage: deploy script: - - curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx - - ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io - - ./cf push + - curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx + - ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io + - ./cf push only: - - master + - master ``` We've used the `java:8` [Docker diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index ec02fb6dd43..3192b365c83 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -45,8 +45,8 @@ All possible parameters can be found here: <https://github.com/travis-ci/dpl#her staging: stage: deploy script: - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY ``` In the above example we use Dpl to deploy `my-app-staging` to Heroku server with API key stored in `HEROKU_STAGING_API_KEY` secure variable. @@ -64,12 +64,12 @@ You will have to install it: staging: stage: deploy script: - - apt-get update -yq - - apt-get install -y ruby-dev - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + - apt-get update -yq + - apt-get install -y ruby-dev + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY only: - - master + - master ``` The first line `apt-get update -yq` updates the list of available packages, @@ -89,18 +89,18 @@ The final `.gitlab-ci.yml` for that setup would look like this: staging: stage: deploy script: - - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + - gem install dpl + - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY only: - - master + - master production: stage: deploy script: - - gem install dpl - - dpl --provider=heroku --app=my-app-production --api-key=$HEROKU_PRODUCTION_API_KEY + - gem install dpl + - dpl --provider=heroku --app=my-app-production --api-key=$HEROKU_PRODUCTION_API_KEY only: - - tags + - tags ``` We created two deploy jobs that are executed on different events: diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index cea6f26181f..067d92c2275 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -150,7 +150,7 @@ before_script: stage_deploy: artifacts: paths: - - build/ + - build/ only: - dev script: diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 51d9f169939..35a35d97a4b 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -274,19 +274,19 @@ just pack them up in the cache. Here is the full `build` job: ```yaml build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules - artifacts: - paths: - - built + stage: build + script: + - npm i gulp -g + - npm i + - gulp + - gulp build-test + cache: + policy: push + paths: + - node_modules + artifacts: + paths: + - built ``` ### Test your game with GitLab CI/CD @@ -301,18 +301,18 @@ Following the YAML structure, the `test` job should look like this: ```yaml test: - stage: test - script: - - npm i gulp -g - - npm i - - gulp run-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ + stage: test + script: + - npm i gulp -g + - npm i + - gulp run-test + cache: + policy: push + paths: + - node_modules/ + artifacts: + paths: + - built/ ``` We have added unit tests for a `Weapon` class that shoots on a specified interval. @@ -325,33 +325,33 @@ Our entire `.gitlab-ci.yml` file should now look like this: image: node:10 build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ + stage: build + script: + - npm i gulp -g + - npm i + - gulp + - gulp build-test + cache: + policy: push + paths: + - node_modules/ + artifacts: + paths: + - built/ test: - stage: test - script: - - npm i gulp -g - - npm i - - gulp run-test - cache: - policy: pull - paths: - - node_modules/ - artifacts: - paths: - - built/ + stage: test + script: + - npm i gulp -g + - npm i + - gulp run-test + cache: + policy: pull + paths: + - node_modules/ + artifacts: + paths: + - built/ ``` ### Run your CI/CD pipeline @@ -445,18 +445,18 @@ trigger the `deploy` job of our pipeline. Put these together to get the followin ```yaml deploy: - stage: deploy - variables: - AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" - AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" - script: - - apt-get update - - apt-get install -y python3-dev python3-pip - - easy_install3 -U pip - - pip3 install --upgrade awscli - - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete - only: - - master + stage: deploy + variables: + AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" + AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" + script: + - apt-get update + - apt-get install -y python3-dev python3-pip + - easy_install3 -U pip + - pip3 install --upgrade awscli + - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete + only: + - master ``` Be sure to update the region and S3 URL in that last script command to fit your setup. @@ -466,46 +466,46 @@ Our final configuration file `.gitlab-ci.yml` looks like: image: node:10 build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ + stage: build + script: + - npm i gulp -g + - npm i + - gulp + - gulp build-test + cache: + policy: push + paths: + - node_modules/ + artifacts: + paths: + - built/ test: - stage: test - script: - - npm i gulp -g - - gulp run-test - cache: - policy: pull - paths: - - node_modules/ - artifacts: - paths: - - built/ + stage: test + script: + - npm i gulp -g + - gulp run-test + cache: + policy: pull + paths: + - node_modules/ + artifacts: + paths: + - built/ deploy: - stage: deploy - variables: - AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" - AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" - script: - - apt-get update - - apt-get install -y python3-dev python3-pip - - easy_install3 -U pip - - pip3 install --upgrade awscli - - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete - only: - - master + stage: deploy + variables: + AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" + AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" + script: + - apt-get update + - apt-get install -y python3-dev python3-pip + - easy_install3 -U pip + - pip3 install --upgrade awscli + - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete + only: + - master ``` ## Conclusion diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 5b442436baf..ee0938b4d4c 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -448,7 +448,7 @@ On your GitLab project repository navigate to the **Registry** tab.  -You may need to [enable Container Registry](../../../user/packages/container_registry/index.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Permissions**. +You may need to [enable Container Registry](../../../user/packages/container_registry/index.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Visibility, project features, permissions**. To start using Container Registry on our machine, we first need to login to the GitLab registry using our GitLab username and password: diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index e7768868c15..cc62e9316f2 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -76,7 +76,7 @@ environment, let's add it in `.gitlab-ci.yml`: ... before_script: -- bash ci/docker_install.sh > /dev/null + - bash ci/docker_install.sh > /dev/null ... ``` @@ -88,7 +88,7 @@ Last step, run the actual tests using `phpunit`: test:app: script: - - phpunit --configuration phpunit_myapp.xml + - phpunit --configuration phpunit_myapp.xml ... ``` @@ -104,11 +104,11 @@ image: php:5.6 before_script: # Install dependencies -- bash ci/docker_install.sh > /dev/null + - bash ci/docker_install.sh > /dev/null test:app: script: - - phpunit --configuration phpunit_myapp.xml + - phpunit --configuration phpunit_myapp.xml ``` ### Test against different PHP versions in Docker builds @@ -119,19 +119,19 @@ with a different Docker image version and the runner will do the rest: ```yaml before_script: # Install dependencies -- bash ci/docker_install.sh > /dev/null + - bash ci/docker_install.sh > /dev/null # We test PHP5.6 test:5.6: image: php:5.6 script: - - phpunit --configuration phpunit_myapp.xml + - phpunit --configuration phpunit_myapp.xml # We test PHP7.0 (good luck with that) test:7.0: image: php:7.0 script: - - phpunit --configuration phpunit_myapp.xml + - phpunit --configuration phpunit_myapp.xml ``` ### Custom PHP configuration in Docker builds @@ -142,7 +142,7 @@ add a `before_script` action: ```yaml before_script: -- cp my_php.ini /usr/local/etc/php/conf.d/test.ini + - cp my_php.ini /usr/local/etc/php/conf.d/test.ini ``` Of course, `my_php.ini` must be present in the root directory of your repository. @@ -166,7 +166,7 @@ Next, add the following snippet to your `.gitlab-ci.yml`: ```yaml test:app: script: - - phpunit --configuration phpunit_myapp.xml + - phpunit --configuration phpunit_myapp.xml ``` Finally, push to GitLab and let the tests begin! @@ -217,11 +217,11 @@ you can use [atoum](https://github.com/atoum/atoum): ```yaml before_script: -- wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar + - wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar test:atoum: script: - - php mageekguy.atoum.phar + - php mageekguy.atoum.phar ``` ### Using Composer @@ -238,16 +238,16 @@ following in your `.gitlab-ci.yml`: # your git repository. cache: paths: - - vendor/ + - vendor/ before_script: # Install composer dependencies -- wget https://composer.github.io/installer.sig -O - -q | tr -d '\n' > installer.sig -- php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" -- php -r "if (hash_file('SHA384', 'composer-setup.php') === file_get_contents('installer.sig')) { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" -- php composer-setup.php -- php -r "unlink('composer-setup.php'); unlink('installer.sig');" -- php composer.phar install + - wget https://composer.github.io/installer.sig -O - -q | tr -d '\n' > installer.sig + - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" + - php -r "if (hash_file('SHA384', 'composer-setup.php') === file_get_contents('installer.sig')) { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" + - php composer-setup.php + - php -r "unlink('composer-setup.php'); unlink('installer.sig');" + - php composer.phar install ... ``` diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 30a64e65607..d01e9663795 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -23,32 +23,32 @@ stages: test: stage: test script: - # this configures Django application to use attached postgres database that is run on `postgres` host - - export DATABASE_URL=postgres://postgres:@postgres:5432/python-test-app - - apt-get update -qy - - apt-get install -y python-dev python-pip - - pip install -r requirements.txt - - python manage.py test + # this configures Django application to use attached postgres database that is run on `postgres` host + - export DATABASE_URL=postgres://postgres:@postgres:5432/python-test-app + - apt-get update -qy + - apt-get install -y python-dev python-pip + - pip install -r requirements.txt + - python manage.py test staging: stage: deploy script: - - apt-get update -qy - - apt-get install -y ruby-dev - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-python-test-staging --api-key=$HEROKU_STAGING_API_KEY + - apt-get update -qy + - apt-get install -y ruby-dev + - gem install dpl + - dpl --provider=heroku --app=gitlab-ci-python-test-staging --api-key=$HEROKU_STAGING_API_KEY only: - - master + - master production: stage: deploy script: - - apt-get update -qy - - apt-get install -y ruby-dev - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-python-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY + - apt-get update -qy + - apt-get install -y ruby-dev + - gem install dpl + - dpl --provider=heroku --app=gitlab-ci-python-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY only: - - tags + - tags ``` This project has three jobs: diff --git a/doc/ci/img/metrics_reports_advanced_v13_0.png b/doc/ci/img/metrics_reports_advanced_v13_0.png Binary files differnew file mode 100644 index 00000000000..c092c6a84e4 --- /dev/null +++ b/doc/ci/img/metrics_reports_advanced_v13_0.png diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index be3741332e0..c79fb5bc926 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -38,10 +38,12 @@ but support [is planned](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/is ## Debugging a running job -NOTE: **Note:** Not all executors are +NOTE: **Note:** +Not all executors are [supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). -NOTE: **Note:** The `docker` executor does not keep running +NOTE: **Note:** +The `docker` executor does not keep running after the build script is finished. At that point, the terminal will automatically disconnect and will not wait for the user to finish. Please follow [this issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3605) for updates on @@ -67,6 +69,6 @@ close the terminal window.  -## Interactive Web Terminals for the Web IDE **(ULTIMATE ONLY)** +## Interactive Web Terminals for the Web IDE -Read the Web IDE docs to learn how to run [Interactive Terminals through the Web IDE](../../user/project/web_ide/index.md). +Read the Web IDE docs to learn how to run [Interactive Terminals through the Web IDE](../../user/project/web_ide/index.md#interactive-web-terminals-for-the-web-ide). diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index 7ba24f9c861..db18624d4e9 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -12,9 +12,9 @@ In this document, we'll present an overview of the concepts of Continuous Integr Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD. -NOTE: **Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more.** -Watch our -["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +NOTE: **Note:** +Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more. +Watch our ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. ## Introduction to CI/CD methodologies @@ -188,8 +188,9 @@ according to each stage (Verify, Package, Release). 1. **Verify**: - Automatically build and test your application with Continuous Integration. - - Analyze your source code quality with [GitLab Code Quality](../../user/project/merge_requests/code_quality.md). **(STARTER)** - - Determine the performance impact of code changes with [Browser Performance Testing](../../user/project/merge_requests/browser_performance_testing.md). **(PREMIUM)** + - Analyze your source code quality with [GitLab Code Quality](../../user/project/merge_requests/code_quality.md). + - Determine the browser performance impact of code changes with [Browser Performance Testing](../../user/project/merge_requests/browser_performance_testing.md). **(PREMIUM)** + - Determine the server performance impact of code changes with [Load Performance Testing](../../user/project/merge_requests/load_performance_testing.md). **(PREMIUM)** - Perform a series of tests, such as [Container Scanning](../../user/application_security/container_scanning/index.md) **(ULTIMATE)**, [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) **(ULTIMATE)**, and [JUnit tests](../junit_test_reports.md). - Deploy your changes with [Review Apps](../review_apps/index.md) to preview the app changes on every branch. 1. **Package**: @@ -202,7 +203,7 @@ according to each stage (Verify, Package, Release). - Continuous Delivery, manually click to deploy your app to production. - Deploy static websites with [GitLab Pages](../../user/project/pages/index.md). - Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature with [Canary Deployments](../../user/project/canary_deployments.md). **(PREMIUM)** - - Deploy your features behind [Feature Flags](../../user/project/operations/feature_flags.md). **(PREMIUM)** + - Deploy your features behind [Feature Flags](../../operations/feature_flags.md). **(PREMIUM)** - Add release notes to any Git tag with [GitLab Releases](../../user/project/releases/index.md). - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). **(PREMIUM)** - Deploy your application to a production environment in a Kubernetes cluster with [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy). @@ -226,7 +227,7 @@ To get started with GitLab CI/CD, you need to familiarize yourself with the [`.gitlab-ci.yml`](../yaml/README.md) configuration file syntax and with its attributes. -This document [introduces the concepts of GitLab CI/CD in the scope of GitLab Pages](../../user/project/pages/getting_started_part_four.md), for deploying static websites. +This document [introduces the concepts of GitLab CI/CD in the scope of GitLab Pages](../../user/project/pages/getting_started/pages_from_scratch.md), for deploying static websites. Although it's meant for users who want to write their own Pages script from scratch, it also serves as an introduction to the setup process for GitLab CI/CD. It covers the very first general steps of writing a CI/CD configuration diff --git a/doc/ci/jenkins/index.md b/doc/ci/jenkins/index.md index 37e0522e74d..1d029dcdd14 100644 --- a/doc/ci/jenkins/index.md +++ b/doc/ci/jenkins/index.md @@ -12,16 +12,25 @@ A lot of GitLab users have successfully migrated to GitLab CI/CD from Jenkins. T easier if you're just getting started, we've collected several resources here that you might find useful before diving in. Think of this page as a "GitLab CI/CD for Jenkins Users" guide. -First of all, our [Quick Start Guide](../quick_start/README.md) contains a good overview of how GitLab CI/CD works. -You may also be interested in [Auto DevOps](../../topics/autodevops/index.md) which can potentially be used to build, test, -and deploy your applications with little to no configuration needed at all. +The following list of recommended steps was created after observing organizations +that were able to quickly complete this migration: + +1. Start by reading the GitLab CI/CD [Quick Start Guide](../quick_start/README.md) and [important product differences](#important-product-differences). +1. Learn the importance of [managing the organizational transition](#managing-the-organizational-transition). +1. [Add Runners](../runners/README.md) to your GitLab instance. +1. Educate and enable your developers to independently perform the following steps in their projects: + 1. Review the [Quick Start Guide](../quick_start/README.md) and [Pipeline Configuration Reference](../yaml/README.md). + 1. Use the [Jenkins Wrapper](#jenkinsfile-wrapper) to temporarily maintain fragile Jenkins jobs. + 1. Migrate the build and CI jobs and configure them to show results directly in your merge requests. They can use [Auto DevOps](../../topics/autodevops/index.md) as a starting point, and [customize](../../topics/autodevops/customize.md) or [decompose](../../topics/autodevops/customize.md#using-components-of-auto-devops) the configuration as needed. + 1. Add [Review Apps](../review_apps/index.md). + 1. Migrate the deployment jobs using [cloud deployment templates](../cloud_deployment/index.md), adding [environments](../environments/index.md), and [deploy boards](../..//user/project/deploy_boards.md). + 1. Work to unwrap any jobs still running with the use of the Jenkins wrapper. +1. Take stock of any common CI/CD job definitions then create and share [templates](#templates) for them. For an example of how to convert a Jenkins pipeline into a GitLab CI/CD pipeline, or how to use Auto DevOps to test your code automatically, watch the [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF5Y) video. -For advanced CI/CD teams, [templates](#templates) can enable the reuse of pipeline configurations. - Otherwise, read on for important information that will help you get the ball rolling. Welcome to GitLab! @@ -53,8 +62,7 @@ We are building a [JenkinsFile Wrapper](https://gitlab.com/gitlab-org/jfr-contai you to run a complete Jenkins instance inside of a GitLab job, including plugins. This can help ease the process of transition, by letting you delay the migration of less urgent pipelines for a period of time. -If you are interested, join our [public testing issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215675) to -If you are interested, you might be able to [help GitLab test the wrapper](https://gitlab.com/gitlab-org/gitlab/-/issues/215675). +If you are interested in helping GitLab test the wrapper, join our [public testing issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215675) for instructions and to provide your feedback. ## Important product differences @@ -173,8 +181,8 @@ pdf: script: xelatex mycv.tex artifacts: paths: - - ./mycv.pdf - - ./output/ + - ./mycv.pdf + - ./output/ expire_in: 1 week ``` diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index aa0d40a4d06..aedda3c8341 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -73,6 +73,7 @@ For a list of supported languages on JUnit tests, check the To enable the JUnit reports in merge requests, you need to add [`artifacts:reports:junit`](pipelines/job_artifacts.md#artifactsreportsjunit) in `.gitlab-ci.yml`, and specify the path(s) of the generated test reports. +The reports must be `.xml` files, otherwise [GitLab returns an Error 500](https://gitlab.com/gitlab-org/gitlab/-/issues/216575). In the following examples, the job in the `test` stage runs and GitLab collects the JUnit test report from each job. After each job is executed, the @@ -103,7 +104,8 @@ ruby: ### Go example -Use the following job in `.gitlab-ci.yml`: +Use the following job in `.gitlab-ci.yml`, and ensure you use `-set-exit-code`, +otherwise the pipeline will be marked successful, even if the tests fail: ```yaml ## Use https://github.com/jstemmer/go-junit-report to generate a JUnit report with go @@ -111,7 +113,7 @@ golang: stage: test script: - go get -u github.com/jstemmer/go-junit-report - - go test -v 2>&1 | go-junit-report > report.xml + - go test -v 2>&1 | go-junit-report -set-exit-code > report.xml artifacts: reports: junit: report.xml @@ -206,7 +208,7 @@ cunit: junit: ./my-cunit-test.xml ``` -### .Net example +### .NET example The [JunitXML.TestLogger](https://www.nuget.org/packages/JunitXml.TestLogger/) NuGet package can generate test reports for .Net Framework and .Net Core applications. The following @@ -234,7 +236,9 @@ Test: ## Viewing JUnit test reports on GitLab -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24792) in GitLab 12.5. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24792) in GitLab 12.5. +> - It's deployed behind a feature flag, disabled by default. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enabling-the-junit-test-reports-feature-core-only). **(CORE ONLY)** If JUnit XML files are generated and uploaded as part of a pipeline, these reports can be viewed inside the pipelines details page. The **Tests** tab on this page will @@ -248,7 +252,7 @@ with failed showing at the top, skipped next and successful cases last. You can also retrieve the reports via the [GitLab API](../api/pipelines.md#get-a-pipelines-test-report). -### Enabling the feature +### Enabling the JUnit test reports feature **(CORE ONLY)** This feature comes with the `:junit_pipeline_view` feature flag disabled by default. This feature is disabled due to some performance issues with very large data sets. @@ -260,13 +264,15 @@ following command: ```ruby Feature.enable(:junit_pipeline_view) -# Enable the feature for a specific project +# Enable the feature for a specific project, GitLab 13.0 and above only. Feature.enable(:junit_pipeline_view, Project.find(<your-project-id-here>)) ``` ## Viewing JUnit screenshots on GitLab -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202114) in GitLab 13.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202114) in GitLab 13.0. +> - It's deployed behind a feature flag, disabled by default. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enabling-the-junit-screenshots-feature-core-only). **(CORE ONLY)** If JUnit XML files contain an `attachment` tag, GitLab parses the attachment. @@ -280,7 +286,7 @@ Upload your screenshots as [artifacts](pipelines/job_artifacts.md#artifactsrepor When [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/6061) is complete, the attached file will be visible on the pipeline details page. -### Enabling the feature +### Enabling the JUnit screenshots feature **(CORE ONLY)** This feature comes with the `:junit_pipeline_screenshots_view` feature flag disabled by default. diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index 7a4ca989fb6..2a6008e6307 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -24,7 +24,7 @@ can run a pipeline for merge requests.  -NOTE: **Note**: +NOTE: **Note:** If you use this feature with [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md), pipelines for merge requests take precedence over the other regular pipelines. @@ -32,7 +32,6 @@ pipelines for merge requests take precedence over the other regular pipelines. To enable pipelines for merge requests: -- You must have maintainer [permissions](../../user/permissions.md). - Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md). - [In GitLab 11.10 and later](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/25504), @@ -66,19 +65,19 @@ build: stage: build script: ./build only: - - master + - master test: stage: test script: ./test only: - - merge_requests + - merge_requests deploy: stage: deploy script: ./deploy only: - - master + - master ``` #### Excluding certain jobs @@ -207,7 +206,7 @@ The variable names begin with the `CI_MERGE_REQUEST_` prefix. ### Two pipelines created when pushing to a merge request If you are experiencing duplicated pipelines when using `rules`, take a look at -the [key details when using `rules`](../yaml/README.md#key-details-when-using-rules), +the [important differences between `rules` and `only`/`except`](../yaml/README.md#differences-between-rules-and-onlyexcept), which will help you get your starting configuration correct. If you are seeing two pipelines when using `only/except`, please see the caveats diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index a43085c8e7a..84fbefb080f 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -30,8 +30,7 @@ can still be successfully merged into the target. When the merge request can't be merged, the pipeline runs against the source branch only. For example, when: - The target branch has changes that conflict with the changes in the source branch. -- The merge request is a - [work in progress](../../../user/project/merge_requests/work_in_progress_merge_requests.md). +- The merge request is a [**Draft** merge request](../../../user/project/merge_requests/work_in_progress_merge_requests.md). In these cases, the pipeline runs as a [pipeline for merge requests](../index.md) and is labeled as `detached`. If these cases no longer exist, new pipelines will diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index fc8e6368d72..e12c9d109cc 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -146,7 +146,7 @@ is recreated and all pipelines restart. ### Merge request dropped from the merge train immediately -If a merge request is not mergeable (for example, it's WIP, there is a merge +If a merge request is not mergeable (for example, it's a draft merge request, there is a merge conflict, etc.), your merge request will be dropped from the merge train automatically. In these cases, the reason for dropping the merge request is in the **system notes**. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 67f9ad00c90..14669edf7eb 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -49,3 +49,10 @@ metrics: reports: metrics: metrics.txt ``` + +## Advanced Example + +An advanced example of an OpenMetrics text file (from the [Prometheus documentation](https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-format-example)) +renders in the merge request widget as: + + diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index f6868abc334..625b15ca4fb 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -250,14 +250,14 @@ image: node:latest cache: key: $CI_COMMIT_REF_SLUG paths: - - .npm/ + - .npm/ before_script: - npm ci --cache .npm --prefer-offline test_async: script: - - node ./specs/start.js ./specs/async.spec.js + - node ./specs/start.js ./specs/async.spec.js ``` ## Contexts and variables diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 48aaebc3456..18b3fe10bec 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -348,17 +348,17 @@ For example, these three jobs will be in a group named `build ruby`: build ruby 1/3: stage: build script: - - echo "ruby1" + - echo "ruby1" build ruby 2/3: stage: build script: - - echo "ruby2" + - echo "ruby2" build ruby 3/3: stage: build script: - - echo "ruby3" + - echo "ruby3" ``` In the pipeline, the result is a group named `build ruby` with three jobs: diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 1d60f412e5e..c4457d17dc2 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -33,7 +33,7 @@ pdf: script: xelatex mycv.tex artifacts: paths: - - mycv.pdf + - mycv.pdf expire_in: 1 week ``` @@ -87,8 +87,8 @@ Below is an example of collecting a JUnit XML file from Ruby's RSpec test tool: rspec: stage: test script: - - bundle install - - rspec --format RspecJunitFormatter --out rspec.xml + - bundle install + - rspec --format RspecJunitFormatter --out rspec.xml artifacts: reports: junit: rspec.xml @@ -114,14 +114,15 @@ The `dotenv` report collects a set of environment variables as artifacts. The collected variables are registered as runtime-created variables of the job, which is useful to [set dynamic environment URLs after a job finishes](../environments/index.md#set-dynamic-environment-urls-after-a-job-finishes). -There are a couple of limitations on top of the [original dotenv rules](https://github.com/motdotla/dotenv#rules). +There are a couple of exceptions to the [original dotenv rules](https://github.com/motdotla/dotenv#rules): -- The variable key can contain only letters, digits and underscore ('_'). -- The size of the dotenv file must be smaller than 5 kilobytes. -- The number of variables must be less than 10. -- It does not support variable substitution in the dotenv file itself. -- It does not support empty lines and comments (`#`) in dotenv file. -- It does not support quote escape, spaces in a quote, a new line expansion in a quote, in dotenv file. +- The variable key can contain only letters, digits, and underscores (`_`). +- The maximum size of the `.env` file is 5 KB. +- The maximum number of variables is 10. +- Variable substitution in the `.env` file is not supported. +- The `.env` file can't have empty lines or comments (starting with `#`). +- Key values in the `env` file cannot have spaces or newline characters (`\n`), including when using single or double quotes. +- Quote escaping during parsing (`key = 'value'` -> `{key: "value"}`) is not supported. #### `artifacts:reports:cobertura` @@ -145,9 +146,10 @@ plan report will be uploaded to GitLab as an artifact and will be automatically in merge requests. For more information, see [Output `terraform plan` information into a merge request](../../user/infrastructure/index.md#output-terraform-plan-information-into-a-merge-request). -#### `artifacts:reports:codequality` **(STARTER)** +#### `artifacts:reports:codequality` -> - Introduced in GitLab 11.5. +> - Introduced in [GitLab Starter](https://about.gitlab.com/pricing/) 11.5. +> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in GitLab 13.2. > - Requires GitLab Runner 11.5 and above. The `codequality` report collects [CodeQuality issues](../../user/project/merge_requests/code_quality.md) @@ -250,12 +252,23 @@ dashboards. > - Introduced in GitLab 11.5. > - Requires GitLab Runner 11.5 and above. -The `performance` report collects [Performance metrics](../../user/project/merge_requests/browser_performance_testing.md) +The `performance` report collects [Browser Performance Testing metrics](../../user/project/merge_requests/browser_performance_testing.md) as artifacts. -The collected Performance report will be uploaded to GitLab as an artifact and will +The collected Browser Performance report will be uploaded to GitLab as an artifact and will be automatically shown in merge requests. +#### `artifacts:reports:load_performance` **(PREMIUM)** + +> - Introduced in [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35260) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. +> - Requires GitLab Runner 11.5 and above. + +The `load_performance` report collects [Load Performance Testing metrics](../../user/project/merge_requests/load_performance_testing.md) +as artifacts. + +The report is uploaded to GitLab as an artifact and is +shown in merge requests automatically. + #### `artifacts:reports:metrics` **(PREMIUM)** > Introduced in GitLab 11.10. @@ -407,7 +420,7 @@ information in the UI. ## Erasing artifacts -DANGER: **Warning:** +DANGER: **Danger:** This is a destructive action that leads to data loss. Use with caution. You can erase a single job via the UI, which will also remove the job's diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index 9176444660d..ace765ddb41 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -133,28 +133,28 @@ build_b: test_a: stage: test - needs: build_a + needs: [build_a] script: - echo "This test job will start as soon as build_a finishes." - echo "It will not wait for build_b, or other jobs in the build stage, to finish." test_b: stage: test - needs: build_b + needs: [build_b] script: - echo "This test job will start as soon as build_b finishes." - echo "It will not wait for other jobs in the build stage to finish." deploy_a: stage: deploy - needs: test_a + needs: [test_a] script: - echo "Since build_a and test_a run quickly, this deploy job can run much earlier." - echo "It does not need to wait for build_b or test_b." deploy_b: stage: deploy - needs: test_b + needs: [test_b] script: - echo "Since build_b and test_b run slowly, this deploy job will run much later." ``` @@ -228,13 +228,13 @@ build_a: test_a: stage: test - needs: build_a + needs: [build_a] script: - echo "This job tests something." deploy_a: stage: deploy - needs: test_a + needs: [test_a] script: - echo "This job deploys something." ``` @@ -257,13 +257,13 @@ build_b: test_b: stage: test - needs: build_b + needs: [build_b] script: - echo "This job tests something else." deploy_b: stage: deploy - needs: test_b + needs: [test_b] script: - echo "This job deploys something else." ``` diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 2c9d4ccf2c4..39acef14443 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -36,7 +36,7 @@ in `.gitlab-ci.yml`. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/28919) in GitLab 12.0. -NOTE: **Note**: +NOTE: **Note:** As of GitLab 12.0, newly created projects will automatically have a default `git depth` value of `50`. @@ -283,7 +283,7 @@ Markdown code will embed the test coverage report badge of the `coverage` job into your `README.md`: ```markdown - + ``` ### Badge styles @@ -296,7 +296,7 @@ Pipeline badges can be rendered in different styles by adding the `style=style_n https://example.gitlab.com/<namespace>/<project>/badges/<branch>/coverage.svg?style=flat ``` - + #### Flat square @@ -306,17 +306,19 @@ https://example.gitlab.com/<namespace>/<project>/badges/<branch>/coverage.svg?st https://example.gitlab.com/<namespace>/<project>/badges/<branch>/coverage.svg?style=flat-square ``` - + ### Custom badge text +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17555) in GitLab 13.1. + The text for a badge can be customized. This can be useful to differentiate between multiple coverage jobs that run in the same pipeline. Customize the badge text and width by adding the `key_text=custom_text` and `key_width=custom_key_width` parameters to the URL: ```plaintext -https://gitlab.com/gitlab-org/gitlab-foss/badges/master/coverage.svg?job=karma&key_text=Frontend+Coverage&key_width=100 +https://gitlab.com/gitlab-org/gitlab/badges/master/coverage.svg?job=karma&key_text=Frontend+Coverage&key_width=100 ``` - + ## Environment Variables diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 67fac70c8de..283e4c69941 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -260,10 +260,6 @@ For example, in a Ruby application, you would need to have this script: Then, when your app is deployed via GitLab CI/CD, those variables should get replaced with their real values. -NOTE: **Note:** -Future enhancements [are planned](https://gitlab.com/gitlab-org/gitlab/-/issues/11322) -to make this process even easier. - ### Determining merge request ID The visual review tools retrieve the merge request ID from the `data-merge-request-id` diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 86fabd5d54f..21c99f928d8 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -6,6 +6,8 @@ type: reference --- # Configuring GitLab Runners +<!-- This topic contains several commented-out sections that were accidentally added in 13.2.--> +<!-- The commented-out sections will be added back in 13.3.--> In GitLab CI/CD, Runners run the code defined in [`.gitlab-ci.yml`](../yaml/README.md). A GitLab Runner is a lightweight, highly-scalable agent that picks up a CI job through @@ -35,12 +37,21 @@ Use shared Runners when you have multiple jobs with similar requirements. Rather having multiple Runners idling for many projects, you can have a few Runners that handle multiple projects. -If you are using a self-managed instance of GitLab, your administrator can create -shared Runners and configure them to use the -[executor](https://docs.gitlab.com/runner/executors/README.html) you want. +If you are using a self-managed instance of GitLab: -If you are using GitLab.com, you can select from a list of -[shared Runners that GitLab maintains](../../user/gitlab_com/index.md#shared-runners). +- Your administrator can install and register shared Runners by viewing the instructions + [here](https://docs.gitlab.com/runner/install/index.html). + <!-- going to your project's + <!-- **Settings > CI / CD**, expanding the **Runners** section, and clicking **Show Runner installation instructions**.--> + <!-- These instructions are also available [here](https://docs.gitlab.com/runner/install/index.html).--> +- The administrator can also configure a maximum number of shared Runner [pipeline minutes for + each group](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota-starter-only). + +If you are using GitLab.com: + +- You can select from a list of [shared Runners that GitLab maintains](../../user/gitlab_com/index.md#shared-runners). +- The shared Runners consume the [pipelines minutes](../../subscriptions/index.md#ci-pipeline-minutes) + included with your account. #### How shared Runners pick jobs @@ -95,16 +106,38 @@ The fair usage algorithm assigns jobs in this order: 1. Job 6 is next, because Project 3 is the only project left with no running jobs. 1. Lastly we choose Job 3... because, again, it's the only job left. -#### Enable a shared Runner +#### Enable shared Runners + +On GitLab.com, [shared Runners](#shared-runners) are enabled in all projects by +default. + +On self-managed instances of GitLab, an administrator must [install](https://docs.gitlab.com/runner/install/index.html) +and [register](https://docs.gitlab.com/runner/register/index.html) them. + +You can also enable shared Runners for individual projects. + +To enable shared Runners: + +1. Go to the project's **{settings}** **Settings > CI/CD** and expand the **Runners** section. +1. Click **Allow shared Runners**. -By default, all projects can use shared Runners, and they are enabled by default. +#### Disable shared Runners -However, you can enable or disable shared Runners for individual projects. +You can disable shared Runners for individual projects<!-- or for groups-->. +You must have Owner permissions for the project<!-- or group-->. -To enable or disable a shared Runner: +To disable shared Runners for a project: 1. Go to the project's **{settings}** **Settings > CI/CD** and expand the **Runners** section. -1. Click **Allow shared Runners** or **Disable shared Runners**. +1. In the **Shared Runners** area, click **Disable shared Runners**. + +<!--To disable shared Runners for a group: + +1. Go to the group's **{settings}** **Settings > CI/CD** and expand the **Runners** section. +1. In the **Shared Runners** area, click **Disable shared Runners globally**. +1. Optionally, to allow shared Runners to be enabled for individual projects or subgroups, + click **Allow projects/subgroups to override the global setting**. +--> ### Group Runners @@ -126,14 +159,43 @@ To create a group Runner: 1. Note the URL and token. 1. [Register the Runner](https://docs.gitlab.com/runner/register/). +<!-- #### View and manage group Runners + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/37366/) in GitLab 13.3. + +You can view and manage all Runners for a group, its subgroups, and projects. +You can do this for your self-managed GitLab instance or for GitLab.com. +You must have [Owner permissions](../../user/permissions.md#group-members-permissions) for the group. + +1. Go to the group where you want to view the Runners. +1. Go to **{settings}** **Settings > CI/CD** and expand the **Runners** section. +1. The following fields are displayed. + + | Attribute | Description | + | ------------ | ----------- | + | Type | One or more of the following states: shared, group, specific, locked, or paused | + | Runner token | Token used to identify the Runner, and which the Runner uses to communicate with the GitLab instance | + | Description | Description given to the Runner when it was created | + | Version | GitLab Runner version | + | IP address | IP address of the host on which the Runner is registered | + | Projects | The count of projects to which the Runner is assigned | + | Jobs | Total of jobs run by the Runner | + | Tags | Tags associated with the Runner | + | Last contact | Timestamp indicating when the GitLab instance last contacted the Runner | + +From this page, you can edit, pause, and remove Runners from the group, its subgroups, and projects. --> + #### Pause or remove a group Runner -You can pause or remove a group Runner. +You can pause or remove a group Runner for your self-managed GitLab instance or for GitLab.com. You must have [Owner permissions](../../user/permissions.md#group-members-permissions) for the group. 1. Go to the group you want to remove or pause the Runner for. 1. Go to **{settings}** **Settings > CI/CD** and expand the **Runners** section. 1. Click **Pause** or **Remove Runner**. +<!-- - If you pause a group Runner that is used by multiple projects, the Runner pauses for all projects. --> +<!-- - From the group view, you cannot remove a Runner that is assigned to more than one project. --> +<!-- You must remove it from each project first. --> 1. On the confirmation dialog, click **OK**. ### Specific Runners diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index c6ac87ef888..47f11a6228c 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -62,9 +62,9 @@ and it creates a dependent pipeline relation visible on the build_docs: stage: deploy script: - - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline + - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline only: - - tags + - tags ``` Pipelines triggered that way also expose a special variable: @@ -86,11 +86,11 @@ build_submodule: image: debian stage: test script: - - apt update && apt install -y unzip - - curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test&job_token=$CI_JOB_TOKEN" - - unzip artifacts.zip + - apt update && apt install -y unzip + - curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test&job_token=$CI_JOB_TOKEN" + - unzip artifacts.zip only: - - tags + - tags ``` This allows you to use that for multi-project pipelines and download artifacts @@ -179,9 +179,9 @@ need to add in project A's `.gitlab-ci.yml`: build_docs: stage: deploy script: - - "curl --request POST --form token=TOKEN --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" + - "curl --request POST --form token=TOKEN --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" only: - - tags + - tags ``` This means that whenever a new tag is pushed on project A, the job will run and the @@ -235,24 +235,24 @@ variable is non-zero, `make upload` is run. ```yaml stages: -- test -- build -- package + - test + - build + - package run_tests: stage: test script: - - make test + - make test build_package: stage: build script: - - make build + - make build upload_package: stage: package script: - - if [ -n "${UPLOAD_TO_S3}" ]; then make upload; fi + - if [ -n "${UPLOAD_TO_S3}" ]; then make upload; fi ``` You can then trigger a rebuild while you pass the `UPLOAD_TO_S3` variable diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md new file mode 100644 index 00000000000..a019f8232a9 --- /dev/null +++ b/doc/ci/troubleshooting.md @@ -0,0 +1,38 @@ +--- +stage: Verify +group: Continuous Integration +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference +--- + +# Troubleshooting CI/CD + +## Merge request pipeline widget + +The merge request pipeline widget shows information about the pipeline status in a Merge Request. It's displayed above the [merge request ability to merge widget](#merge-request-ability-to-merge-widget). + +There are several messages that can be displayed depending on the status of the pipeline. + +### "Checking pipeline status" + +This message is shown when the merge request has no pipeline associated with the latest commit yet and [Pipelines must succeed](../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) is turned on. This might be because: + +- GitLab hasn't finished creating the pipeline yet. +- You are using an external CI service and GitLab hasn't heard back from the service yet. +- You are not using CI/CD pipelines in your project. + +After the pipeline is created, the message will update with the pipeline status. + +Note: Currently if you delete the latest pipeline of a Merge Request, this message will be shown instead of a meaningful error message. This is a known issue and should be resolved soon. + +## Merge request ability to merge widget + +The merge request status widget shows the **Merge** button and whether or not a merge request is ready to merge. If the merge request can't be merged, the reason for this is displayed. + +If the pipeline is still running, the **Merge** button is replaced with the **Merge when pipeline succeeds** button. + +If [**Merge Trains**](merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md) are enabled, the button is either **Add to merge train** or **Add to merge train when pipeline succeeds**. **(PREMIUM)** + +### "A CI/CD pipeline must run and be successful before merge" + +This message is shown if the [Pipelines must succeed](../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) setting is enabled in the project and a pipeline has not yet run successfully. This also applies if the pipeline has not been created yet, or if you are waiting for an external CI service. If you don't use pipelines for your project, then you should disable **Pipelines must succeed** so you can accept merge requests. diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 541e739082f..4f9a1d8dd27 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -227,18 +227,18 @@ To protect a variable: 1. Select the **Protect variable** check box. 1. Click **Update variable**. -The variable is available for all subsequent pipelines. +The variable is available for all subsequent pipelines. Protected variables can only +be updated or viewed by project members with [maintainer permissions](../../user/permissions.md#project-members-permissions). ### Custom variables validated by GitLab Some variables are listed in the UI so you can choose them more quickly. -GitLab validates the values of these variables to ensure they are in the correct format. | Variable | Allowed Values | Introduced in | |-------------------------|----------------------------------------------------|---------------| -| `AWS_ACCESS_KEY_ID` | 20 characters: letters, digits | 12.10 | +| `AWS_ACCESS_KEY_ID` | Any | 12.10 | | `AWS_DEFAULT_REGION` | Any | 12.10 | -| `AWS_SECRET_ACCESS_KEY` | 40 characters: letters, digits, special characters | 12.10 | +| `AWS_SECRET_ACCESS_KEY` | Any | 12.10 | NOTE: **Note:** When you store credentials, there are security implications. If you are using AWS keys, @@ -341,6 +341,7 @@ export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-foss" export CI_PROJECT_NAME="gitlab-foss" export CI_PROJECT_TITLE="GitLab FOSS" export CI_PROJECT_NAMESPACE="gitlab-org" +export CI_PROJECT_ROOT_NAMESPACE="gitlab-org" export CI_PROJECT_PATH="gitlab-org/gitlab-foss" export CI_PROJECT_URL="https://example.com/gitlab-org/gitlab-foss" export CI_REGISTRY="registry.example.com" @@ -457,9 +458,6 @@ The UI interface for Instance-level CI/CD variables is under development but rea It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) can opt to disable it for your instance. -NOTE: **Note:** -This feature will not work if the [instance-level CI/CD variables API feature flag is disabled](../../api/instance_level_ci_variables.md#enable-or-disable-instance-level-cicd-variables-core-only). - To disable it: ```ruby @@ -895,8 +893,8 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_SERVER_VERSION_PATCH=0 ++ export CI_SERVER_REVISION=f4cc00ae823 ++ CI_SERVER_REVISION=f4cc00ae823 -++ export GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_clusters,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal -++ GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_clusters,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal +++ export GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal +++ GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal ++ export CI_PROJECT_ID=17893 ++ CI_PROJECT_ID=17893 ++ export CI_PROJECT_NAME=ci-debug-trace @@ -909,6 +907,8 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_PROJECT_PATH_SLUG=gitlab-examples-ci-debug-trace ++ export CI_PROJECT_NAMESPACE=gitlab-examples ++ CI_PROJECT_NAMESPACE=gitlab-examples +++ export CI_PROJECT_ROOT_NAMESPACE=gitlab-examples +++ CI_PROJECT_ROOT_NAMESPACE=gitlab-examples ++ export CI_PROJECT_URL=https://gitlab.com/gitlab-examples/ci-debug-trace ++ CI_PROJECT_URL=https://gitlab.com/gitlab-examples/ci-debug-trace ++ export CI_PROJECT_VISIBILITY=public diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 8463bbeb582..98a2e7ae22f 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -23,15 +23,17 @@ future GitLab releases.** You can add a command to your `.gitlab-ci.yml` file to [output the values of all variables available for a job](README.md#list-all-environment-variables). +Kubernetes-specific environment variables are detailed in the +[Kubernetes deployment variables](../../user/project/clusters/index.md#deployment-variables) section. + | Variable | GitLab | Runner | Description | |-----------------------------------------------|--------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `ARTIFACT_DOWNLOAD_ATTEMPTS` | 8.15 | 1.9 | Number of attempts to download artifacts running a job | | `CHAT_CHANNEL` | 10.6 | all | Source chat channel which triggered the [ChatOps](../chatops/README.md) command | | `CHAT_INPUT` | 10.6 | all | Additional arguments passed in the [ChatOps](../chatops/README.md) command | | `CI` | all | 0.4 | Mark that job is executed in CI environment | | `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL | | `CI_BUILDS_DIR` | all | 11.10 | Top-level directory where builds are executed. | -| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch before a merge request. Only populated when there is a merge request associated with the pipeline. | +| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch. Is always `0000000000000000000000000000000000000000` in pipelines for merge requests. | | `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. | | `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | | `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built | @@ -69,8 +71,8 @@ You can add a command to your `.gitlab-ci.yml` file to | `CI_JOB_URL` | 11.1 | 0.5 | Job details URL | | `CI_KUBERNETES_ACTIVE` | 13.0 | all | Included with the value `true` only if the pipeline has a Kubernetes cluster available for deployments. Not included if no cluster is available. Can be used as an alternative to [`only:kubernetes`/`except:kubernetes`](../yaml/README.md#onlykubernetesexceptkubernetes) with [`rules:if`](../yaml/README.md#rulesif) | | `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of username(s) of assignee(s) for the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_ID` | 11.6 | all | The ID of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | -| `CI_MERGE_REQUEST_IID` | 11.6 | all | The IID of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | +| `CI_MERGE_REQUEST_ID` | 11.6 | all | The project-level ID of the merge request. Only available if [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. | +| `CI_MERGE_REQUEST_IID` | 11.6 | all | The instance-level IID of the merge request. Only available If [the pipelines are for merge requests](../merge_request_pipelines/index.md) and the merge request is created. | | `CI_MERGE_REQUEST_LABELS` | 11.9 | all | Comma-separated label names of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | | `CI_MERGE_REQUEST_MILESTONE` | 11.9 | all | The milestone title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | | `CI_MERGE_REQUEST_PROJECT_ID` | 11.6 | all | The ID of the project of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | @@ -92,13 +94,14 @@ You can add a command to your `.gitlab-ci.yml` file to | `CI_PAGES_URL` | 11.8 | all | URL to GitLab Pages-built pages. Always belongs to a subdomain of `CI_PAGES_DOMAIN`. | | `CI_PIPELINE_ID` | 8.10 | all | The unique ID of the current pipeline that GitLab CI/CD uses internally | | `CI_PIPELINE_IID` | 11.0 | all | The unique ID of the current pipeline scoped to project | -| `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `trigger`, `schedule`, `api`, `pipeline`, `parent_pipeline`, `external`, `chat`, `merge_request_event`, and `external_pull_request_event`. For pipelines created before GitLab 9.5, this will show as `unknown` | +| `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/README.md#authentication-tokens) (renamed to `cross_project_pipeline` since 13.0). For pipelines created before GitLab 9.5, this will show as `unknown`. | | `CI_PIPELINE_TRIGGERED` | all | all | The flag to indicate that job was [triggered](../triggers/README.md) | | `CI_PIPELINE_URL` | 11.1 | 0.5 | Pipeline details URL | | `CI_PROJECT_DIR` | all | all | The full path where the repository is cloned and where the job is run. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see [Advanced configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) for GitLab Runner. | | `CI_PROJECT_ID` | all | all | The unique ID of the current project that GitLab CI/CD uses internally | | `CI_PROJECT_NAME` | 8.10 | 0.5 | The name of the directory for the project that is currently being built. For example, if the project URL is `gitlab.example.com/group-name/project-1`, the `CI_PROJECT_NAME` would be `project-1`. | | `CI_PROJECT_NAMESPACE` | 8.10 | 0.5 | The project namespace (username or group name) that is currently being built | +| `CI_PROJECT_ROOT_NAMESPACE` | 13.2 | 0.5 | The **root** project namespace (username or group name) that is currently being built. For example, if `CI_PROJECT_NAME` is `root-group/child-group/grandchild-group`, `CI_PROJECT_ROOT_NAMESPACE` would be `root-group`. | | `CI_PROJECT_PATH` | 8.10 | 0.5 | The namespace with project name | | `CI_PROJECT_PATH_SLUG` | 9.3 | all | `$CI_PROJECT_PATH` lowercased and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | | `CI_PROJECT_REPOSITORY_LANGUAGES` | 12.3 | all | Comma-separated, lowercased list of the languages used in the repository (e.g. `ruby,javascript,html,css`) | @@ -107,8 +110,8 @@ You can add a command to your `.gitlab-ci.yml` file to | `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility (internal, private, public) | | `CI_REGISTRY` | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry. This variable will include a `:port` value if one has been specified in the registry configuration. | | `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | -| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to use to push containers to the GitLab Container Registry | -| `CI_REGISTRY_USER` | 9.0 | all | The username to use to push containers to the GitLab Container Registry | +| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to use to push containers to the GitLab Container Registry, for the current project. | +| `CI_REGISTRY_USER` | 9.0 | all | The username to use to push containers to the GitLab Container Registry, for the current project. | | `CI_REPOSITORY_URL` | 9.0 | all | The URL to clone the Git repository | | `CI_RUNNER_DESCRIPTION` | 8.10 | 0.5 | The description of the runner as saved in GitLab | | `CI_RUNNER_EXECUTABLE_ARCH` | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor) | @@ -129,11 +132,9 @@ You can add a command to your `.gitlab-ci.yml` file to | `CI_SERVER_VERSION_MINOR` | 11.4 | all | GitLab version minor component | | `CI_SERVER_VERSION_PATCH` | 11.4 | all | GitLab version patch component | | `CI_SHARED_ENVIRONMENT` | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | -| `GET_SOURCES_ATTEMPTS` | 8.15 | 1.9 | Number of attempts to fetch sources running a job | | `GITLAB_CI` | all | all | Mark that job is executed in GitLab CI/CD environment | | `GITLAB_FEATURES` | 10.6 | all | The comma separated list of licensed features available for your instance and plan | | `GITLAB_USER_EMAIL` | 8.12 | all | The email of the user who started the job | | `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the job | | `GITLAB_USER_LOGIN` | 10.0 | all | The login username of the user who started the job | | `GITLAB_USER_NAME` | 10.0 | all | The real name of the user who started the job | -| `RESTORE_CACHE_ATTEMPTS` | 8.15 | 1.9 | Number of attempts to restore the cache running a job | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 7ed5a8fec01..e1d1d27efed 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD Pipeline Configuration Reference +# GitLab CI/CD pipeline configuration reference GitLab CI/CD [pipelines](../pipelines/index.md) are configured using a YAML file called `.gitlab-ci.yml` within each project. @@ -117,7 +117,7 @@ The following table lists available parameters for jobs: | [`when`](#when) | When to run job. Also available: `when:manual` and `when:delayed`. | | [`environment`](#environment) | Name of an environment to which the job deploys. Also available: `environment:name`, `environment:url`, `environment:on_stop`, `environment:auto_stop_in` and `environment:action`. | | [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, and `cache:policy`. | -| [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, `artifacts:reports`, `artifacts:reports:junit`, `artifacts:reports:cobertura`, and `artifacts:reports:terraform`.<br><br>In GitLab [Enterprise Edition](https://about.gitlab.com/pricing/), these are available: `artifacts:reports:codequality`, `artifacts:reports:sast`, `artifacts:reports:dependency_scanning`, `artifacts:reports:container_scanning`, `artifacts:reports:dast`, `artifacts:reports:license_scanning`, `artifacts:reports:license_management` (removed in GitLab 13.0),`artifacts:reports:performance` and `artifacts:reports:metrics`. | +| [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, `artifacts:reports`, `artifacts:reports:codequality`, `artifacts:reports:junit`, `artifacts:reports:cobertura`, and `artifacts:reports:terraform`.<br><br>In GitLab [Enterprise Edition](https://about.gitlab.com/pricing/), these are available: `artifacts:reports:sast`, `artifacts:reports:dependency_scanning`, `artifacts:reports:container_scanning`, `artifacts:reports:dast`, `artifacts:reports:license_scanning`, `artifacts:reports:license_management` (removed in GitLab 13.0), `artifacts:reports:performance`, `artifacts:reports:load_performance`, and `artifacts:reports:metrics`. | | [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. | | [`coverage`](#coverage) | Code coverage settings for a given job. | | [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. | @@ -130,6 +130,7 @@ The following table lists available parameters for jobs: | [`variables`](#variables) | Define job variables on a job level. | | [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | | [`resource_group`](#resource_group) | Limit job concurrency. | +| [`release`](#release) | Instructs the Runner to generate a [Release](../../user/project/releases/index.md) object. | NOTE: **Note:** Parameters `types` and `type` are [deprecated](#deprecated-parameters). @@ -256,8 +257,8 @@ karma: ### `stages` -`stages` is used to define stages that can be used by jobs and is defined -globally. +`stages` is used to define stages that contain jobs and is defined +globally for the pipeline. The specification of `stages` allows for having flexible multi stage pipelines. The ordering of elements in `stages` defines the ordering of jobs' execution: @@ -297,6 +298,26 @@ determine whether or not a pipeline is created. It currently accepts a single `rules:` key that operates similarly to [`rules:` defined within jobs](#rules), enabling dynamic configuration of the pipeline. +If you are new to GitLab CI/CD and `workflow: rules`, you may find the [`workflow:rules` templates](#workflowrules-templates) useful. + +To define your own `workflow: rules`, the configuration options currently available are: + +- [`if`](#rulesif): Define a rule. +- [`when`](#when): May be set to `always` or `never` only. If not provided, the default value is `always`​. + +The list of `if` rules is evaluated until a single one is matched. If none +match, the last `when` will be used: + +```yaml +workflow: + rules: + - if: $CI_COMMIT_REF_NAME =~ /-wip$/ + when: never + - if: $CI_COMMIT_TAG + when: never + - when: always +``` + #### `workflow:rules` templates > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217732) in GitLab 13.0. @@ -334,24 +355,6 @@ include: - template: 'Workflows/MergeRequest-Pipelines.gitlab-ci.yml' ``` -If you prefer to define your own rules, the configuration options currently available are:​ - -- [`if`](#rulesif): Define a rule. -- [`when`](#when): May be set to `always` or `never` only. If not provided, the default value is `always`​. - -The list of `if` rules is evaluated until a single one is matched. If none -match, the last `when` will be used: - -```yaml -workflow: - rules: - - if: $CI_COMMIT_REF_NAME =~ /-wip$/ - when: never - - if: $CI_COMMIT_TAG - when: never - - when: always -``` - ### `include` > - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. @@ -375,12 +378,14 @@ otherwise the external file won't be included. | [`remote`](#includeremote) | Include a file from a remote URL. Must be publicly accessible. | | [`template`](#includetemplate) | Include templates which are provided by GitLab. | +The `include` methods do not support [variable expansion](../variables/where_variables_can_be_used.md#variables-usage). + NOTE: **Note:** `.gitlab-ci.yml` configuration included by all methods is evaluated at pipeline creation. The configuration is a snapshot in time and persisted in the database. Any changes to referenced `.gitlab-ci.yml` configuration won't be reflected in GitLab until the next pipeline is created. -The files defined in `include` are: +The files defined by `include` are: - Deep merged with those in `.gitlab-ci.yml`. - Always evaluated first and merged with the content of `.gitlab-ci.yml`, @@ -388,11 +393,11 @@ The files defined in `include` are: TIP: **Tip:** Use merging to customize and override included CI/CD configurations with local -definitions. +definitions. Local definitions in `.gitlab-ci.yml` will override included definitions. NOTE: **Note:** -Using YAML aliases across different YAML files sourced by `include` is not -supported. You must only refer to aliases in the same file. Instead +Using [YAML anchors](#anchors) across different YAML files sourced by `include` is not +supported. You must only refer to anchors in the same file. Instead of using YAML anchors, you can use the [`extends` keyword](#extends). #### `include:local` @@ -699,6 +704,101 @@ job: - Write-Host "This text is not colored" ``` +#### Multiline commands + +You can split long commands into multi-line commands to improve readability +using [`|` (literal) and `>` (folded) YAML multiline block scalar indicators](https://yaml-multiline.info/). + +CAUTION: **Warning:** +If multiple commands are combined into one command string, only the last command's +failure or success will be reported, +[incorrectly ignoring failures from earlier commands due to a bug](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25394). +If the success of the job depends on the success or failure of these commands, +you can run the commands as separate `script:` items, or add `exit 1` commands +as appropriate to the command string where needed. + +You can use the `|` (literal) YAML multiline block scalar indicator to write +commands over multiple lines in the `script` section of a job description. +Each line is treated as a separate command. +Only the first command is repeated in the job log, but additional +commands are still executed: + +```yaml +job: + script: + - | + echo "First command line." + echo "Second command line." + echo "Third command line." +``` + +The example above renders in the job log as: + +```shell +$ echo First command line # collapsed multi-line command +First command line +Second command line. +Third command line. +``` + +The `>` (folded) YAML multiline block scalar indicator treats empty lines between +sections as the start of a new command: + +```yaml +job: + script: + - > + echo "First command line + is split over two lines." + + echo "Second command line." +``` + +This behaves similarly to writing multiline commands without the `>` or `|` block +scalar indicators: + +```yaml +job: + script: + - echo "First command line + is split over two lines." + + echo "Second command line." +``` + +Both examples above render in the job log as: + +```shell +$ echo First command line is split over two lines. # collapsed multi-line command +First command line is split over two lines. +Second command line. +``` + +When the `>` or `|` block scalar indicators are omitted, GitLab will form the command +by concatenating non-empty lines, so make sure the lines can run when concatenated. + +Shell [here documents](https://en.wikipedia.org/wiki/Here_document) work with the +`|` and `>` operators as well. The example below transliterates the lower case letters +to upper case: + +```yaml +job: + script: + - | + tr a-z A-Z << END_TEXT + one two three + four five six + END_TEXT +``` + +Results in: + +```shell +$ tr a-z A-Z << END_TEXT # collapsed multi-line command + ONE TWO THREE + FOUR FIVE SIX +``` + ### `stage` `stage` is defined per-job and relies on [`stages`](#stages) which is defined @@ -874,24 +974,38 @@ spinach: ``` In GitLab 12.0 and later, it's also possible to use multiple parents for -`extends`. The algorithm used for merge is "closest scope wins", so -keys from the last member will always shadow anything defined on other +`extends`. + +#### Merge details + +`extends` is able to merge hashes but not arrays. +The algorithm used for merge is "closest scope wins", so +keys from the last member will always override anything defined on other levels. For example: ```yaml .only-important: + variables: + URL: "http://my-url.internal" + IMPORTANT_VAR: "the details" only: - master - stable tags: - production + script: + - echo "Hello world!" .in-docker: + variables: + URL: "http://docker-url.internal" tags: - docker image: alpine rspec: + variables: + GITLAB: "is-awesome" extends: - .only-important - .in-docker @@ -903,6 +1017,10 @@ This results in the following `rspec` job: ```yaml rspec: + variables: + URL: "http://docker-url.internal" + IMPORTANT_VAR: "the details" + GITLAB: "is-awesome" only: - master - stable @@ -913,6 +1031,15 @@ rspec: - rake rspec ``` +Note that in the example above: + +- `variables` sections have been merged but that `URL: "http://my-url.internal"` +has been overwritten by `URL: "http://docker-url.internal"`. +- `tags: ['production']` has been overwritten by `tags: ['docker']`. +- `script` has not been merged but rather `script: ['echo "Hello world!"']` has + been overwritten by `script: ['rake rspec']`. Arrays can be + merged using [YAML anchors](#anchors). + #### Using `extends` and `include` together `extends` works across configuration files combined with `include`. @@ -942,113 +1069,295 @@ the `.template` job, and uses the `alpine` Docker image as defined in the local > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3. -`rules` allows for a list of individual rule objects to be evaluated -*in order*, until one matches and dynamically provides attributes to the job. +The `rules` keyword can be used to include or exclude jobs in pipelines. + +Rules are evaluated *in order* until the first match. When matched, the job +is either included or excluded from the pipeline, depending on the configuration. +If included, the job also has [certain attributes](#rules-attributes) +added to it. CAUTION: **Caution:** -`rules` can't be used in combination with `only/except` as it is a replacement for that functionality. If you attempt to do this, the linter will return a +`rules` can't be used in combination with [`only/except`](#onlyexcept-basic) because it is a replacement for +that functionality. If you attempt to do this, the linter returns a `key may not be used with rules` error. -#### Key details when using `rules` +#### Rules attributes -A very important difference between `rules` and `only/except`, is that jobs defined -with `rules` trigger merge request pipelines by default, but `only/except` jobs do not. -This may be surprising if migrating from `only` and `except`, so new users of `rules` -can use one of the [`workflow: rules` templates](#workflowrules-templates) to get started. -This will ensure that the behavior is more stable as you start adding additional `rules` -blocks, and will avoid issues like creating a duplicate, merge request (detached) pipeline. +The job attributes allowed by `rules` are: -We don't recommend mixing `only/except` jobs with `rules` jobs in the same pipeline. -It may not cause YAML errors, but debugging the exact execution behavior can be complex -due to the different default behaviors of `only/except` and `rules`. +- [`when`](#when): If not defined, defaults to `when: on_success`. + - If used as `when: delayed`, `start_in` is also required. +- [`allow_failure`](#allow_failure): If not defined, defaults to `allow_failure: false`. + +If a rule evaluates to true, and `when` has any value except `never`, the job is included in the pipeline. + +For example: + +```yaml +docker build: + script: docker build -t my-image:$CI_COMMIT_REF_SLUG . + rules: + - if: '$CI_COMMIT_BRANCH == "master"' + when: delayed + start_in: '3 hours' + allow_failure: true +``` + +Additional job configuration may be added to rules in the future. If something +useful is not available, please [open an issue](https://gitlab.com/gitlab-org/gitlab/-/issues). + +#### Rules clauses + +Available rule clauses are: + +| Clause | Description | +|----------------------------|------------------------------------------------------------------------------------------------------------------------------------| +| [`if`](#rulesif) | Add or exclude jobs from a pipeline by evaluating an `if` statement. Similar to [`only:variables`](#onlyvariablesexceptvariables). | +| [`changes`](#ruleschanges) | Add or exclude jobs from a pipeline based on what files are changed. Same as [`only:changes`](#onlychangesexceptchanges). | +| [`exists`](#rulesexists) | Add or exclude jobs from a pipeline based on the presence of specific files. | -### Rules clauses +Rules are evaluated in order until a match is found. If a match is found, the attributes +are checked to see if the job should be added to the pipeline. If no attributes are defined, +the defaults are: -Available rule clauses include: +- `when: on_success` +- `allow_failure: false` -- [`if`](#rulesif) (similar to [`only:variables`](#onlyvariablesexceptvariables)) -- [`changes`](#ruleschanges) (same as [`only:changes`](#onlychangesexceptchanges)) -- [`exists`](#rulesexists) +The job is added to the pipeline: -For example, using `if`. This configuration specifies that `job` should be built -and run for every pipeline on merge requests targeting `master`, regardless of -the status of other builds: +- If a rule matches and has `when: on_success`, `when: delayed` or `when: always`. +- If no rules match, but the last clause is `when: on_success`, `when: delayed` + or `when: always` (with no rule). + +The job is not added to the pipeline: + +- If no rules match, and there is no standalone `when: on_success`, `when: delayed` or + `when: always`. +- If a rule matches, and has `when: never` as the attribute. + +For example, using `if` clauses to strictly limit when jobs run: ```yaml job: script: "echo Hello, Rules!" rules: - - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' - when: always - - if: '$VAR =~ /pattern/' + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' when: manual + allow_failure: true + - if: '$CI_PIPELINE_SOURCE == "schedule"' +``` + +In this example: + +- If the pipeline is for a merge request, the first rule matches, and the job + is added to the [merge request pipeline](../merge_request_pipelines/index.md) + with attributes of: + - `when: manual` (manual job) + - `allow_failure: true` (allows the pipeline to continue running even if the manual job is not run) +- If the pipeline is **not** for a merge request, the first rule doesn't match, and the + second rule is evaluated. +- If the pipeline is a scheduled pipeline, the second rule matches, and the job + is added to the scheduled pipeline. Since no attributes were defined, it is added + with: + - `when: on_success` (default) + - `allow_failure: false` (default) +- In **all other cases**, no rules match, so the job is **not** added to any other pipeline. + +Alternatively, you can define a set of rules to exclude jobs in a few cases, but +run them in all other cases: + +```yaml +job: + script: "echo Hello, Rules!" + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + when: never + - if: '$CI_PIPELINE_SOURCE == "schedule"' + when: never - when: on_success ``` -In this example, if the first rule: +- If the pipeline is for a merge request, the job is **not** be added to the pipeline. +- If the pipeline is a scheduled pipeline, the job is **not** be added to the pipeline. +- In **all other cases**, the job is added to the pipeline, with `when: on_success`. + +CAUTION: **Caution:** +If you use `when: on_success`, `always`, or `delayed` as the final rule, two +simultaneous pipelines may start. Both push pipelines and merge request pipelines can +be triggered by the same event (a push to the source branch for an open merge request). +See the [important differences between `rules` and `only`/`except`](#differences-between-rules-and-onlyexcept) +for more details. + +#### Differences between `rules` and `only`/`except` + +Jobs defined with `only/except` do not trigger merge request pipelines by default. +You must explicitly add `only: merge_requests`. + +Jobs defined with `rules` can trigger all types of pipelines. +You do not have to explicitly configure each type. + +For example: + +```yaml +job: + script: "echo This creates double pipelines!" + rules: + - if: '$CUSTOM_VARIABLE == "false"' + when: never + - when: always +``` + +This job does not run when `$CUSTOM_VARIABLE` is false, but it *does* run in **all** +other pipelines, including **both** push (branch) and merge request pipelines. With +this configuration, every push to an open merge request's source branch +causes duplicated pipelines. Explicitly allowing both push and merge request pipelines +in the same job could have the same effect. + +We recommend using [`workflow: rules`](#workflowrules) to limit which types of pipelines +are permitted. Allowing only merge request pipelines, or only branch pipelines, +eliminates duplicated pipelines. Alternatively, you can rewrite the rules to be +stricter, or avoid using a final `when` (`always`, `on_success` or `delayed`). + +Also, we don't recommend mixing `only/except` jobs with `rules` jobs in the same pipeline. +It may not cause YAML errors, but debugging the exact execution behavior can be complex +due to the different default behaviors of `only/except` and `rules`. + +##### `rules:if` -- Matches, the job will be given the `when:always` attribute. -- Does not match, the second and third rules will be evaluated sequentially - until a match is found. That is, the job will be given either the: - - `when: manual` attribute if the second rule matches. **The stage won't complete until this manual job is triggered and completes successfully.** - - `when: on_success` attribute if the second rule does not match. The third - rule will always match when reached because it has no conditional clauses. +`rules:if` clauses determine whether or not jobs are added to a pipeline by evaluating +a simple `if` statement. If the `if` statement is true, the job is either included +or excluded from a pipeline. In plain English, `if` rules can be interpreted as one of: -#### `rules:if` +- "If this rule evaluates to true, add the job" (default). +- "If this rule evaluates to true, do not add the job" (by adding `when: never`). `rules:if` differs slightly from `only:variables` by accepting only a single -expression string, rather than an array of them. Any set of expressions to be -evaluated should be conjoined into a single expression using `&&` or `||`, and use +expression string per rule, rather than an array of them. Any set of expressions to be +evaluated can be conjoined into a single expression by using `&&` or `||`, and use the [variable matching syntax](../variables/README.md#syntax-of-environment-variable-expressions). +`if:` clauses are evaluated based on the values of [predefined environment variables](../variables/predefined_variables.md) +or [custom environment variables](../variables/README.md#custom-environment-variables). + For example: ```yaml job: script: "echo Hello, Rules!" rules: - - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' # This rule will be evaluated + - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' when: always - - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/' # This rule will only be evaluated if the target branch is not "master" + - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/' when: manual - - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME' # If neither of the first two match but the simple presence does, we set to "on_success" by default + allow_failure: true + - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME' # Checking for the presence of a variable is possible ``` Some details regarding the logic that determines the `when` for the job: -- If none of the provided rules match, the job is set to `when: never`, and is +- If none of the provided rules match, the job is set to `when: never` and is not included in the pipeline. - A rule without any conditional clause, such as a `when` or `allow_failure` rule without `if` or `changes`, always matches, and is always used if reached. -- If a rule matches and has no `when` defined, the rule will use the `when` +- If a rule matches and has no `when` defined, the rule uses the `when` defined for the job, which defaults to `on_success` if not defined. +- You can define `when` once per rule, or once at the job-level, which applies to + all rules. You can't mix `when` at the job-level with `when` in rules. + +For behavior similar to the [`only`/`except` keywords](#onlyexcept-basic), you can +check the value of the `$CI_PIPELINE_SOURCE` variable. + +| Value | Description | +|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `push` | For pipelines triggered by a `git push` event, including for branches and tags. | +| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. | +| `trigger` | For pipelines created by using a trigger token. | +| `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | +| `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). | +| `external` | When using CI services other than GitLab. | +| `pipelines` | For multi-project pipelines created by [using the API with `CI_JOB_TOKEN`](../triggers/README.md#when-used-with-multi-project-pipelines). | +| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/README.md) command. | +| `webide` | For pipelines created by using the [WebIDE](../../user/project/web_ide/index.md). | +| `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). | +| `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | +| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`, use this in the child pipeline configuration so that it can be triggered by the parent pipeline. | + +For example: + +```yaml +job: + script: "echo Hello, Rules!" + rules: + - if: '$CI_PIPELINE_SOURCE == "schedule"' + when: manual + allow_failure: true + - if: '$CI_PIPELINE_SOURCE == "push"' +``` + +This example runs the job as a manual job in scheduled pipelines or in push +pipelines (to branches or tags), with `when: on_success` (default). It does not +add the job to any other pipeline type. + +Another example: + +```yaml +job: + script: "echo Hello, Rules!" + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: '$CI_PIPELINE_SOURCE == "schedule"' +``` + +This example runs the job as a `when: on_success` job in [merge request pipelines](../merge_request_pipelines/index.md) +and scheduled pipelines. It does not run in any other pipeline type. -#### `rules:changes` +Other commonly used variables for `if` clauses: -`rules: changes` works exactly the same way as `only: changes` and `except: changes`, -accepting an array of paths. Similarly, it will always return true if there is no -Git push event. See [`only/except: changes`](#onlychangesexceptchanges) for more information. +- `if: $CI_COMMIT_TAG`: If changes are pushed for a tag. +- `if: $CI_COMMIT_BRANCH`: If changes are pushed to any branch. +- `if: '$CI_COMMIT_BRANCH == "master"'`: If changes are pushed to `master`. +- `if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'`: If changes are pushed to the default + branch (usually `master`). Useful if reusing the same configuration in multiple + projects with potentially different default branches. +- `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression. +- `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/README.md#custom-environment-variables) + `CUSTOM_VARIABLE` does **not** match a regular expression. +- `if: '$CUSTOM_VARIABLE == "value1"'`: If the custom variable `CUSTOM_VARIABLE` is + exactly `value1`. + +##### `rules:changes` + +To determine if jobs should be added to a pipeline, `rules: changes` clauses check +the files changed by Git push events. + +`rules: changes` works exactly the same way as [`only: changes` and `except: changes`](#onlychangesexceptchanges), +accepting an array of paths. Similarly, it always returns true if there is no +Git push event. It should only be used for branch pipelines or merge request pipelines. For example: ```yaml +workflow: + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + docker build: script: docker build -t my-image:$CI_COMMIT_REF_SLUG . rules: - - changes: # Will include the job and set to when:manual if any of the follow paths match a modified file. - - Dockerfile + - changes: + - Dockerfile when: manual - - if: '$VAR == "string value"' - when: manual # Will include the job and set to when:manual if the expression evaluates to true, after the `changes:` rule fails to match. - - when: on_success # If neither of the first rules match, set to on_success + allow_failure: true ``` -In this example, a job either set to: +In this example: -- Run manually if `Dockerfile` has changed OR `$VAR == "string value"`. -- `when:on_success` by the last rule, where no earlier clauses evaluate to true. +- [`workflow: rules`](#workflowrules) allows only pipelines for merge requests for all jobs. +- If `Dockerfile` has changed, add the job to the pipeline as a manual job, and allow the pipeline + to continue running even if the job is not triggered (`allow_failure: true`). +- If `Dockerfile` has not changed, do not add job to any pipeline (same as `when: never`). -#### `rules:exists` +##### `rules:exists` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4. @@ -1062,7 +1371,7 @@ job: script: docker build -t my-image:$CI_COMMIT_REF_SLUG . rules: - exists: - - Dockerfile + - Dockerfile ``` You can also use glob patterns to match multiple files in any directory within @@ -1075,14 +1384,14 @@ job: script: bundle exec rspec rules: - exists: - - spec/**.rb + - spec/**.rb ``` NOTE: **Note:** For performance reasons, using `exists` with patterns is limited to 10000 checks. After the 10000th check, rules with patterned globs will always match. -#### `rules:allow_failure` +##### `rules:allow_failure` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30235) in GitLab 12.8. @@ -1122,53 +1431,18 @@ docker build: rules: - if: '$VAR == "string value"' changes: # Will include the job and set to when:manual if any of the follow paths match a modified file. - - Dockerfile - - docker/scripts/* + - Dockerfile + - docker/scripts/* when: manual # - when: never would be redundant here, this is implied any time rules are listed. ``` -The only clauses currently available are: - -- `if` -- `changes` -- `exists` - Keywords such as `branches` or `refs` that are currently available for `only`/`except` are not yet available in `rules` as they are being individually considered for their usage and behavior in this context. Future keyword improvements are being discussed in our [epic for improving `rules`](https://gitlab.com/groups/gitlab-org/-/epics/2783), where anyone can add suggestions or requests. -#### Permitted attributes - -The only job attributes currently set by `rules` are: - -- `when`. -- `start_in`, if `when` is set to `delayed`. -- `allow_failure`. - -A job will be included in a pipeline if `when` is evaluated to any value -except `never`. - -Delayed jobs require a `start_in` value, so rule objects do as well. For -example: - -```yaml -docker build: - script: docker build -t my-image:$CI_COMMIT_REF_SLUG . - rules: - - changes: # Will include the job and delay 3 hours when the Dockerfile has changed - - Dockerfile - when: delayed - start_in: '3 hours' - - when: on_success # Otherwise include the job and set to run normally -``` - -Additional job configuration may be added to rules in the future. If something -useful is not available, please -[open an issue](https://gitlab.com/gitlab-org/gitlab/-/issues). - ### `only`/`except` (basic) NOTE: **Note:** @@ -1193,20 +1467,20 @@ There are a few rules that apply to the usage of job policy: In addition, `only` and `except` allow the use of special keywords: -| **Value** | **Description** | -| --------- | ---------------- | -| `branches` | When a Git reference of a pipeline is a branch. | -| `tags` | When a Git reference of a pipeline is a tag. | -| `api` | When pipeline has been triggered by a second pipelines API (not triggers API). | -| `external` | When using CI services other than GitLab. | -| `pipelines` | For multi-project triggers, created using the API with `CI_JOB_TOKEN`. | -| `pushes` | Pipeline is triggered by a `git push` by the user. | -| `schedules` | For [scheduled pipelines](../pipelines/schedules.md). | -| `triggers` | For pipelines created using a trigger token. | -| `web` | For pipelines created using **Run pipeline** button in GitLab UI (under your project's **Pipelines**). | -| `merge_requests` | When a merge request is created or updated (See [pipelines for merge requests](../merge_request_pipelines/index.md)). | -| `external_pull_requests`| When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). | -| `chat` | For jobs created using a [GitLab ChatOps](../chatops/README.md) command. | +| **Value** | **Description** | +|--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `branches` | When the Git reference for a pipeline is a branch. | +| `tags` | When the Git reference for a pipeline is a tag. | +| `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). | +| `external` | When using CI services other than GitLab. | +| `pipelines` | For multi-project pipelines created by using the API with `CI_JOB_TOKEN`. | +| `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. | +| `schedules` | For [scheduled pipelines](../pipelines/schedules.md). | +| `triggers` | For pipelines created by using a trigger token. | +| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. | +| `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). | +| `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). | +| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/README.md) command. | In the example below, `job` will run only for refs that start with `issue-`, whereas all branches will be skipped: @@ -1295,7 +1569,7 @@ and must be surrounded by `/`. So `issue-/.*/` won't work to match all tag names or branch names that begin with `issue-`. -TIP: **Tip** +TIP: **Tip:** Use anchors `^` and `$` to avoid the regular expression matching only a substring of the tag name or branch name. For example, `/^issue-.*$/` is equivalent to `/^issue-/`, @@ -1701,8 +1975,11 @@ Feature::enable(:ci_dag_limit_needs) > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14311) in GitLab v12.6. -When using `needs`, artifact downloads are controlled with `artifacts: true` or `artifacts: false`. -The `dependencies` keyword should not be used with `needs`, as this is deprecated since GitLab 12.6. +When using `needs`, artifact downloads are controlled with `artifacts: true` (default) or `artifacts: false`. + +Since GitLab 12.6, you can't combine the [`dependencies`](#dependencies) keyword +with `needs` to control artifact downloads in jobs. `dependencies` is still valid +in jobs that do not use `needs`. In the example below, the `rspec` job will download the `build_job` artifacts, while the `rubocop` job won't: @@ -2160,7 +2437,8 @@ review_app: stage: deploy script: make deploy-app environment: - name: review + name: review/$CI_COMMIT_REF_NAME + url: https://$CI_ENVIRONMENT_SLUG.example.com on_stop: stop_review_app stop_review_app: @@ -2170,7 +2448,7 @@ stop_review_app: script: make delete-app when: manual environment: - name: review + name: review/$CI_COMMIT_REF_NAME action: stop ``` @@ -2195,8 +2473,6 @@ The `stop_review_app` job is **required** to have the following keywords defined - `when` - [reference](#when) - `environment:name` - `environment:action` -- `stage` should be the same as the `review_app` in order for the environment - to stop automatically when the branch is deleted Additionally, both jobs should have matching [`rules`](../yaml/README.md#onlyexcept-basic) or [`only/except`](../yaml/README.md#onlyexcept-basic) configuration. In the example @@ -2554,7 +2830,12 @@ be available for download in the GitLab UI. Paths are relative to the project directory (`$CI_PROJECT_DIR`) and can't directly link outside it. Wildcards can be used that follow the [glob](https://en.wikipedia.org/wiki/Glob_(programming)) -patterns and [`filepath.Match`](https://golang.org/pkg/path/filepath/#Match). +patterns and: + +- In [GitLab Runner 13.0](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2620) and later, +[`doublestar.Glob`](https://pkg.go.dev/github.com/bmatcuk/doublestar@v1.2.2?tab=doc#Match). +- In GitLab Runner 12.10 and earlier, +[`filepath.Match`](https://pkg.go.dev/path/filepath/#Match). To restrict which jobs a specific job will fetch artifacts from, see [dependencies](#dependencies). @@ -2645,10 +2926,10 @@ For example, to match a single file: ```yaml test: - script: [ 'echo 1' ] + script: [ "echo 'test' > file.txt" ] artifacts: expose_as: 'artifact 1' - paths: ['path/to/file.txt'] + paths: ['file.txt'] ``` With this configuration, GitLab will add a link **artifact 1** to the relevant merge request @@ -2658,14 +2939,15 @@ An example that will match an entire directory: ```yaml test: - script: [ 'echo 1' ] + script: [ "mkdir test && echo 'test' > test/file.txt" ] artifacts: expose_as: 'artifact 1' - paths: ['path/to/directory/'] + paths: ['test/'] ``` Note the following: +- Artifacts do not display in the merge request UI when using variables to define the `artifacts:paths`. - A maximum of 10 job artifacts per merge request can be exposed. - Glob patterns are unsupported. - If a directory is specified, the link will be to the job [artifacts browser](../pipelines/job_artifacts.md#browsing-artifacts) if there is more than @@ -2818,7 +3100,7 @@ job: expire and are therefore deleted, counting from the time they are uploaded and stored on GitLab. If the expiry time is not defined, it defaults to the [instance wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration-core-only) -(30 days by default, forever on GitLab.com). +(30 days by default). You can use the **Keep** button on the job page to override expiration and keep artifacts forever. @@ -2846,8 +3128,10 @@ job: ``` NOTE: **Note:** -For artifacts created in [GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/issues/16267) -and later, the latest artifact for a ref is always kept, regardless of the expiry time. +Since [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/16267), the latest +artifacts for refs can be locked against deletion, and kept regardless of the expiry time. This feature is disabled +by default and is not ready for production use. It can be enabled for testing by +enabling the `:keep_latest_artifact_for_ref` and `:destroy_only_unlocked_expired_artifacts` [feature flags](../../administration/feature_flags.md). #### `artifacts:reports` @@ -2863,14 +3147,15 @@ These are the available report types: | [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) | The `dotenv` report collects a set of environment variables. | | [`artifacts:reports:cobertura`](../pipelines/job_artifacts.md#artifactsreportscobertura) | The `cobertura` report collects Cobertura coverage XML files. | | [`artifacts:reports:terraform`](../pipelines/job_artifacts.md#artifactsreportsterraform) | The `terraform` report collects Terraform `tfplan.json` files. | -| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality-starter) **(STARTER)** | The `codequality` report collects CodeQuality issues. | +| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects CodeQuality issues. | | [`artifacts:reports:sast`](../pipelines/job_artifacts.md#artifactsreportssast-ultimate) **(ULTIMATE)** | The `sast` report collects Static Application Security Testing vulnerabilities. | | [`artifacts:reports:dependency_scanning`](../pipelines/job_artifacts.md#artifactsreportsdependency_scanning-ultimate) **(ULTIMATE)** | The `dependency_scanning` report collects Dependency Scanning vulnerabilities. | | [`artifacts:reports:container_scanning`](../pipelines/job_artifacts.md#artifactsreportscontainer_scanning-ultimate) **(ULTIMATE)** | The `container_scanning` report collects Container Scanning vulnerabilities. | | [`artifacts:reports:dast`](../pipelines/job_artifacts.md#artifactsreportsdast-ultimate) **(ULTIMATE)** | The `dast` report collects Dynamic Application Security Testing vulnerabilities. | | [`artifacts:reports:license_management`](../pipelines/job_artifacts.md#artifactsreportslicense_management-ultimate) **(ULTIMATE)** | The `license_management` report collects Licenses (*removed from GitLab 13.0*). | | [`artifacts:reports:license_scanning`](../pipelines/job_artifacts.md#artifactsreportslicense_scanning-ultimate) **(ULTIMATE)** | The `license_scanning` report collects Licenses. | -| [`artifacts:reports:performance`](../pipelines/job_artifacts.md#artifactsreportsperformance-premium) **(PREMIUM)** | The `performance` report collects Performance metrics. | +| [`artifacts:reports:performance`](../pipelines/job_artifacts.md#artifactsreportsperformance-premium) **(PREMIUM)** | The `performance` report collects Browser Performance metrics. | +| [`artifacts:reports:load_performance`](../pipelines/job_artifacts.md#artifactsreportsload_performance-premium) **(PREMIUM)** | The `load_performance` report collects load performance metrics. | | [`artifacts:reports:metrics`](../pipelines/job_artifacts.md#artifactsreportsmetrics-premium) **(PREMIUM)** | The `metrics` report collects Metrics. | #### `dependencies` @@ -3127,6 +3412,10 @@ This keyword allows the creation of two different types of downstream pipelines: - [Multi-project pipelines](../multi_project_pipelines.md#creating-multi-project-pipelines-from-gitlab-ciyml) - [Child pipelines](../parent_child_pipelines.md) +[Since GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/197140/), you can +see which job triggered a downstream pipeline by hovering your mouse cursor over +the downstream pipeline job in the [pipeline graph](../pipelines/index.md#visualize-pipelines). + NOTE: **Note:** Using a `trigger` with `when:manual` together results in the error `jobs:#{job-name} when should be on_success, on_failure or always`, because `when:manual` prevents @@ -3271,7 +3560,7 @@ is enabled. When enabled, a pipeline on the same branch will be canceled when: -- it's made redundant by a newer pipeline run. +- It's made redundant by a newer pipeline run. - Either all jobs are set as interruptible, or any uninterruptible jobs haven't started. Pending jobs are always considered interruptible. @@ -3352,6 +3641,182 @@ It can't start or end with `/`. For more information, see [Deployments Safety](../environments/deployment_safety.md). +### `release` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/19298) in GitLab 13.2. + +`release` indicates that the job creates a [Release](../../user/project/releases/index.md), +and optionally includes URLs for Release assets. + +These methods are supported: + +- [`tag_name`](#releasetag_name) +- [`name`](#releasename) (optional) +- [`description`](#releasedescription) (optional) +- [`ref`](#releaseref) (optional) +- [`milestones`](#releasemilestones) (optional) +- [`released_at`](#releasereleased_at) (optional) + +The Release is created only if the job processes without error. If the Rails API +returns an error during Release creation, the `release` job fails. + +#### `release-cli` Docker image + +The Docker image to use for the `release-cli` must be specified, using the following directive: + +```yaml +image: registry.gitlab.com/gitlab-org/release-cli:latest +``` + +#### Script + +All jobs require a `script` tag at a minimum. A `:release` job can use the output of a +`:script` tag, but if this is not necessary, a placeholder script can be used, for example: + +```yaml +script: + - echo 'release job' +``` + +An [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/223856) exists to remove this requirement in an upcoming version of GitLab. + +A pipeline can have multiple `release` jobs, for example: + +```yaml +ios-release: + script: + - echo 'iOS release job' + release: + tag_name: v1.0.0-ios + description: 'iOS release v1.0.0' + +android-release: + script: + - echo 'Android release job' + release: + tag_name: v1.0.0-android + description: 'Android release v1.0.0' +``` + +#### `release:tag_name` + +The `tag_name` must be specified. It can refer to an existing Git tag or can be specified by the user. + +When the specified tag doesn't exist in the repository, a new tag is created from the associated SHA of the pipeline. + +For example, when creating a Release from a Git tag: + +```yaml +job: + release: + tag_name: $CI_COMMIT_TAG + description: changelog.txt +``` + +It is also possible to create any unique tag, in which case `only: tags` is not mandatory. +A semantic versioning example: + +```yaml +job: + release: + tag_name: ${MAJOR}_${MINOR}_${REVISION} + description: changelog.txt +``` + +- The Release is created only if the job's main script succeeds. +- If the Release already exists, it is not updated and the job with the `release` keyword fails. +- The `release` section executes after the `script` tag and before the `after_script`. + +#### `release:name` + +The Release name. If omitted, it is populated with the value of `release: tag_name`. + +#### `release:description` + +Specifies the longer description of the Release. + +#### `release:ref` + +If the `release: tag_name` doesn’t exist yet, the release is created from `ref`. +`ref` can be a commit SHA, another tag name, or a branch name. + +#### `release:milestones` + +The title of each milestone the release is associated with. + +#### `release:released_at` + +The date and time when the release is ready. Defaults to the current date and time if not +defined. Expected in ISO 8601 format (2019-03-15T08:00:00Z). + +#### Complete example for `release` + +Combining the individual examples given above for `release` results in the following +code snippets. There are two options, depending on how you generate the +tags. These options cannot be used together, so choose one: + +- To create a release when you push a Git tag, or when you add a Git tag + in the UI by going to **Repository > Tags**: + + ```yaml + release_job: + stage: release + image: registry.gitlab.com/gitlab-org/release-cli:latest + rules: + - if: $CI_COMMIT_TAG # Run this job when a tag is created manually + script: + - echo 'running release_job' + release: + name: 'Release $CI_COMMIT_TAG' + description: 'Created using the release-cli $EXTRA_DESCRIPTION' # $EXTRA_DESCRIPTION must be defined + tag_name: '$CI_COMMIT_TAG' # elsewhere in the pipeline. + ref: '$CI_COMMIT_TAG' + milestones: + - 'm1' + - 'm2' + - 'm3' + released_at: '2020-07-15T08:00:00Z' # Optional, will auto generate if not defined, + # or can use a variable. + ``` + +- To create a release automatically when changes are pushed to the default branch, + using a new Git tag that is defined with variables: + + ```yaml + release_job: + stage: release + image: registry.gitlab.com/gitlab-org/release-cli:latest + rules: + - if: $CI_COMMIT_TAG + when: never # Do not run this job when a tag is created manually + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run this job when the default branch changes + script: + - echo 'running release_job' + release: + name: 'Release $CI_COMMIT_SHA' + description: 'Created using the release-cli $EXTRA_DESCRIPTION' # $EXTRA_DESCRIPTION and the tag_name + tag_name: 'v${MAJOR}.${MINOR}.${REVISION}' # variables must be defined elsewhere + ref: '$CI_COMMIT_SHA' # in the pipeline. + milestones: + - 'm1' + - 'm2' + - 'm3' + released_at: '2020-07-15T08:00:00Z' # Optional, will auto generate if not defined, + # or can use a variable. + ``` + +#### `releaser-cli` command line + +The entries under the `:release` node are transformed into a `bash` command line and sent +to the Docker container, which contains the [release-cli](https://gitlab.com/gitlab-org/release-cli). +You can also call the `release-cli` directly from a `script` entry. + +The YAML described above would be translated into a CLI command like this: + +```shell +release-cli create --name "Release $CI_COMMIT_SHA" --description "Created using the release-cli $EXTRA_DESCRIPTION" --tag-name "v${MAJOR}.${MINOR}.${REVISION}" --ref "$CI_COMMIT_SHA" --released-at "2020-07-15T08:00:00Z" --milestone "m1" --milestone "m2" --milestone "m3" +``` + ### `pages` `pages` is a special job that is used to upload static content to GitLab that @@ -3460,7 +3925,8 @@ variables: GIT_STRATEGY: none ``` -NOTE: **Note:** `GIT_STRATEGY` is not supported for +NOTE: **Note:** +`GIT_STRATEGY` is not supported for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html), but may be in the future. See the [support Git strategy with Kubernetes executor feature proposal](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3847) for updates. @@ -3625,7 +4091,7 @@ You can set them globally or per-job in the [`variables`](#variables) section. > Introduced in GitLab 8.9 as an experimental feature. -NOTE: **Note**: +NOTE: **Note:** As of GitLab 12.0, newly created projects will automatically have a [default `git depth` value of `50`](../pipelines/settings.md#git-shallow-clone). You can specify the depth of fetching and cloning using `GIT_DEPTH`. This allows @@ -3748,6 +4214,10 @@ of `.gitlab-ci.yml`. Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/). +In most cases, the [`extends` keyword](#extends) is more user friendly and should +be used over these special YAML features. YAML anchors may still +need to be used to merge arrays. + ### Anchors > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. @@ -3755,7 +4225,8 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya YAML has a handy feature called 'anchors', which lets you easily duplicate content across your document. Anchors can be used to duplicate/inherit properties, and is a perfect example to be used with [hidden jobs](#hide-jobs) -to provide templates for your jobs. +to provide templates for your jobs. When there is duplicate keys, GitLab will +perform a reverse deep merge based on the keys. The following example uses anchors and map merging. It will create two jobs, `test1` and `test2`, that will inherit the parameters of `.job_template`, each @@ -3816,6 +4287,8 @@ directive defined in `.postgres_services` and `.mysql_services` respectively: .job_template: &job_definition script: - test project + tags: + - dev .postgres_services: services: &postgres_definition @@ -3830,6 +4303,8 @@ directive defined in `.postgres_services` and `.mysql_services` respectively: test:postgres: <<: *job_definition services: *postgres_definition + tags: + - postgres test:mysql: <<: *job_definition @@ -3842,6 +4317,8 @@ The expanded version looks like this: .job_template: script: - test project + tags: + - dev .postgres_services: services: @@ -3859,6 +4336,8 @@ test:postgres: services: - postgres - ruby + tags: + - postgres test:mysql: script: @@ -3866,13 +4345,19 @@ test:mysql: services: - mysql - ruby + tags: + - dev ``` You can see that the hidden jobs are conveniently used as templates. NOTE: **Note:** +Note that `tags: [dev]` has been overwritten by `tags: [postgres]`. + +NOTE: **Note:** You can't use YAML anchors across multiple files when leveraging the [`include`](#include) -feature. Anchors are only valid within the file they were defined in. +feature. Anchors are only valid within the file they were defined in. Instead +of using YAML anchors, you can use the [`extends` keyword](#extends). #### YAML anchors for `before_script` and `after_script` @@ -3886,11 +4371,11 @@ Example: ```yaml .something_before: &something_before -- echo 'something before' + - echo 'something before' .something_after: &something_after -- echo 'something after' -- echo 'another thing after' + - echo 'something after' + - echo 'another thing after' job_name: before_script: @@ -3912,7 +4397,7 @@ For example: ```yaml .something: &something -- echo 'something' + - echo 'something' job_name: script: diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 969b54d5be8..f7ed7248dc4 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD YAML includes +# GitLab CI/CD include examples In addition to the [`includes` examples](README.md#include) listed in the [GitLab CI YAML reference](README.md), this page lists more variations of `include` |
