diff options
Diffstat (limited to 'doc/ci')
100 files changed, 1638 insertions, 2456 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 740be7d1dbd..9b555c0ee68 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -7,10 +7,10 @@ description: "Learn how to use GitLab CI/CD, the GitLab built-in Continuous Inte type: index --- -# GitLab CI/CD +# GitLab CI/CD **(FREE)** GitLab CI/CD is a tool built into GitLab for software development -through the [continuous methodologies](introduction/index.md#introduction-to-cicd-methodologies): +through the [continuous methodologies](introduction/index.md): - Continuous Integration (CI) - Continuous Delivery (CD) @@ -55,9 +55,9 @@ at the repository's root. This file creates a [pipeline](pipelines/index.md), wh To get started with GitLab CI/CD, we recommend you read through the following documents: -- [Get started with GitLab CI/CD](quick_start/README.md). +- [Get started with GitLab CI/CD](quick_start/index.md). - [Fundamental pipeline architectures](pipelines/pipeline_architectures.md). -- [GitLab CI/CD basic workflow](introduction/index.md#basic-cicd-workflow). +- [GitLab CI/CD basic workflow](introduction/index.md#gitlab-cicd-workflow). - [Step-by-step guide for writing `.gitlab-ci.yml` for the first time](../user/project/pages/getting_started/pages_from_scratch.md). If you're migrating from another CI/CD tool, check out our handy references: @@ -71,11 +71,11 @@ available through the UI. You can use them by creating a new file, choosing a template that suits your application, and adjusting it to your needs: - + While building your `.gitlab-ci.yml`, you can use the [CI/CD configuration visualization](pipeline_editor/index.md#visualize-ci-configuration) to facilitate your writing experience. -For a broader overview, see the [CI/CD getting started](quick_start/README.md) guide. +For a broader overview, see the [CI/CD getting started](quick_start/index.md) guide. After you're familiar with how GitLab CI/CD works, see the [`.gitlab-ci.yml` full reference](yaml/README.md) @@ -90,7 +90,7 @@ GitLab CI/CD uses a number of concepts to describe and run your build and deploy | Concept | Description | |:--------------------------------------------------------|:-------------------------------------------------------------------------------| | [Pipelines](pipelines/index.md) | Structure your CI/CD process through pipelines. | -| [Environment variables](variables/README.md) | Reuse values based on a variable/value key pair. | +| [CI/CD variables](variables/README.md) | Reuse values based on a variable/value key pair. | | [Environments](environments/index.md) | Deploy your application to different environments (e.g., staging, production). | | [Job artifacts](pipelines/job_artifacts.md) | Output, use, and reuse job artifacts. | | [Cache dependencies](caching/index.md) | Cache your dependencies for a faster execution. | @@ -105,9 +105,9 @@ GitLab CI/CD supports numerous configuration options: | Configuration | Description | |:----------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------| | [Schedule pipelines](pipelines/schedules.md) | Schedule pipelines to run as often as you need. | -| [Custom path for `.gitlab-ci.yml`](pipelines/settings.md#custom-ci-configuration-path) | Define a custom path for the CI/CD configuration file. | +| [Custom path for `.gitlab-ci.yml`](pipelines/settings.md#custom-cicd-configuration-path) | Define a custom path for the CI/CD configuration file. | | [Git submodules for CI/CD](git_submodules.md) | Configure jobs for using Git submodules. | -| [SSH keys for CI/CD](ssh_keys/README.md) | Using SSH keys in your CI pipelines. | +| [SSH keys for CI/CD](ssh_keys/index.md) | Using SSH keys in your CI pipelines. | | [Pipeline triggers](triggers/README.md) | Trigger pipelines through the API. | | [Pipelines for Merge Requests](merge_request_pipelines/index.md) | Design a pipeline structure for running a pipeline in merge requests. | | [Integrate with Kubernetes clusters](../user/project/clusters/index.md) | Connect your project to Google Kubernetes Engine (GKE) or an existing Kubernetes cluster. | @@ -126,15 +126,15 @@ Its feature set is listed on the table below according to DevOps stages. |:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------| | **Configure** | | | [Auto DevOps](../topics/autodevops/index.md) | Set up your app's entire lifecycle. | -| [ChatOps](chatops/README.md) | Trigger CI jobs from chat, with results sent back to the channel. | +| [ChatOps](chatops/index.md) | Trigger CI jobs from chat, with results sent back to the channel. | |-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| | **Verify** | | | [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | | [Load Performance Testing](../user/project/merge_requests/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | -| [CI services](services/README.md) | Link Docker containers with your base image. | +| [CI services](services/index.md) | Link Docker containers with your base image. | | [Code Quality](../user/project/merge_requests/code_quality.md) | Analyze your source code quality. | | [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) **(PREMIUM)** | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | -| [Interactive Web Terminals](interactive_web_terminal/index.md) **(CORE ONLY)** | Open an interactive web terminal to debug the running jobs. | +| [Interactive Web Terminals](interactive_web_terminal/index.md) **(FREE SELF)** | Open an interactive web terminal to debug the running jobs. | | [Unit test reports](unit_test_reports.md) | Identify script failures directly on merge requests. | | [Using Docker images](docker/using_docker_images.md) | Use GitLab and GitLab Runner with Docker to build and test applications. | |-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| @@ -160,9 +160,7 @@ Its feature set is listed on the table below according to DevOps stages. Find example project code and tutorials for using GitLab CI/CD with a variety of app frameworks, languages, and platforms on the [CI Examples](examples/README.md) page. -GitLab also provides [example projects](https://gitlab.com/gitlab-examples) pre-configured to use GitLab CI/CD. - -## Administration **(CORE ONLY)** +## Administration **(FREE SELF)** As a GitLab administrator, you can change the default behavior of GitLab CI/CD for: @@ -206,7 +204,7 @@ been necessary. These are: #### 12.0 -- [Use refspec to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4069). +- [Use `refspec` to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4069). - [Old cache configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4070). - [Old metrics server configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4072). - [Remove `FF_K8S_USE_ENTRYPOINT_OVER_COMMAND`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4073). diff --git a/doc/ci/autodeploy/index.md b/doc/ci/autodeploy/index.md deleted file mode 100644 index 8c7d9c1da64..00000000000 --- a/doc/ci/autodeploy/index.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../topics/autodevops/stages.md#auto-deploy' ---- - -This document was moved to [another location](../../topics/autodevops/stages.md#auto-deploy). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/autodeploy/quick_start_guide.md b/doc/ci/autodeploy/quick_start_guide.md deleted file mode 100644 index 8c7d9c1da64..00000000000 --- a/doc/ci/autodeploy/quick_start_guide.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../topics/autodevops/stages.md#auto-deploy' ---- - -This document was moved to [another location](../../topics/autodevops/stages.md#auto-deploy). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md deleted file mode 100644 index 4344a544798..00000000000 --- a/doc/ci/build_artifacts/README.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/project/pipelines/job_artifacts.md' ---- - -This document was moved to [pipelines/job_artifacts.md](../../user/project/pipelines/job_artifacts.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 08a45714de3..bfc332e35b1 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -87,7 +87,7 @@ use one or more of the following: that are only available to a particular project. - [Use a `key`](../yaml/README.md#cachekey) that fits your workflow (for example, different caches on each branch). For that, you can take advantage of the - [CI/CD predefined variables](../variables/README.md#predefined-environment-variables). + [predefined CI/CD variables](../variables/README.md#predefined-cicd-variables). For runners to work with caches efficiently, you must do one of the following: diff --git a/doc/ci/chatops/README.md b/doc/ci/chatops/README.md index 5d6af0b78db..c94d6e3ea80 100644 --- a/doc/ci/chatops/README.md +++ b/doc/ci/chatops/README.md @@ -1,116 +1,8 @@ --- -stage: Configure -group: Configure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: index, concepts, howto +redirect_to: 'index.md' --- -# GitLab ChatOps +This document was moved to [another location](index.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4466) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24780) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.9. - -GitLab ChatOps provides a method to interact with CI/CD jobs through chat services -like Slack. Many organizations' discussion, collaboration, and troubleshooting takes -place in chat services. Having a method to run CI/CD jobs with output -posted back to the channel can significantly augment your team's workflow. - -## How GitLab ChatOps works - -GitLab ChatOps is built upon [GitLab CI/CD](../README.md) and -[Slack Slash Commands](../../user/project/integrations/slack_slash_commands.md). -ChatOps provides a `run` action for [slash commands](../../integration/slash_commands.md) -with the following arguments: - -- A `<job name>` to execute. -- The `<job arguments>`. - -ChatOps passes the following [CI/CD variables](../variables/README.md#predefined-environment-variables) -to the job: - -- `CHAT_INPUT` contains any additional arguments. -- `CHAT_CHANNEL` is set to the name of channel the action was triggered in. - -When executed, ChatOps looks up the specified job name and attempts to match it -to a corresponding job in [`.gitlab-ci.yml`](../yaml/README.md). If a matching job -is found on `master`, a pipeline containing only that job is scheduled. After the -job completes: - -- If the job completes in *less than 30 minutes*, the ChatOps sends the job's output to Slack. -- If the job completes in *more than 30 minutes*, the job must use the - [Slack API](https://api.slack.com/) to send data to the channel. - -To use the `run` command, you must have -[Developer access or above](../../user/permissions.md#project-members-permissions). -If a job shouldn't be able to be triggered from chat, you can set the job to `except: [chat]`. - -## Best practices for ChatOps CI jobs - -Since ChatOps is built upon GitLab CI/CD, the job has all the same features and -functions available. Consider these best practices when creating ChatOps jobs: - -- GitLab strongly recommends you set `only: [chat]` so the job does not run as part - of the standard CI pipeline. -- If the job is set to `when: manual`, ChatOps creates the pipeline, but the job waits to be started. -- ChatOps provides limited support for access control. If the user triggering the - slash command has [Developer access or above](../../user/permissions.md#project-members-permissions) - in the project, the job runs. The job itself can use existing - [CI/CD variables](../variables/README.md#predefined-environment-variables) like - `GITLAB_USER_ID` to perform additional rights validation, but - these variables can be [overridden](../variables/README.md#priority-of-environment-variables). - -### Controlling the ChatOps reply - -The output for jobs with a single command is sent to the channel as a reply. For -example, the chat reply of the following job is `Hello World` in the channel: - -```yaml -hello-world: - stage: chatops - only: [chat] - script: - - echo "Hello World" -``` - -Jobs that contain multiple commands (or `before_script`) return additional -content in the chat reply. In these cases, both the commands and their output are -included, with the commands wrapped in ANSI color codes. - -To selectively reply with the output of one command, its output must be bounded by -the `chat_reply` section. For example, the following job lists the files in the -current directory: - -```yaml -ls: - stage: chatops - only: [chat] - script: - - echo "This command will not be shown." - - echo -e "section_start:$( date +%s ):chat_reply\r\033[0K\n$( ls -la )\nsection_end:$( date +%s ):chat_reply\r\033[0K" -``` - -## GitLab ChatOps examples - -The GitLab.com team created a repository of [common ChatOps scripts](https://gitlab.com/gitlab-com/chatops) -they use to interact with our Production instance of GitLab. Administrators of -other GitLab instances may find them useful. They can serve as inspiration for ChatOps -scripts you can write to interact with your own applications. - -## GitLab ChatOps icon - -The [official GitLab ChatOps icon](img/gitlab-chatops-icon.png) is available for download. -You can find and download the official GitLab ChatOps icon here. - - - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after 2021-05-01. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md new file mode 100644 index 00000000000..48f8e595df6 --- /dev/null +++ b/doc/ci/chatops/index.md @@ -0,0 +1,116 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: index, concepts, howto +--- + +# GitLab ChatOps **(FREE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4466) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24780) to [GitLab Free](https://about.gitlab.com/pricing/) in 11.9. + +GitLab ChatOps provides a method to interact with CI/CD jobs through chat services +like Slack. Many organizations' discussion, collaboration, and troubleshooting takes +place in chat services. Having a method to run CI/CD jobs with output +posted back to the channel can significantly augment your team's workflow. + +## How GitLab ChatOps works + +GitLab ChatOps is built upon [GitLab CI/CD](../README.md) and +[Slack Slash Commands](../../user/project/integrations/slack_slash_commands.md). +ChatOps provides a `run` action for [slash commands](../../integration/slash_commands.md) +with the following arguments: + +- A `<job name>` to execute. +- The `<job arguments>`. + +ChatOps passes the following [CI/CD variables](../variables/README.md#predefined-cicd-variables) +to the job: + +- `CHAT_INPUT` contains any additional arguments. +- `CHAT_CHANNEL` is set to the name of channel the action was triggered in. + +When executed, ChatOps looks up the specified job name and attempts to match it +to a corresponding job in [`.gitlab-ci.yml`](../yaml/README.md). If a matching job +is found on `master`, a pipeline containing only that job is scheduled. After the +job completes: + +- If the job completes in *less than 30 minutes*, the ChatOps sends the job's output to Slack. +- If the job completes in *more than 30 minutes*, the job must use the + [Slack API](https://api.slack.com/) to send data to the channel. + +To use the `run` command, you must have +[Developer access or above](../../user/permissions.md#project-members-permissions). +If a job shouldn't be able to be triggered from chat, you can set the job to `except: [chat]`. + +## Best practices for ChatOps CI jobs + +Since ChatOps is built upon GitLab CI/CD, the job has all the same features and +functions available. Consider these best practices when creating ChatOps jobs: + +- GitLab strongly recommends you set `only: [chat]` so the job does not run as part + of the standard CI pipeline. +- If the job is set to `when: manual`, ChatOps creates the pipeline, but the job waits to be started. +- ChatOps provides limited support for access control. If the user triggering the + slash command has [Developer access or above](../../user/permissions.md#project-members-permissions) + in the project, the job runs. The job itself can use existing + [CI/CD variables](../variables/README.md#predefined-cicd-variables) like + `GITLAB_USER_ID` to perform additional rights validation, but + these variables can be [overridden](../variables/README.md#priority-of-cicd-variables). + +### Controlling the ChatOps reply + +The output for jobs with a single command is sent to the channel as a reply. For +example, the chat reply of the following job is `Hello World` in the channel: + +```yaml +hello-world: + stage: chatops + only: [chat] + script: + - echo "Hello World" +``` + +Jobs that contain multiple commands (or `before_script`) return additional +content in the chat reply. In these cases, both the commands and their output are +included, with the commands wrapped in ANSI color codes. + +To selectively reply with the output of one command, its output must be bounded by +the `chat_reply` section. For example, the following job lists the files in the +current directory: + +```yaml +ls: + stage: chatops + only: [chat] + script: + - echo "This command will not be shown." + - echo -e "section_start:$( date +%s ):chat_reply\r\033[0K\n$( ls -la )\nsection_end:$( date +%s ):chat_reply\r\033[0K" +``` + +## GitLab ChatOps examples + +The GitLab.com team created a repository of [common ChatOps scripts](https://gitlab.com/gitlab-com/chatops) +they use to interact with our Production instance of GitLab. Administrators of +other GitLab instances may find them useful. They can serve as inspiration for ChatOps +scripts you can write to interact with your own applications. + +## GitLab ChatOps icon + +The [official GitLab ChatOps icon](img/gitlab-chatops-icon.png) is available for download. +You can find and download the official GitLab ChatOps icon here. + + + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index a466214374b..38930eb60ad 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -14,8 +14,9 @@ GitLab CI/CD can be used with Bitbucket Cloud by: To use GitLab CI/CD with a Bitbucket Cloud repository: -1. In GitLab create a **CI/CD for external repository**, select **Repo by URL** and - create the project. +1. <!-- vale gitlab.Spelling = NO --> In GitLab create a **CI/CD for external repository**, select + **Repo by URL** and create the project. + <!-- vale gitlab.Spelling = YES -->  @@ -49,7 +50,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:  -1. In GitLab, from **Settings > CI/CD > Environment variables**, add variables to allow +1. In GitLab, from **Settings > CI/CD > Variables**, add variables to allow communication with Bitbucket via the Bitbucket API: `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 8e3d609b5dc..2a8b050b224 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -35,13 +35,13 @@ repositories: your project, update commit statuses, and create a web hook to notify GitLab of new commits. -1. In GitLab, go to the [new project page](../../gitlab-basics/create-project.md#create-a-project-in-gitlab), select the **CI/CD for external repository** tab, and then click +1. In GitLab, go to the [new project page](../../user/project/working_with_projects.md#create-a-project), select the **CI/CD for external repository** tab, and then click **GitHub**. 1. Paste the token into the **Personal access token** field and click **List Repositories**. Click **Connect** to select the repository. -1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). +1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/index.md). GitLab: diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 44534ddf793..cc6c629fb47 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -25,11 +25,15 @@ snippets disabled. These features To connect to an external repository: +<!-- vale gitlab.Spelling = NO --> + 1. From your GitLab dashboard, click **New project**. 1. Switch to the **CI/CD for external repository** tab. 1. Choose **GitHub** or **Repo by URL**. 1. The next steps are similar to the [import flow](../../user/project/import/index.md). +<!-- vale gitlab.Spelling = YES --> +  ## Pipelines for external pull requests diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 0be86527cb5..ccacb3c61d3 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -38,7 +38,7 @@ Some credentials are required to be able to run `aws` commands: A new **Access key ID** and **Secret access key** are generated. Please take a note of them right away. 1. In your GitLab project, go to **Settings > CI / CD**. Set the following as - [environment variables](../variables/README.md#gitlab-cicd-environment-variables) + [CI/CD variables](../variables/README.md) (see table below): - Access key ID. @@ -47,11 +47,11 @@ Some credentials are required to be able to run `aws` commands: You might want to check if the AWS service you intend to use is [available in the chosen region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). - | Env. variable name | Value | - |:------------------------|:-----------------------| - | `AWS_ACCESS_KEY_ID` | Your Access key ID | - | `AWS_SECRET_ACCESS_KEY` | Your Secret access key | - | `AWS_DEFAULT_REGION` | Your region code | + | Environment variable name | Value | + |:-------------------------------|:-----------------------| + | `AWS_ACCESS_KEY_ID` | Your Access key ID | + | `AWS_SECRET_ACCESS_KEY` | Your Secret access key | + | `AWS_DEFAULT_REGION` | Your region code | 1. You can now use `aws` commands in the `.gitlab-ci.yml` file of this project: @@ -109,7 +109,7 @@ The ECS task definition can be: After you have these prerequisites ready, follow these steps: -1. Make sure your AWS credentials are set up as environment variables for your +1. Make sure your AWS credentials are set up as CI/CD variables for your project. You can follow [the steps above](#run-aws-commands-from-gitlab-cicd) to complete this setup. 1. Add these variables to your project's `.gitlab-ci.yml` file, or in the project's [CI/CD settings](../variables/README.md#create-a-custom-variable-in-the-ui): @@ -141,15 +141,15 @@ After you have these prerequisites ready, follow these steps: ``` You can create your `CI_AWS_ECS_TASK_DEFINITION_FILE` variable as a - [file-typed environment variable](../variables/README.md#custom-environment-variables-of-type-file) instead of a - regular environment variable. If you choose to do so, set the variable value to be the full contents of + [file-typed CI/CD variable](../variables/README.md#custom-cicd-variables-of-type-file) instead of a + regular CI/CD variable. If you choose to do so, set the variable value to be the full contents of the JSON task definition. You can then remove the JSON file from your project. In both cases, make sure that the value for the `containerDefinitions[].name` attribute is the same as the `Container name` defined in your targeted ECS service. WARNING: - `CI_AWS_ECS_TASK_DEFINITION_FILE` takes precedence over `CI_AWS_ECS_TASK_DEFINITION` if both these environment + `CI_AWS_ECS_TASK_DEFINITION_FILE` takes precedence over `CI_AWS_ECS_TASK_DEFINITION` if both these variables are defined within your project. NOTE: @@ -242,7 +242,7 @@ pass three JSON input objects, based on existing templates: have two ways to pass in these JSON objects: - They can be three actual files located in your project. You must specify their path relative to - your project root in your `.gitlab-ci.yml` file, using the following variables. For example, if + your project root in your `.gitlab-ci.yml` file, using the following CI/CD variables. For example, if your files are in a `<project_root>/aws` folder: ```yaml @@ -252,12 +252,12 @@ pass three JSON input objects, based on existing templates: CI_AWS_EC2_DEPLOYMENT_FILE: 'aws/create_deployment.json' ``` - - Alternatively, you can provide these JSON objects as [file-typed environment variables](../variables/README.md#custom-environment-variables-of-type-file). - In your project, go to **Settings > CI / CD > Variables** and add - the three variables listed above as file-typed environment variables. - For each variable, set the value to its corresponding JSON object. + - Alternatively, you can provide these JSON objects as [file-typed CI/CD variables](../variables/README.md#custom-cicd-variables-of-type-file). + In your project, go to **Settings > CI/CD > Variables** and add + the three variables listed above as file-typed CI/CD variables. + For each variable, set the value to its corresponding JSON object. -1. Provide the name of the stack you're creating and/or targeting, as an environment variable: +1. Provide the name of the stack you're creating and/or targeting, as a CI/CD variable: ```yaml variables: @@ -286,7 +286,7 @@ When running your project pipeline at this point: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216008) in GitLab 13.6. To leverage [Auto DevOps](../../topics/autodevops/index.md) for your project when deploying to -AWS EC2, first you must define [your AWS credentials as environment variables](#run-aws-commands-from-gitlab-cicd). +AWS EC2, first you must define [your AWS credentials as CI/CD variables](#run-aws-commands-from-gitlab-cicd). Next, define a job for the `build` stage. To do so, you must reference the `Auto-DevOps.gitlab-ci.yml` template and include a job named `build_artifact` in your @@ -299,7 +299,7 @@ include: - template: Auto-DevOps.gitlab-ci.yml variables: - - AUTO_DEVOPS_PLATFORM_TARGET: EC2 + AUTO_DEVOPS_PLATFORM_TARGET: EC2 build_artifact: stage: build diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index a8ce46b9845..5089aa6c9a5 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -82,11 +82,8 @@ are certain use cases that you may need to work around. For more information: ## Needs Visualization > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](https://about.gitlab.com/handbook/product/#beta). -> - It was deployed behind a feature flag, disabled by default. -> - It became [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36802) in 13.2. > - It became a [standard feature](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38517) in 13.3. -> - It's enabled on GitLab.com. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-needs-visualization). +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52208) in GitLab 13.9. The needs visualization makes it easier to visualize the relationships between dependent jobs in a DAG. This graph displays all the jobs in a pipeline that need or are needed by other jobs. Jobs with no relationships are not displayed in this view. @@ -97,16 +94,3 @@ To see the needs visualization, click on the **Needs** tab when viewing a pipeli Clicking a node highlights all the job paths it depends on.  - -### Enable or disable Needs Visualization **(CORE ONLY)** - -The needs visualization is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it for your instance: - -```ruby -# Instance-wide -Feature.disable(:dag_pipeline_tab) -# or by project -Feature.disable(:dag_pipeline_tab, Project.find(<project ID>)) -``` diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md index 2b97f317fc1..c94d6e3ea80 100644 --- a/doc/ci/docker/README.md +++ b/doc/ci/docker/README.md @@ -1,18 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -comments: false -type: index +redirect_to: 'index.md' --- -# Docker integration +This document was moved to [another location](index.md). -GitLab CI/CD can be combined with [Docker](https://www.docker.com) to enable -integration between the two. - -The following documentation is available for using GitLab CI/CD with Docker: - -- [Using Docker images](using_docker_images.md). -- [Building Docker images with GitLab CI/CD](using_docker_build.md). -- [Building images with kaniko and GitLab CI/CD](using_kaniko.md). +<!-- This redirect file can be deleted after 2021-05-01. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/docker/index.md b/doc/ci/docker/index.md new file mode 100644 index 00000000000..18a9d63b694 --- /dev/null +++ b/doc/ci/docker/index.md @@ -0,0 +1,18 @@ +--- +stage: Verify +group: Continuous Integration +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +type: index +--- + +# Docker integration + +GitLab CI/CD can be combined with [Docker](https://www.docker.com) to enable +integration between the two. + +The following documentation is available for using GitLab CI/CD with Docker: + +- [Building Docker images with GitLab CI/CD](using_docker_build.md). +- [Using Docker images](using_docker_images.md). +- [Building images with kaniko and GitLab CI/CD](using_kaniko.md). diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index af88a006156..46ced9b4d6d 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -126,7 +126,7 @@ not without its own challenges: instance of Docker engine so they don't conflict with each other. But this also means that jobs can be slower because there's no caching of layers. - By default, Docker 17.09 and higher uses `--storage-driver overlay2` which is - the recommended storage driver. See [Using the overlayfs driver](#use-the-overlayfs-driver) + the recommended storage driver. See [Using the OverlayFS driver](#use-the-overlayfs-driver) for details. - Since the `docker:19.03.12-dind` container and the runner container don't share their root file system, the job's working directory can be used as a mount point for @@ -801,7 +801,7 @@ NOTE: The shared runners on GitLab.com use the `overlay2` driver by default. By default, when using `docker:dind`, Docker uses the `vfs` storage driver which -copies the filesystem on every run. This is a disk-intensive operation +copies the file system on every run. This is a disk-intensive operation which can be avoided if a different driver is used, for example `overlay2`. ### Requirements @@ -830,7 +830,7 @@ which can be avoided if a different driver is used, for example `overlay2`. ### Use the OverlayFS driver per project You can enable the driver for each project individually by using the `DOCKER_DRIVER` -environment [variable](../yaml/README.md#variables) in `.gitlab-ci.yml`: +[CI/CD variable](../yaml/README.md#variables) in `.gitlab-ci.yml`: ```yaml variables: diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 630e106b72c..67450d442a9 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -91,7 +91,7 @@ Services inherit the same DNS servers, search domains, and additional hosts as the CI container itself. You can see some widely used services examples in the relevant documentation of -[CI services examples](../services/README.md). +[CI services examples](../services/index.md). ### How services are linked to the job @@ -272,11 +272,11 @@ test: - bundle exec rake spec ``` -## Passing environment variables to services +## Passing CI/CD variables to services -You can also pass custom environment [variables](../variables/README.md) +You can also pass custom CI/CD [variables](../variables/README.md) to fine tune your Docker `images` and `services` directly in the `.gitlab-ci.yml` file. -For more information, read [custom environment variables](../variables/README.md#gitlab-ciyml-defined-variables) +For more information, read about [`.gitlab-ci.yml` defined variables](../variables/README.md#gitlab-ciyml-defined-variables). ```yaml # The following variables are automatically passed down to the Postgres container @@ -528,7 +528,7 @@ To access private container registries, the GitLab Runner process can use: To define which should be used, the GitLab Runner process reads the configuration in the following order: - `DOCKER_AUTH_CONFIG` variable provided as either: - - A [variable](../variables/README.md#gitlab-cicd-environment-variables) in `.gitlab-ci.yml`. + - A [CI/CD variable](../variables/README.md) in `.gitlab-ci.yml`. - A project's variables stored on the projects **Settings > CI/CD** page. - `DOCKER_AUTH_CONFIG` variable provided as environment variable in `config.toml` of the runner. - `config.json` file placed in `$HOME/.docker` directory of the user running GitLab Runner process. @@ -627,7 +627,7 @@ Use one of the following methods to determine the value of `DOCKER_AUTH_CONFIG`: To configure a single job with access for `registry.example.com:5000`, follow these steps: -1. Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the +1. Create a [CI/CD variable](../variables/README.md) `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: ```json @@ -702,7 +702,7 @@ To configure credentials store, follow these steps: 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: - Create a - [variable](../variables/README.md#gitlab-cicd-environment-variables) + [CI/CD variable](../variables/README.md) `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: @@ -734,7 +734,7 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th Make sure that GitLab Runner can access the credentials. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: - - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) + - Create a [CI/CD variable](../variables/README.md) `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: @@ -781,13 +781,13 @@ registries to the `"credHelpers"` hash as described above. Many services accept environment variables, which you can use to change database names or set account names, depending on the environment. -GitLab Runner 0.5.0 and up passes all YAML-defined variables to the created +GitLab Runner 0.5.0 and up passes all YAML-defined CI/CD variables to the created service containers. For all possible configuration variables, check the documentation of each image provided in their corresponding Docker hub page. -All variables are passed to all services containers. It's not +All CI/CD variables are passed to all services containers. It's not designed to distinguish which variable should go where. ### PostgreSQL service example diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 7eb2a8286c7..2122cf2bf16 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -49,7 +49,7 @@ In the following example, kaniko is used to: The job runs only when a tag is pushed. A `config.json` file is created under `/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the -[environment variables](../variables/README.md#predefined-environment-variables) +[predefined CI/CD variables](../variables/README.md#predefined-cicd-variables) GitLab CI/CD provides. In the last step, kaniko uses the `Dockerfile` under the @@ -76,7 +76,7 @@ If you use a custom GitLab Runner behind an http(s) proxy, kaniko needs to be se up accordingly. This means: - Adding the proxy to `/kaniko/.docker/config.json` -- Passing the `http_proxy` environment variables as build args so the Dockerfile +- Passing the `http_proxy` environment variables as build arguments so the Dockerfile instructions can use the proxy when building the image. The previous example can be extended as follows: diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index f59e32fb46d..72fd9833df1 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -13,7 +13,7 @@ To effectively use GitLab CI/CD, you need: of your project. - A [runner](runners/README.md) properly set up. -You can read our [quick start guide](quick_start/README.md) to get you started. +You can read our [quick start guide](quick_start/index.md) to get you started. If you are using an external CI/CD server like Jenkins or Drone CI, it is advised to disable GitLab CI/CD in order to not have any conflicts with the commits status diff --git a/doc/ci/environments.md b/doc/ci/environments.md deleted file mode 100644 index 2ce0618c8e7..00000000000 --- a/doc/ci/environments.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'environments/index.md' ---- - -This document was moved to [another location](environments/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 4dac076ffb7..eecc8ffd18f 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -110,7 +110,7 @@ for an explanation of these roles and the permissions of each. Production secrets are needed to deploy successfully. For example, when deploying to the cloud, cloud providers require these secrets to connect to their services. In the project settings, you can -define and protect environment variables for these secrets. [Protected variables](../variables/README.md#protect-a-custom-variable) +define and protect CI/CD variables for these secrets. [Protected variables](../variables/README.md#protect-a-custom-variable) are only passed to pipelines running on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). The other pipelines don't get the protected variable. You can also @@ -141,7 +141,7 @@ reference a file in another project with a completely different set of permissio In this scenario, the `gitlab-ci.yml` is publicly accessible, but can only be edited by users with appropriate permissions in the other project. -For more information, see [Custom CI configuration path](../pipelines/settings.md#custom-ci-configuration-path). +For more information, see [Custom CI/CD configuration path](../pipelines/settings.md#custom-cicd-configuration-path). ## Troubleshooting diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index 84f8b1b1dbf..ef222ba5779 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -36,7 +36,7 @@ environments are not displayed. To add a project to the dashboard: -1. Click the **Add projects** button in the homescreen of the dashboard. +1. Click the **Add projects** button in the home screen of the dashboard. 1. Search and add one or more projects using the **Search your projects** field. 1. Click the **Add projects** button. @@ -53,4 +53,4 @@ You can add up to 150 projects for GitLab to display on this dashboard. GitLab.com users can add public projects to the Environments Dashboard for free. If your project is private, the group it belongs -to must have a [GitLab Silver](https://about.gitlab.com/pricing/) plan. +to must have a [GitLab Premium](https://about.gitlab.com/pricing/) plan. diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index 15eb4d2c526..39e3dd1ca75 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -117,6 +117,10 @@ available, [demonstrating configuration of timed rollouts](https://gitlab.com/gl ## Blue-Green Deployment +NOTE: +As of GitLab 13.7, teams can leverage an Ingress annotation and [set traffic weight](../../user/project/canary_deployments.md#how-to-change-the-traffic-weight-on-a-canary-ingress) +as an alternative approach to the blue-green deployment strategy documented here. + Also sometimes known as A/B deployment or red-black deployment, this technique is used to reduce downtime and risk during a deployment. When combined with incremental rollouts, you can minimize the impact of a deployment causing an issue. diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 7bf30ef1b95..b49fcd72172 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -57,7 +57,7 @@ Configuring environments involves: The rest of this section illustrates how to configure environments and deployments using an example scenario. It assumes you have already: -- Created a [project](../../gitlab-basics/create-project.md) in GitLab. +- Created a [project](../../user/project/working_with_projects.md#create-a-project) in GitLab. - Set up [a runner](../runners/README.md). In the scenario: @@ -135,12 +135,12 @@ In summary, with the above `.gitlab-ci.yml` we have achieved the following: job deploys our code to a staging server while the deployment is recorded in an environment named `staging`. -#### Environment variables and runners +#### CI/CD variables and runners Starting with GitLab 8.15, the environment name is exposed to the runner in two forms: -- `$CI_ENVIRONMENT_NAME`. The name given in `.gitlab-ci.yml` (with any variables +- `$CI_ENVIRONMENT_NAME`. The name given in `.gitlab-ci.yml` (with any CI/CD variables expanded). - `$CI_ENVIRONMENT_SLUG`. A "cleaned-up" version of the name, suitable for use in URLs, DNS, etc. @@ -221,7 +221,7 @@ The assigned URL for the `review/your-branch-name` environment is [visible in th Note the following: - `stop_review` doesn't generate a dotenv report artifact, so it doesn't recognize the - `DYNAMIC_ENVIRONMENT_URL` variable. Therefore you shouldn't set `environment:url:` in the + `DYNAMIC_ENVIRONMENT_URL` environment variable. Therefore you shouldn't set `environment:url:` in the `stop_review` job. - If the environment URL isn't valid (for example, the URL is malformed), the system doesn't update the environment URL. @@ -313,9 +313,9 @@ Dynamic environments are a fundamental part of [Review apps](../review_apps/inde The `name` and `url` keywords for dynamic environments can use most available CI/CD variables, including: -- [Predefined environment variables](../variables/README.md#predefined-environment-variables) -- [Project and group variables](../variables/README.md#gitlab-cicd-environment-variables) -- [`.gitlab-ci.yml` variables](../yaml/README.md#variables) +- [Predefined CI/CD variables](../variables/README.md#predefined-cicd-variables) +- [Project and group CI/CD variables](../variables/README.md) +- [`.gitlab-ci.yml` CI/CD variables](../yaml/README.md#variables) However, you cannot use variables defined: @@ -327,7 +327,7 @@ For more information, see [Where variables can be used](../variables/where_varia #### Example configuration -Runners expose various [environment variables](../variables/README.md) when a job runs, so +Runners expose various [predefined CI/CD variables](../variables/predefined_variables.md) when a job runs, so you can use them as environment names. In the following example, the job deploys to all branches except `master`: @@ -825,7 +825,7 @@ build with the specified environment runs. Newer deployments can also You may want to specify an environment keyword to [protect builds from unauthorized access](protected_environments.md), or to get -access to [scoped variables](#scoping-environments-with-specs). In these cases, +access to [environment-scoped variables](#scoping-environments-with-specs). In these cases, you can use the `action: prepare` keyword to ensure deployments aren't created, and no builds are canceled: @@ -846,7 +846,7 @@ build: As documented in [Configuring dynamic environments](#configuring-dynamic-environments), you can prepend environment name with a word, followed by a `/`, and finally the branch -name, which is automatically defined by the `CI_COMMIT_REF_NAME` variable. +name, which is automatically defined by the `CI_COMMIT_REF_NAME` predefined CI/CD variable. In short, environments that are named like `type/foo` are all presented under the same group, named `type`. @@ -1009,9 +1009,9 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/* ### Scoping environments with specs > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/2112) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.4. -> - [Scoping for environment variables was moved to Core](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30779) in GitLab 12.2. +> - [Environment scoping for CI/CD variables was moved to all tiers](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30779) in GitLab 12.2. -You can limit the environment scope of a variable by +You can limit the environment scope of a CI/CD variable by defining which environments it can be available for. Wildcards can be used and the default environment scope is `*`. This means that @@ -1025,7 +1025,7 @@ with `review/` would have that particular variable. Some GitLab features can behave differently for each environment. For example, you can -[create a secret variable to be injected only into a production environment](../variables/README.md#limit-the-environment-scopes-of-environment-variables). +[create a secret variable to be injected only into a production environment](../variables/README.md#limit-the-environment-scopes-of-cicd-variables). In most cases, these features use the _environment specs_ mechanism, which offers an efficient way to implement scoping within each environment group. @@ -1061,7 +1061,7 @@ environment's operational health. ## Limitations -In the `environment: name`, you are limited to only the [predefined environment variables](../variables/predefined_variables.md). +In the `environment: name`, you are limited to only the [predefined CI/CD variables](../variables/predefined_variables.md). Re-using variables defined inside `script` as part of the environment name doesn't work. ## Further reading diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 0e4ad1df65f..2636e59723a 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Protected Environments **(PREMIUM)** +# Protected environments **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6303) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.3. @@ -31,7 +31,7 @@ To protect, update, or unprotect an environment, you need to have at least To protect an environment: 1. Navigate to your project's **Settings > CI/CD**. -1. Expand the **Protected Environments** section. +1. Expand the **Protected environments** section. 1. From the **Environment** dropdown menu, select the environment you want to protect. 1. In the **Allowed to Deploy** dropdown menu, select the role, users, or groups you want to give deploy access to. Keep in mind that: diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 9fa0bb080ac..b48dd561a66 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -23,31 +23,35 @@ Examples are available in several forms. As a collection of: The following table lists examples with step-by-step tutorials that are contained in this section: | Use case | Resource | -|:------------------------------|:---------| +|-------------------------------|----------| | Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | | Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | | Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | | GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | | End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | -| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | -| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | | Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | | Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | -| NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | +| npm with semantic-release | [Publish npm packages to the GitLab Package Registry using semantic-release](semantic-release.md). | | PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | -| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | -| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| PHP with npm, SCP | [Running Composer and npm scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | +| PHP with PHPunit, `atoum` | [Testing PHP projects](php.md). | | Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | | Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | -| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | -| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | +| Secrets management with Vault | [Authenticating and Reading Secrets With HashiCorp Vault](authenticating-with-hashicorp-vault/index.md). | + +### Contributed examples -### How to contributing examples +You can help people that use your favorite programming language by submitting a link +to a guide for that language. These contributed guides are hosted externally or in +separate example projects: -Contributions are welcome! You can help your favorite programming -language users and GitLab by sending a merge request with a guide for that language. +| Use case | Resource | +|-------------------------------|----------| +| Game development | [DevOps and Game Development with GitLab CI/CD](https://gitlab.com/gitlab-examples/gitlab-game-demo/). | +| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](https://gitlab.com/gitlab-examples/maven/simple-maven-example). | +| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). | +| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Scala on Heroku | [Test and deploy a Scala application to Heroku](https://gitlab.com/gitlab-examples/scala-sbt). | ## CI/CD templates @@ -81,7 +85,7 @@ choose one of these templates: - [LaTeX (`LaTeX.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/LaTeX.gitlab-ci.yml) - [Maven (`Maven.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Maven.gitlab-ci.yml) - [Mono (`Mono.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Mono.gitlab-ci.yml) -- [NPM (`npm.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/npm.gitlab-ci.yml) +- [npm (`npm.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/npm.gitlab-ci.yml) - [Node.js (`Nodejs.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Nodejs.gitlab-ci.yml) - [OpenShift (`OpenShift.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/OpenShift.gitlab-ci.yml) - [Packer (`Packer.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Packer.gitlab-ci.yml) @@ -95,9 +99,9 @@ choose one of these templates: If a programming language or framework template is not in this list, you can contribute one. To create a template, submit a merge request -to <https://gitlab.com/gitlab-org/gitlab/tree/master/lib/gitlab/ci/templates>. +to [the templates list](https://gitlab.com/gitlab-org/gitlab/tree/master/lib/gitlab/ci/templates). -### Adding templates to your GitLab installation **(PREMIUM ONLY)** +### Adding templates to your GitLab installation **(PREMIUM SELF)** You can add custom examples and templates to your self-managed GitLab instance. Your GitLab administrator can [designate an instance template repository](../../user/admin_area/settings/instance_template_repository.md) @@ -127,7 +131,7 @@ See also the following video overviews: For some customer experiences with GitLab CI/CD, see: -- [How Verizon Connect reduced datacenter deploys from 30 days to under 8 hours with GitLab](https://about.gitlab.com/blog/2019/02/14/verizon-customer-story/) +- [How Verizon Connect reduced data center deploys from 30 days to under 8 hours with GitLab](https://about.gitlab.com/blog/2019/02/14/verizon-customer-story/) - [How Wag! cut their release process from 40 minutes to just 6](https://about.gitlab.com/blog/2019/01/16/wag-labs-blog-post/) - [How Jaguar Land Rover embraced CI to speed up their software lifecycle](https://about.gitlab.com/blog/2018/07/23/chris-hill-devops-enterprise-summit-talk/) diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index c1df21297e3..a1a7de26cf2 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -1,289 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' -author: Fabio Busatto -author_gitlab: bikebilly -type: tutorial -date: 2017-08-15 +redirect_to: '../README.md#contributed-examples' --- -<!-- vale off --> +This document was moved to [another location](../README.md#contributed-examples). -# How to deploy Maven projects to Artifactory with GitLab CI/CD - -## Introduction - -In this article, we show how you can leverage the power of [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) -to build a [Maven](https://maven.apache.org/) project, deploy it to [Artifactory](https://jfrog.com/artifactory/), and then use it from another Maven application as a dependency. - -You'll create two different projects: - -- `simple-maven-dep`: the app built and deployed to Artifactory (see the [simple-maven-dep](https://gitlab.com/gitlab-examples/maven/simple-maven-dep) example project) -- `simple-maven-app`: the app using the previous one as a dependency (see the [simple-maven-app](https://gitlab.com/gitlab-examples/maven/simple-maven-app) example project) - -We assume that you already have a GitLab account on [GitLab.com](https://gitlab.com/), and that you know the basic usage of Git and [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/). -We also assume that an Artifactory instance is available and reachable from the internet, and that you have valid credentials to deploy on it. - -## Create the simple Maven dependency - -First, you need an application to work with: in this specific case we'll use a -simple one, but it could be any Maven application. This will be the dependency -you want to package and deploy to Artifactory, to be available to other -projects. - -### Prepare the dependency application - -For this article you'll use a Maven app that can be cloned from our example -project: - -1. Log in to your GitLab account -1. Create a new project by selecting **Import project from > Repo by URL** -1. Add the following URL: - - ```plaintext - https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git - ``` - -1. Click **Create project** - -This application is nothing more than a basic class with a stub for a JUnit based test suite. -It exposes a method called `hello` that accepts a string as input, and prints a hello message on the screen. - -The project structure is really simple, and you should consider these two resources: - -- `pom.xml`: project object model (POM) configuration file -- `src/main/java/com/example/dep/Dep.java`: source of our application - -### Configure the Artifactory deployment - -The application is ready to use, but you need some additional steps to deploy it to Artifactory: - -1. Log in to Artifactory with your user's credentials. -1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. -1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. -1. Change the `url` value to have it configurable by using variables. -1. Copy the snippet in the `pom.xml` file for your project, just after the - `dependencies` section. The snippet should look like this: - - ```xml - <distributionManagement> - <repository> - <id>central</id> - <name>83d43b5afeb5-releases</name> - <url>${env.MAVEN_REPO_URL}/libs-release-local</url> - </repository> - </distributionManagement> - ``` - -Another step you need to do before you can deploy the dependency to Artifactory -is to configure the authentication data. It is a simple task, but Maven requires -it to stay in a file called `settings.xml` that has to be in the `.m2` subdirectory -in the user's homedir. - -Since you want to use a runner to automatically deploy the application, you -should create the file in the project's home directory and set a command line -parameter in `.gitlab-ci.yml` to use the custom location instead of the default one: - -1. Create a folder called `.m2` in the root of your repository -1. Create a file called `settings.xml` in the `.m2` folder -1. Copy the following content into a `settings.xml` file: - - ```xml - <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" - xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> - <servers> - <server> - <id>central</id> - <username>${env.MAVEN_REPO_USER}</username> - <password>${env.MAVEN_REPO_PASS}</password> - </server> - </servers> - </settings> - ``` - - Username and password will be replaced by the correct values using variables. - -### Configure GitLab CI/CD for `simple-maven-dep` - -Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) to automatically build, test and deploy the dependency! - -GitLab CI/CD uses a file in the root of the repository, named `.gitlab-ci.yml`, to read the definitions for jobs -that will be executed by the configured runners. You can read more about this file in the [GitLab Documentation](../../yaml/README.md). - -First of all, remember to set up variables for your deployment. Navigate to your project's **Settings > CI/CD > Environment variables** page -and add the following ones (replace them with your current values, of course): - -- **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) -- **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) -- **MAVEN_REPO_PASS**: `AKCp2WXr3G61Xjz1PLmYa3arm3yfBozPxSta4taP3SeNu2HPXYa7FhNYosnndFNNgoEds8BCS` (your Artifactory Encrypted Password) - -Now it's time to define jobs in `.gitlab-ci.yml` and push it to the repository: - -```yaml -image: maven:latest - -variables: - MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" - MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" - -cache: - paths: - - .m2/repository/ - - target/ - -build: - stage: build - script: - - mvn $MAVEN_CLI_OPTS compile - -test: - stage: test - script: - - mvn $MAVEN_CLI_OPTS test - -deploy: - stage: deploy - script: - - mvn $MAVEN_CLI_OPTS deploy - only: - - master -``` - -The runner uses the latest [Maven Docker image](https://hub.docker.com/_/maven/), -which contains all of the tools and dependencies needed to manage the project -and to run the jobs. - -Environment variables are set to instruct Maven to use the `homedir` of the repository instead of the user's home when searching for configuration and dependencies. - -Caching the `.m2/repository folder` (where all the Maven files are stored), and the `target` folder (where our application will be created), is useful for speeding up the process -by running all Maven phases in a sequential order, therefore, executing `mvn test` will automatically run `mvn compile` if necessary. - -Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. - -Deploy to Artifactory is done as defined by the variables we have just set up. -The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. - -Done! Now you have all the changes in the GitLab repository, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. -If the deployment has been successful, the deploy job log will output: - -```plaintext -[INFO] ------------------------------------------------------------------------ -[INFO] BUILD SUCCESS -[INFO] ------------------------------------------------------------------------ -[INFO] Total time: 1.983 s -``` - ->**Note**: -the `mvn` command downloads a lot of files from the internet, so you'll see a lot of extra activity in the log the first time you run it. - -Yay! You did it! Checking in Artifactory will confirm that you have a new artifact available in the `libs-release-local` repository. - -## Create the main Maven application - -Now that you have the dependency available on Artifactory, it's time to use it! -Let's see how we can have it as a dependency to our main application. - -### Prepare the main application - -We'll use again a Maven app that can be cloned from our example project: - -1. Create a new project by selecting **Import project from ➔ Repo by URL** -1. Add the following URL: - - ```plaintext - https://gitlab.com/gitlab-examples/maven/simple-maven-app.git - ``` - -1. Click **Create project** - -This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` -file you can see that it imports the `com.example.dep.Dep` class and calls the `hello` method passing `GitLab` as a parameter. - -Since Maven doesn't know how to resolve the dependency, you need to modify the configuration: - -1. Go back to Artifactory -1. Browse the `libs-release-local` repository -1. Select the `simple-maven-dep-1.0.jar` file -1. Find the configuration snippet from the **Dependency Declaration** section of the main panel -1. Copy the snippet in the `dependencies` section of the `pom.xml` file. - The snippet should look like this: - - ```xml - <dependency> - <groupId>com.example.dep</groupId> - <artifactId>simple-maven-dep</artifactId> - <version>1.0</version> - </dependency> - ``` - -### Configure the Artifactory repository location - -At this point you defined the dependency for the application, but you still miss where you can find the required files. -You need to create a `.m2/settings.xml` file as you did for the dependency project, and let Maven know the location using environment variables. - -Here is how you can get the content of the file directly from Artifactory: - -1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel -1. Click on **Generate Maven Settings** -1. Click on **Generate Settings** -1. Copy to clipboard the configuration file -1. Save the file as `.m2/settings.xml` in your repository - -Now you are ready to use the Artifactory repository to resolve dependencies and use `simple-maven-dep` in your main application! - -### Configure GitLab CI/CD for `simple-maven-app` - -You need a last step to have everything in place: configure the `.gitlab-ci.yml` file for this project, as you already did for `simple-maven-dep`. - -You want to leverage [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) to automatically build, test and run your awesome application, -and see if you can get the greeting as expected! - -All you need to do is to add the following `.gitlab-ci.yml` to the repository: - -```yaml -image: maven:latest - -stages: - - build - - test - - run - -variables: - MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" - MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" - -cache: - paths: - - .m2/repository/ - - target/ - -build: - stage: build - script: - - mvn $MAVEN_CLI_OPTS compile - -test: - stage: test - script: - - mvn $MAVEN_CLI_OPTS test - -run: - stage: run - script: - - mvn $MAVEN_CLI_OPTS package - - mvn $MAVEN_CLI_OPTS exec:java -Dexec.mainClass="com.example.app.App" -``` - -It is very similar to the configuration used for `simple-maven-dep`, but instead of the `deploy` job there is a `run` job. -Probably something that you don't want to use in real projects, but here it is useful to see the application executed automatically. - -And that's it! In the `run` job output log you will find a friendly hello to GitLab! - -## Conclusion - -In this article we covered the basic steps to use an Artifactory Maven repository to automatically publish and consume artifacts. - -A similar approach could be used to interact with any other Maven compatible Binary Repository Manager. -Obviously, you can improve these examples, optimizing the `.gitlab-ci.yml` file to better suit your needs, and adapting to your workflow. +<!-- This redirect file can be deleted after 2021-04-18. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index fccc62a4ca0..2d8c92a1a74 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -5,13 +5,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Authenticating and Reading Secrets With Hashicorp Vault +# Authenticating and Reading Secrets With HashiCorp Vault This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD. NOTE: [GitLab Premium](https://about.gitlab.com/pricing/) supports read access to a -Hashicorp Vault, and enables you to +HashiCorp Vault, and enables you to [use Vault secrets in a CI job](../../secrets/index.md#use-vault-secrets-in-a-ci-job). To learn more, read [Using external secrets in CI](../../secrets/index.md). @@ -30,7 +30,7 @@ You must replace the `vault.example.com` URL below with the URL of your Vault se ## How it works -Each job has JSON Web Token (JWT) provided as environment variable named `CI_JOB_JWT`. This JWT can be used to authenticate with Vault using the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt#jwt-authentication) method. +Each job has JSON Web Token (JWT) provided as CI/CD variable named `CI_JOB_JWT`. This JWT can be used to authenticate with Vault using the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt#jwt-authentication) method. The JWT's payload looks like this: @@ -155,11 +155,11 @@ This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_ Combined with [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets. -[token_explicit_max_ttl](https://www.vaultproject.io/api/auth/jwt#token_explicit_max_ttl) specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds. +[`token_explicit_max_ttl`](https://www.vaultproject.io/api/auth/jwt#token_explicit_max_ttl) specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds. -[user_claim](https://www.vaultproject.io/api/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login. +[`user_claim`](https://www.vaultproject.io/api/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login. -[bound_claims_type](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values are interpreted as globs, with `*` matching any number of characters. +[`bound_claims_type`](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values are interpreted as globs, with `*` matching any number of characters. For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). @@ -187,7 +187,7 @@ read_secrets: - echo $CI_COMMIT_REF_NAME # and is this ref protected - echo $CI_COMMIT_REF_PROTECTED - # Vault's address can be provided here or as CI variable + # Vault's address can be provided here or as CI/CD variable - export VAULT_ADDR=http://vault.example.com:8200 # Authenticate and get token. Token expiry time and other properties can be configured # when configuring JWT Auth - https://www.vaultproject.io/api/auth/jwt#parameters-1 @@ -211,7 +211,7 @@ read_secrets: - echo $CI_COMMIT_REF_NAME # and is this ref protected - echo $CI_COMMIT_REF_PROTECTED - # Vault's address can be provided here or as CI variable + # Vault's address can be provided here or as CI/CD variable - export VAULT_ADDR=http://vault.example.com:8200 # Authenticate and get token. Token expiry time and other properties can be configured # when configuring JWT Auth - https://www.vaultproject.io/api/auth/jwt#parameters-1 diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md deleted file mode 100644 index 131a539499d..00000000000 --- a/doc/ci/examples/browser_performance.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/project/merge_requests/browser_performance_testing.md#configuring-browser-performance-testing' ---- - -This document was moved to [another location](../../user/project/merge_requests/browser_performance_testing.md#configuring-browser-performance-testing). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md deleted file mode 100644 index 2b7b2574ef7..00000000000 --- a/doc/ci/examples/code_climate.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'code_quality.md' ---- - -This document was moved to [another location](code_quality.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/code_quality.md b/doc/ci/examples/code_quality.md deleted file mode 100644 index 808ad6d8fef..00000000000 --- a/doc/ci/examples/code_quality.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/project/merge_requests/code_quality.md#example-configuration' ---- - -This document was moved to [another location](../../user/project/merge_requests/code_quality.md#example-configuration). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md deleted file mode 100644 index eecf939d959..00000000000 --- a/doc/ci/examples/container_scanning.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/application_security/container_scanning/index.md' ---- - -This document was moved to [another location](../../user/application_security/container_scanning/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/dast.md b/doc/ci/examples/dast.md deleted file mode 100644 index 97c5cafae95..00000000000 --- a/doc/ci/examples/dast.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/application_security/dast/index.md' ---- - -This document was moved to [another location](../../user/application_security/dast/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/dependency_scanning.md b/doc/ci/examples/dependency_scanning.md deleted file mode 100644 index dc4b7bd764a..00000000000 --- a/doc/ci/examples/dependency_scanning.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/application_security/dependency_scanning/index.md' ---- - -This document was moved to [another location](../../user/application_security/dependency_scanning/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_variables.png b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_variables.png Binary files differdeleted file mode 100644 index e76767741ce..00000000000 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_variables.png +++ /dev/null diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png Binary files differdeleted file mode 100644 index f3761632556..00000000000 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png +++ /dev/null diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md index 9c145677f6e..a1a7de26cf2 100644 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -1,145 +1,8 @@ --- -stage: Release -group: Release -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -author: Dylan Griffith -author_gitlab: DylanGriffith -type: tutorial -date: 2018-06-07 -description: "Continuous Deployment of a Spring Boot application to Cloud Foundry with GitLab CI/CD" +redirect_to: '../README.md#contributed-examples' --- -<!-- vale off --> +This document was moved to [another location](../README.md#contributed-examples). -# Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD - -## Introduction - -This article demonstrates how to use the [Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment) -method to deploy a [Spring Boot](https://projects.spring.io/spring-boot/) application to -[Cloud Foundry (CF)](https://www.cloudfoundry.org/) -with GitLab CI/CD. - -All the code for this project can be found in this [GitLab -repository](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). - -In case you're interested in deploying Spring Boot applications to Kubernetes -using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/blog/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/). - -## Requirements - -This tutorial assumes you are familiar with Java, GitLab, Cloud Foundry, and GitLab CI/CD. - -To follow along, you need: - -- An account on [Pivotal Web Services (PWS)](https://run.pivotal.io/) or any - other Cloud Foundry (CF) instance. -- An account on GitLab. - -NOTE: -If you're not deploying to PWS, you must replace the `api.run.pivotal.io` URL in all the below -commands with the [API URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) -of your CF instance. - -## Create your project - -To create your Spring Boot application you can use the Spring template in -GitLab when creating a new project: - - - -## Configure the deployment to Cloud Foundry - -To deploy to Cloud Foundry you must add a `manifest.yml` file. This -is the configuration for the CF CLI you must use to deploy the application. -Create this in the root directory of your project with the following -content: - -```yaml ---- -applications: - - name: gitlab-hello-world - random-route: true - memory: 1G - path: target/demo-0.0.1-SNAPSHOT.jar -``` - -## Configure GitLab CI/CD to deploy your application - -Now you must add the GitLab CI/CD configuration file -([`.gitlab-ci.yml`](../../yaml/README.md)) -to your project's root. This is how GitLab figures out what commands must run whenever -code is pushed to your repository. Add the following `.gitlab-ci.yml` -file to the root directory of the repository. GitLab detects it -automatically and runs the defined steps once you push your code: - -```yaml -image: java:8 - -stages: - - build - - deploy - -before_script: - - chmod +x mvnw - -build: - stage: build - script: ./mvnw package - artifacts: - paths: - - target/demo-0.0.1-SNAPSHOT.jar - -production: - stage: deploy - script: - - curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx - - ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io - - ./cf push - only: - - master -``` - -This uses the `java:8` [Docker image](../../docker/using_docker_images.md) -to build your application, as it provides the up-to-date Java 8 JDK on [Docker Hub](https://hub.docker.com/). -You also added the [`only` clause](../../yaml/README.md#onlyexcept-basic) -to ensure your deployments only happen when you push to the master branch. - -Because the steps defined in `.gitlab-ci.yml` require credentials to sign in to -CF, you must add your CF credentials as -[environment variables](../../variables/README.md#predefined-environment-variables) -in GitLab CI/CD. To set the environment variables, navigate to your project's -**Settings > CI/CD**, and then expand **Variables**. Name the variables -`CF_USERNAME` and `CF_PASSWORD` and set them to the correct values. - - - -After set up, GitLab CI/CD deploys your app to CF at every push to your -repository's default branch. To review the build logs or watch your builds -running live, navigate to **CI/CD > Pipelines**. - -WARNING: -It's considered best practice for security to create a separate deploy user for -your application and add its credentials to GitLab instead of using a -developer's credentials. - -To start a manual deployment in GitLab go to **CI/CD > Pipelines** then click -**Run Pipeline**. After the app is finished deploying, it displays the -URL of your application in the logs for the `production` job: - -```shell -requested state: started -instances: 1/1 -usage: 1G x 1 instances -urls: gitlab-hello-world-undissembling-hotchpot.cfapps.io -last uploaded: Mon Nov 6 10:02:25 UTC 2017 -stack: cflinuxfs2 -buildpack: client-certificate-mapper=1.2.0_RELEASE container-security-provider=1.8.0_RELEASE java-buildpack=v4.5-offline-https://github.com/cloudfoundry/java-buildpack.git#ffeefb9 java-main java-opts jvmkill-agent=1.10.0_RELEASE open-jdk-like-jre=1.8.0_1... - - state since cpu memory disk details -#0 running 2017-11-06 09:03:22 PM 120.4% 291.9M of 1G 137.6M of 1G -``` - -You can then visit your deployed application (for this example, -`https://gitlab-hello-world-undissembling-hotchpot.cfapps.io/`) and you should -see the "Spring is here!" message. +<!-- This redirect file can be deleted after 2021-04-18. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 958093364af..779ca98084f 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -38,15 +38,15 @@ apt-get install ruby-dev The Dpl provides support for vast number of services, including: Heroku, Cloud Foundry, AWS/S3, and more. To use it simply define provider and any additional parameters required by the provider. -For example if you want to use it to deploy your application to Heroku, you need to specify `heroku` as provider, specify `api-key` and `app`. -All possible parameters can be found here: <https://github.com/travis-ci/dpl#heroku-api>. +For example if you want to use it to deploy your application to Heroku, you need to specify `heroku` as provider, specify `api_key` and `app`. +All possible parameters can be found in the [Heroku API section](https://github.com/travis-ci/dpl#heroku-api). ```yaml staging: stage: deploy script: - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY ``` In the above example we use Dpl to deploy `my-app-staging` to Heroku server with API key stored in `HEROKU_STAGING_API_KEY` secure variable. @@ -67,7 +67,7 @@ staging: - apt-get update -yq - apt-get install -y ruby-dev - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY only: - master ``` @@ -90,7 +90,7 @@ staging: stage: deploy script: - gem install dpl - - dpl --provider=heroku --app=my-app-staging --api-key=$HEROKU_STAGING_API_KEY + - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY only: - master @@ -98,7 +98,7 @@ production: stage: deploy script: - gem install dpl - - dpl --provider=heroku --app=my-app-production --api-key=$HEROKU_PRODUCTION_API_KEY + - dpl --provider=heroku --app=my-app-production --api_key=$HEROKU_PRODUCTION_API_KEY only: - tags ``` diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 6bc96ae6c30..2d7ba2bc759 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -5,9 +5,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD +# Running Composer and npm scripts with deployment via SCP in GitLab CI/CD -This guide covers the building of dependencies of a PHP project while compiling assets via an NPM script using [GitLab CI/CD](../../README.md). +This guide covers the building of dependencies of a PHP project while compiling assets via an npm script using [GitLab CI/CD](../../README.md). While it is possible to create your own image with custom PHP and Node.js versions, for brevity we use an existing [Docker image](https://hub.docker.com/r/tetraweb/php/) that contains both PHP and Node.js installed. diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png Binary files differdeleted file mode 100644 index 09eef98202f..00000000000 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png +++ /dev/null diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png Binary files differdeleted file mode 100644 index 71ffcdea289..00000000000 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png +++ /dev/null diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png Binary files differdeleted file mode 100644 index a9452577a42..00000000000 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png +++ /dev/null diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 298ffff568a..9408c26f06f 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -1,534 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -author: Ryan Hall -author_gitlab: blitzgren -type: tutorial -date: 2018-03-07 +redirect_to: '../README.md#contributed-examples' --- -<!-- vale off --> +This document was moved to [another location](../README.md#contributed-examples). -# DevOps and Game Dev with GitLab CI/CD - -With advances in WebGL and WebSockets, browsers are extremely viable as game development -platforms without the use of plugins like Adobe Flash. Furthermore, by using GitLab and [AWS](https://aws.amazon.com/), -single game developers, as well as game dev teams, can easily host browser-based games online. - -In this tutorial, we'll focus on DevOps, as well as testing and hosting games with Continuous -Integration/Deployment methods using [GitLab CI/CD](../../README.md). We assume you are familiar with GitLab, JavaScript, -and the basics of game development. - -## The game - -Our [demo game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) consists of a simple spaceship traveling in space that shoots by clicking the mouse in a given direction. - -Creating a strong CI/CD pipeline at the beginning of developing another game, [Dark Nova](https://www.darknova.io), -was essential for the fast pace the team worked at. This tutorial will build upon my -[previous introductory article](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) and go through the following steps: - -1. Using code from the previous article to start with a bare-bones [Phaser](https://phaser.io) game built by a gulp file -1. Adding and running unit tests -1. Creating a `Weapon` class that can be triggered to spawn a `Bullet` in a given direction -1. Adding a `Player` class that uses this weapon and moves around the screen -1. Adding the sprites we will use for the `Player` and `Weapon` -1. Testing and deploying with Continuous Integration and Continuous Deployment methods - -By the end, we'll have the core of a [playable game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) -that's tested and deployed on every push to the `master` branch of the [codebase](https://gitlab.com/blitzgren/gitlab-game-demo). -This will also provide -boilerplate code for starting a browser-based game with the following components: - -- Written in [TypeScript](https://www.typescriptlang.org/) and [PhaserJs](https://phaser.io) -- Building, running, and testing with [Gulp](https://gulpjs.com) -- Unit tests with [Chai](https://www.chaijs.com) and [Mocha](https://mochajs.org/) -- CI/CD with GitLab -- Hosting the codebase on GitLab.com -- Hosting the game on AWS -- Deploying to AWS - -## Requirements and setup - -Please refer to my previous article [DevOps and Game Dev](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) to learn the foundational -development tools, running a Hello World-like game, and building this game using GitLab -CI/CD from every new push to master. The `master` branch for this game's [repository](https://gitlab.com/blitzgren/gitlab-game-demo) -contains a completed version with all configurations. If you would like to follow along -with this article, you can clone and work from the `devops-article` branch: - -```shell -git clone git@gitlab.com:blitzgren/gitlab-game-demo.git -git checkout devops-article -``` - -Next, we'll create a small subset of tests that exemplify most of the states I expect -this `Weapon` class to go through. To get started, create a folder called `lib/tests` -and add the following code to a new file `weaponTests.ts`: - -```typescript -import { expect } from 'chai'; -import { Weapon, BulletFactory } from '../lib/weapon'; - -describe('Weapon', () => { - var subject: Weapon; - var shotsFired: number = 0; - // Mocked bullet factory - var bulletFactory: BulletFactory = <BulletFactory>{ - generate: function(px, py, vx, vy, rot) { - shotsFired++; - } - }; - var parent: any = { x: 0, y: 0 }; - - beforeEach(() => { - shotsFired = 0; - subject = new Weapon(bulletFactory, parent, 0.25, 1); - }); - - it('should shoot if not in cooldown', () => { - subject.trigger(true); - subject.update(0.1); - expect(shotsFired).to.equal(1); - }); - - it('should not shoot during cooldown', () => { - subject.trigger(true); - subject.update(0.1); - subject.update(0.1); - expect(shotsFired).to.equal(1); - }); - - it('should shoot after cooldown ends', () => { - subject.trigger(true); - subject.update(0.1); - subject.update(0.3); // longer than timeout - expect(shotsFired).to.equal(2); - }); - - it('should not shoot if not triggered', () => { - subject.update(0.1); - subject.update(0.1); - expect(shotsFired).to.equal(0); - }); -}); -``` - -To build and run these tests using gulp, let's also add the following gulp functions -to the existing `gulpfile.js` file: - -```typescript -gulp.task('build-test', function () { - return gulp.src('src/tests/**/*.ts', { read: false }) - .pipe(tap(function (file) { - // replace file contents with browserify's bundle stream - file.contents = browserify(file.path, { debug: true }) - .plugin(tsify, { project: "./tsconfig.test.json" }) - .bundle(); - })) - .pipe(buffer()) - .pipe(sourcemaps.init({loadMaps: true}) ) - .pipe(gulp.dest('built/tests')); -}); - -gulp.task('run-test', function() { - gulp.src(['./built/tests/**/*.ts']).pipe(mocha()); -}); -``` - -We will start implementing the first part of our game and get these `Weapon` tests to pass. -The `Weapon` class will expose a method to trigger the generation of a bullet at a given -direction and speed. Later we will implement a `Player` class that ties together the user input -to trigger the weapon. In the `src/lib` folder create a `weapon.ts` file. We'll add two classes -to it: `Weapon` and `BulletFactory` which will encapsulate Phaser's **sprite** and -**group** objects, and the logic specific to our game. - -```typescript -export class Weapon { - private isTriggered: boolean = false; - private currentTimer: number = 0; - - constructor(private bulletFactory: BulletFactory, private parent: Phaser.Sprite, private cooldown: number, private bulletSpeed: number) { - } - - public trigger(on: boolean): void { - this.isTriggered = on; - } - - public update(delta: number): void { - this.currentTimer -= delta; - - if (this.isTriggered && this.currentTimer <= 0) { - this.shoot(); - } - } - - private shoot(): void { - // Reset timer - this.currentTimer = this.cooldown; - - // Get velocity direction from player rotation - var parentRotation = this.parent.rotation + Math.PI / 2; - var velx = Math.cos(parentRotation); - var vely = Math.sin(parentRotation); - - // Apply a small forward offset so bullet shoots from head of ship instead of the middle - var posx = this.parent.x - velx * 10 - var posy = this.parent.y - vely * 10; - - this.bulletFactory.generate(posx, posy, -velx * this.bulletSpeed, -vely * this.bulletSpeed, this.parent.rotation); - } -} - -export class BulletFactory { - - constructor(private bullets: Phaser.Group, private poolSize: number) { - // Set all the defaults for this BulletFactory's bullet object - this.bullets.enableBody = true; - this.bullets.physicsBodyType = Phaser.Physics.ARCADE; - this.bullets.createMultiple(30, 'bullet'); - this.bullets.setAll('anchor.x', 0.5); - this.bullets.setAll('anchor.y', 0.5); - this.bullets.setAll('outOfBoundsKill', true); - this.bullets.setAll('checkWorldBounds', true); - } - - public generate(posx: number, posy: number, velx: number, vely: number, rot: number): Phaser.Sprite { - // Pull a bullet from Phaser's Group pool - var bullet = this.bullets.getFirstExists(false); - - // Set the few unique properties about this bullet: rotation, position, and velocity - if (bullet) { - bullet.reset(posx, posy); - bullet.rotation = rot; - bullet.body.velocity.x = velx; - bullet.body.velocity.y = vely; - } - - return bullet; - } -} -``` - -Lastly, we'll redo our entry point, `game.ts`, to tie together both `Player` and `Weapon` objects -as well as add them to the update loop. Here is what the updated `game.ts` file looks like: - -```typescript -import { Player } from "./player"; -import { Weapon, BulletFactory } from "./weapon"; - -window.onload = function() { - var game = new Phaser.Game(800, 600, Phaser.AUTO, 'gameCanvas', { preload: preload, create: create, update: update }); - var player: Player; - var weapon: Weapon; - - // Import all assets prior to loading the game - function preload () { - game.load.image('player', 'assets/player.png'); - game.load.image('bullet', 'assets/bullet.png'); - } - - // Create all entities in the game, after Phaser loads - function create () { - // Create and position the player - var playerSprite = game.add.sprite(400, 550, 'player'); - playerSprite.anchor.setTo(0.5); - player = new Player(game.input, playerSprite, 150); - - var bulletFactory = new BulletFactory(game.add.group(), 30); - weapon = new Weapon(bulletFactory, player.sprite, 0.25, 1000); - - player.loadWeapon(weapon); - } - - // This function is called once every tick, default is 60fps - function update() { - var deltaSeconds = game.time.elapsedMS / 1000; // convert to seconds - player.update(deltaSeconds); - weapon.update(deltaSeconds); - } -} -``` - -Run `gulp serve` and you can run around and shoot. Wonderful! Let's update our CI -pipeline to include running the tests along with the existing build job. - -## Continuous Integration - -To ensure our changes don't break the build and all tests still pass, we use -Continuous Integration (CI) to run these checks automatically for every push. -Read through this article to understand [Continuous Integration, Continuous Delivery, and Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/), -and how these methods are leveraged by GitLab. -From the [last tutorial](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) we already have a `.gitlab-ci.yml` file set up for building our app from -every push. We need to set up a new CI job for testing, which GitLab CI/CD will run after the build job using our generated artifacts from gulp. - -Please read through the [documentation on CI/CD configuration](../../../ci/yaml/README.md) file to explore its contents and adjust it to your needs. - -### Build your game with GitLab CI/CD - -We need to update our build job to ensure tests get run as well. Add `gulp build-test` -to the end of the `script` array for the existing `build` job. After these commands run, -we know we will need to access everything in the `built` folder, given by GitLab CI/CD's `artifacts`. -We'll also cache `node_modules` to avoid having to do a full re-pull of those dependencies: -just pack them up in the cache. Here is the full `build` job: - -```yaml -build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules - artifacts: - paths: - - built -``` - -### Test your game with GitLab CI/CD - -For testing locally, we simply run `gulp run-tests`, which requires gulp to be installed -globally like in the `build` job. We pull `node_modules` from the cache, so the `npm i` -command won't have to do much. In preparation for deployment, we know we will still need -the `built` folder in the artifacts, which will be brought over as default behavior from -the previous job. Lastly, by convention, we let GitLab CI/CD know this needs to be run after -the `build` job by giving it a `test` [stage](../../../ci/yaml/README.md#stages). -Following the YAML structure, the `test` job should look like this: - -```yaml -test: - stage: test - script: - - npm i gulp -g - - npm i - - gulp run-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ -``` - -We have added unit tests for a `Weapon` class that shoots on a specified interval. -The `Player` class implements `Weapon` along with the ability to move around and shoot. Also, -we've added test artifacts and a test stage to our GitLab CI/CD pipeline using `.gitlab-ci.yml`, -allowing us to run our tests by every push. -Our entire `.gitlab-ci.yml` file should now look like this: - -```yaml -image: node:10 - -build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ - -test: - stage: test - script: - - npm i gulp -g - - npm i - - gulp run-test - cache: - policy: pull - paths: - - node_modules/ - artifacts: - paths: - - built/ -``` - -### Run your CI/CD pipeline - -That's it! Add all your new files, commit, and push. For a reference of what our repository should -look like at this point, please refer to the [final commit related to this article on my sample repository](https://gitlab.com/blitzgren/gitlab-game-demo/commit/8b36ef0ecebcf569aeb251be4ee13743337fcfe2). -By applying both build and test stages, GitLab will run them sequentially at every push to -our repository. If all goes well you'll end up with a green check mark on each job for the pipeline: - - - -You can confirm that the tests passed by clicking on the `test` job to enter the full build logs. -Scroll to the bottom and observe, in all its passing glory: - -```shell -$ gulp run-test -[18:37:24] Using gulpfile /builds/blitzgren/gitlab-game-demo/gulpfile.js -[18:37:24] Starting 'run-test'... -[18:37:24] Finished 'run-test' after 21 ms - - - Weapon - ✓ should shoot if not in cooldown - ✓ should not shoot during cooldown - ✓ should shoot after cooldown ends - ✓ should not shoot if not triggered - - - 4 passing (18ms) - -Uploading artifacts... -built/: found 17 matching files -Uploading artifacts to coordinator... ok id=17095874 responseStatus=201 Created token=aaaaaaaa Job succeeded -``` - -## Continuous Deployment - -We have our codebase built and tested on every push. To complete the full pipeline with Continuous Deployment, -let's set up [free web hosting with AWS S3](https://aws.amazon.com/free/) and a job through which our build artifacts get -deployed. GitLab also has a free static site hosting service we can use, [GitLab Pages](https://about.gitlab.com/stages-devops-lifecycle/pages/), -however Dark Nova specifically uses other AWS tools that necessitates using `AWS S3`. -Read through this article that describes [deploying to both S3 and GitLab Pages](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/) -and further delves into the principles of GitLab CI/CD than discussed in this article. - -### Set up S3 Bucket - -1. Log into your AWS account and go to [S3](https://console.aws.amazon.com/s3/home) -1. Click the **Create Bucket** link at the top -1. Enter a name of your choosing and click next -1. Keep the default **Properties** and click next -1. Click the **Manage group permissions** and allow **Read** for the **Everyone** group, click next -1. Create the bucket, and select it in your S3 bucket list -1. On the right side, click **Properties** and enable the **Static website hosting** category -1. Update the radio button to the **Use this bucket to host a website** selection. Fill in `index.html` and `error.html` respectively - -### Set up AWS Secrets - -We need to be able to deploy to AWS with our AWS account credentials, but we certainly -don't want to put secrets into source code. Luckily GitLab provides a solution for this -with [Variables](../../../ci/variables/README.md). This can get complicated -due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't -use root security credentials. Proper IAM credential management is beyond the scope of this -article, but AWS will remind you that using root credentials is unadvised and against their -best practices, as they should. Feel free to follow best practices and use a custom IAM user's -credentials, which will be the same two credentials (Key ID and Secret). It's a good idea to -fully understand [IAM Best Practices in AWS](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html). We need to add these credentials to GitLab: - -1. Log into your AWS account and go to the [Security Credentials page](https://console.aws.amazon.com/iam/home#/security_credential) -1. Click the **Access Keys** section and **Create New Access Key**. Create the key and keep the ID and secret around, you'll need them later - -  - -1. Go to your GitLab project, click **Settings > CI/CD** on the left sidebar -1. Expand the **Variables** section - -  - -1. Add a key named `AWS_KEY_ID` and copy the key ID from Step 2 into the **Value** field -1. Add a key named `AWS_KEY_SECRET` and copy the key secret from Step 2 into the **Value** field - -### Deploy your game with GitLab CI/CD - -To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on -the shared runner. The shared runner also needs to be able to authenticate with your AWS -account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID` -and `AWS_SECRET_ACCESS_KEY`. GitLab CI/CD gives us a way to pass the variables we -set up in the prior section using the `variables` portion of the `deploy` job. At the end, -we add directives to ensure deployment `only` happens on pushes to `master`. This way, every -single branch still runs through CI, and only merging (or committing directly) to master will -trigger the `deploy` job of our pipeline. Put these together to get the following: - -```yaml -deploy: - stage: deploy - variables: - AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" - AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" - script: - - apt-get update - - apt-get install -y python3-dev python3-pip - - easy_install3 -U pip - - pip3 install --upgrade awscli - - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete - only: - - master -``` - -Be sure to update the region and S3 URL in that last script command to fit your setup. -Our final configuration file `.gitlab-ci.yml` looks like: - -```yaml -image: node:10 - -build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ - -test: - stage: test - script: - - npm i gulp -g - - gulp run-test - cache: - policy: pull - paths: - - node_modules/ - artifacts: - paths: - - built/ - -deploy: - stage: deploy - variables: - AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" - AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" - script: - - apt-get update - - apt-get install -y python3-dev python3-pip - - easy_install3 -U pip - - pip3 install --upgrade awscli - - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete - only: - - master -``` - -## Conclusion - -Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you can also find a handful of boilerplate code to get -[TypeScript](https://www.typescriptlang.org/), [Mocha](https://mochajs.org/), [Gulp](https://gulpjs.com/) and [Phaser](https://phaser.io) all playing -together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](https://www.darknova.io). -Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation, -and unit tests, all running and deployed at every push to master - with shockingly little code. -Errors can be easily debugged through GitLab build logs, and within minutes of a successful commit, -you can see the changes live on your game. - -Setting up Continuous Integration and Continuous Deployment from the start with Dark Nova enables -rapid but stable development. We can easily test changes in a separate [environment](../../environments/index.md), -or multiple environments if needed. Balancing and updating a multiplayer game can be ongoing -and tedious, but having faith in a stable deployment with GitLab CI/CD allows -a lot of breathing room in quickly getting changes to players. - -## Further settings - -Here are some ideas to further investigate that can speed up or improve your pipeline: - -- [Yarn](https://yarnpkg.com) instead of npm -- Set up a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) image that can pre-load dependencies and tools (like AWS CLI) -- Forward a [custom domain](https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website -- Combine jobs if you find it unnecessary for a small project -- Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/blog/2016/03/01/gitlab-runner-with-docker/) +<!-- This redirect file can be deleted after 2021-04-19. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 4521c2ed52e..e20e86e8936 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -191,7 +191,7 @@ option as an argument to `npm run confidence-check` on the command line. However, we still need to tell WebdriverIO which browser is available for it to use. [GitLab CI/CD makes -a number of variables available](../../variables/README.md#predefined-environment-variables) +a number of variables available](../../variables/README.md#predefined-cicd-variables) with information about the current CI job. We can use this information to dynamically set up our WebdriverIO configuration according to the job that is running. More specifically, we can tell WebdriverIO what browser to execute the test on depending on the name of the currently running diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index c6ddeefb916..a02a5347734 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -69,7 +69,7 @@ This test will be used later for continuously testing our app with GitLab CI/CD. ### Push to GitLab Since we have our app up and running locally, it's time to push the codebase to our remote repository. -Let's create [a new project](../../../gitlab-basics/create-project.md) in GitLab named `laravel-sample`. +Let's create [a new project](../../../user/project/working_with_projects.md#create-a-project) in GitLab named `laravel-sample`. After that, follow the command line instructions displayed on the project's homepage to initiate the repository on our machine and push the first commit. ```shell @@ -119,8 +119,8 @@ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_rsa ``` -Now, let's add it to your GitLab project as a [variable](../../variables/README.md#gitlab-cicd-environment-variables). -Variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. +Now, let's add it to your GitLab project as a [CI/CD variable](../../variables/README.md). +Project CI/CD variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. They can be added per project by navigating to the project's **Settings** > **CI/CD**. To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. @@ -544,11 +544,11 @@ services: ... ``` -If you wish to test your app with different PHP versions and [database management systems](../../services/README.md), you can define different `image` and `services` keywords for each test job. +If you wish to test your app with different PHP versions and [database management systems](../../services/index.md), you can define different `image` and `services` keywords for each test job. -#### Variables +#### CI/CD variables -GitLab CI/CD allows us to use [environment variables](../../yaml/README.md#variables) in our jobs. +GitLab CI/CD allows us to use [CI/CD variables](../../yaml/README.md#variables) in our jobs. We defined MySQL as our database management system, which comes with a superuser root created by default. So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`. @@ -567,7 +567,7 @@ variables: #### Unit Test as the first job -We defined the required shell scripts as an array of the [script](../../yaml/README.md#script) variable to be executed when running `unit_test` job. +We defined the required shell scripts as an array of the [script](../../yaml/README.md#script) keyword to be executed when running `unit_test` job. These scripts are some Artisan commands to prepare the Laravel, and, at the end of the script, we'll run the tests by `PHPUnit`. @@ -589,7 +589,7 @@ unit_test: #### Deploy to production The job `deploy_production` will deploy the app to the production server. -To deploy our app with Envoy, we had to set up the `$SSH_PRIVATE_KEY` variable as an [SSH private key](../../ssh_keys/README.md#ssh-keys-when-using-the-docker-executor). +To deploy our app with Envoy, we had to set up the `$SSH_PRIVATE_KEY` variable as an [SSH private key](../../ssh_keys/index.md#ssh-keys-when-using-the-docker-executor). If the SSH keys have added successfully, we can run Envoy. As mentioned before, GitLab supports [Continuous Delivery](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) methods as well. diff --git a/doc/ci/examples/license_management.md b/doc/ci/examples/license_management.md deleted file mode 100644 index 46be93c9676..00000000000 --- a/doc/ci/examples/license_management.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/compliance/license_compliance/index.md' ---- - -This document was moved to [another location](../../user/compliance/license_compliance/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 31f0cc76023..53014585f2e 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -66,8 +66,7 @@ docker-php-ext-install pdo_mysql You might wonder what `docker-php-ext-install` is. In short, it is a script provided by the official PHP Docker image that you can use to easily install -extensions. For more information read the documentation at -<https://hub.docker.com/_/php>. +extensions. For more information read [the documentation](https://hub.docker.com/_/php). Now that we created the script that contains all prerequisites for our build environment, let's add it in `.gitlab-ci.yml`: @@ -179,8 +178,8 @@ phpenv config-add my_config.ini *__Important note:__ It seems `phpenv/phpenv` [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork - at [madumlao/phpenv](https://github.com/madumlao/phpenv) that tries to bring - the project back to life. [CHH/phpenv](https://github.com/CHH/phpenv) also + at [`madumlao/phpenv`](https://github.com/madumlao/phpenv) that tries to bring + the project back to life. [`CHH/phpenv`](https://github.com/CHH/phpenv) also seems like a good alternative. Picking any of the mentioned tools works with the basic phpenv commands. Guiding you to choose the right phpenv is out of the scope of this tutorial.* @@ -201,10 +200,10 @@ command once, only to set up the build environment. ## Extend your tests -### Using atoum +### Using `atoum` Instead of PHPUnit, you can use any other tool to run unit tests. For example -you can use [atoum](https://github.com/atoum/atoum): +you can use [`atoum`](https://github.com/atoum/atoum): ```yaml before_script: @@ -242,7 +241,7 @@ before_script: ## Access private packages or dependencies If your test suite needs to access a private repository, you need to configure -the [SSH keys](../ssh_keys/README.md) to be able to clone it. +the [SSH keys](../ssh_keys/index.md) to be able to clone it. ## Use databases or other services @@ -251,7 +250,7 @@ run. If you're using the Docker executor, you can leverage Docker's ability to link to other containers. With GitLab Runner, this can be achieved by defining a `service`. -This functionality is covered in [the CI services](../services/README.md) +This functionality is covered in [the CI services](../services/index.md) documentation. ## Testing things locally diff --git a/doc/ci/examples/sast.md b/doc/ci/examples/sast.md deleted file mode 100644 index ebb3c1f66c1..00000000000 --- a/doc/ci/examples/sast.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/application_security/sast/index.md' ---- - -This document was moved to [another location](../../user/application_security/sast/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/sast_docker.md b/doc/ci/examples/sast_docker.md deleted file mode 100644 index eecf939d959..00000000000 --- a/doc/ci/examples/sast_docker.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/application_security/container_scanning/index.md' ---- - -This document was moved to [another location](../../user/application_security/container_scanning/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 037faaf66a2..c0fc93fe1b3 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -4,9 +4,9 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Publish NPM packages to the GitLab Package Registry using semantic-release +# Publish npm packages to the GitLab Package Registry using semantic-release -This guide demonstrates how to automatically publish NPM packages to the [GitLab Package Registry](../../user/packages/npm_registry/index.md) by using [semantic-release](https://github.com/semantic-release/semantic-release). +This guide demonstrates how to automatically publish npm packages to the [GitLab Package Registry](../../user/packages/npm_registry/index.md) by using [semantic-release](https://github.com/semantic-release/semantic-release). You can also view or fork the complete [example source](https://gitlab.com/gitlab-examples/semantic-release-npm). @@ -15,7 +15,7 @@ You can also view or fork the complete [example source](https://gitlab.com/gitla 1. Open a terminal and navigate to the project's repository 1. Run `npm init`. Name the module according to [the Package Registry's naming conventions](../../user/packages/npm_registry/index.md#package-naming-convention). For example, if the project's path is `gitlab-examples/semantic-release-npm`, name the module `@gitlab-examples/semantic-release-npm`. -1. Install the following NPM packages: +1. Install the following npm packages: ```shell npm install semantic-release @semantic-release/git @semantic-release/gitlab @semantic-release/npm --save-dev @@ -35,7 +35,7 @@ You can also view or fork the complete [example source](https://gitlab.com/gitla } ``` -1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in NPM's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#files). +1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in npm's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#files). 1. Add a `.gitignore` file to the project to avoid committing `node_modules`: @@ -80,13 +80,13 @@ publish: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH ``` -This example configures the pipeline with a single job, `publish`, which runs `semantic-release`. The semantic-release library publishes new versions of the NPM package and creates new GitLab releases (if necessary). +This example configures the pipeline with a single job, `publish`, which runs `semantic-release`. The semantic-release library publishes new versions of the npm package and creates new GitLab releases (if necessary). The default `before_script` generates a temporary `.npmrc` that is used to authenticate to the Package Registry during the `publish` job. -## Set up environment variables +## Set up CI/CD variables -As part of publishing a package, semantic-release increases the version number in `package.json`. For semantic-release to commit this change and push it back to GitLab, the pipeline requires a custom environment variable named `GITLAB_TOKEN`. To create this variable: +As part of publishing a package, semantic-release increases the version number in `package.json`. For semantic-release to commit this change and push it back to GitLab, the pipeline requires a custom CI/CD variable named `GITLAB_TOKEN`. To create this variable: 1. Navigate to **Project > Settings > Access Tokens**. 1. Give the token a name, and select the `api` scope. diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 87291f4e8b8..28d00362309 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -59,7 +59,7 @@ This project has three jobs: ## Store API keys -You'll need to create two variables in **Settings > CI/CD > Environment variables** in your GitLab project: +You'll need to create two variables in **Settings > CI/CD > Variables** in your GitLab project: - `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. - `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 1204a1ae837..5bf0b3d01be 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -50,7 +50,7 @@ This project has three jobs: ## Store API keys -You'll need to create two variables in your project's **Settings > CI/CD > Environment variables** and do not check **Protect variable** and **Mask variable**: +You'll need to create two CI/CD variables in your project's **Settings > CI/CD > Variables** and do not check **Protect variable** or **Mask variable**: - `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. - `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. diff --git a/doc/ci/examples/test-phoenix-application.md b/doc/ci/examples/test-phoenix-application.md deleted file mode 100644 index 52db5740c34..00000000000 --- a/doc/ci/examples/test-phoenix-application.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -redirect_to: '../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md' ---- - -The content of this page was incorporated in [this document](../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md). diff --git a/doc/ci/examples/test-scala-application.md b/doc/ci/examples/test-scala-application.md index 5c499d6a855..057b950289d 100644 --- a/doc/ci/examples/test-scala-application.md +++ b/doc/ci/examples/test-scala-application.md @@ -1,81 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: 'README.md#contributed-examples' --- -# Test and deploy a Scala application to Heroku +This document was moved to [another location](README.md#contributed-examples). -This example demonstrates the integration of GitLab CI/CD with Scala -applications using SBT. You can view or fork the [example project](https://gitlab.com/gitlab-examples/scala-sbt) -and view the logs of its past [CI jobs](https://gitlab.com/gitlab-examples/scala-sbt/-/jobs?scope=finished). - -## Add `.gitlab-ci.yml` file to project - -The following `.gitlab-ci.yml` should be added in the root of your -repository to trigger CI: - -``` yaml -image: openjdk:8 - -stages: - - test - - deploy - -before_script: - - apt-get update -y - - apt-get install apt-transport-https -y - ## Install SBT - - echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list - - apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 - - apt-get update -y - - apt-get install sbt -y - - sbt sbtVersion - -test: - stage: test - script: - - sbt clean coverage test coverageReport - -deploy: - stage: deploy - script: - - apt-get update -yq - - apt-get install rubygems ruby-dev -y - - gem install dpl - - dpl --provider=heroku --app=gitlab-play-sample-app --api-key=$HEROKU_API_KEY -``` - -In the above configuration: - -- The `before_script` installs [SBT](https://www.scala-sbt.org/) and - displays the version that is being used. -- The `test` stage executes SBT to compile and test the project. - - [sbt-scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT - plugin to measure test coverage. -- The `deploy` stage automatically deploys the project to Heroku using dpl. - -You can use other versions of Scala and SBT by defining them in -`build.sbt`. - -## Display test coverage in job - -Add the `Coverage was \[\d+.\d+\%\]` regular expression in the -**Settings > Pipelines > Coverage report** project setting to -retrieve the [test coverage](../pipelines/settings.md#test-coverage-report-badge) -rate from the build trace and have it displayed with your jobs. - -**Pipelines** must be enabled for this option to appear. - -## Heroku application - -A Heroku application is required. You can create one through the -[Dashboard](https://dashboard.heroku.com/). Substitute `gitlab-play-sample-app` -in the `.gitlab-ci.yml` file with your application's name. - -## Heroku API key - -You can look up your Heroku API key in your -[account](https://dashboard.heroku.com/account). Add a [protected variable](../variables/README.md#protect-a-custom-variable) with -this value in **Project ➔ Variables** with key `HEROKU_API_KEY`. +<!-- This redirect file can be deleted after 2021-04-18. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index fb42ed64e78..d9a40c1feb6 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -13,7 +13,7 @@ type: reference > are encouraged to upgrade your GitLab instance if you haven't done already. > If you are **not** using GitLab 8.12 or higher, you would need to work your way > around submodules in order to access the sources of e.g., `gitlab.com/group/project` -> with the use of [SSH keys](ssh_keys/README.md). +> with the use of [SSH keys](ssh_keys/index.md). > - With GitLab 8.12 onward, your permissions are used to evaluate what a CI job > can access. More information about how this system works can be found in the > [Jobs permissions model](../user/permissions.md#job-permissions). diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index c04bcd6f549..307dcdf258c 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -6,162 +6,114 @@ description: "An overview of Continuous Integration, Continuous Delivery, and Co type: concepts --- -# Introduction to CI/CD with GitLab +# CI/CD concepts **(FREE)** -This document presents an overview of the concepts of Continuous Integration, -Continuous Delivery, and Continuous Deployment, as well as an introduction to -GitLab CI/CD. +With the continuous method of software development, you continuously build, +test, and deploy iterative code changes. This iterative process helps reduce +the chance that you develop new code based on buggy or failed previous versions. +With this method, you strive to have less human intervention or even no intervention at all, +from the development of new code until its deployment. + +The three primary approaches for the continuous method are: + +- [Continuous Integration](#continuous-integration) +- [Continuous Delivery](#continuous-delivery) +- [Continuous Deployment](#continuous-deployment) NOTE: Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more. Watch our ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) -webcast to learn about continuous methods and how the GitLab built-in CI can help you simplify and scale software development. +webcast to learn about continuous methods and how built-in GitLab CI/CD can help you simplify and scale software development. -> For some additional information about GitLab CI/CD: -> -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch the [CI/CD Ease of configuration](https://www.youtube.com/embed/opdLqwz6tcE) video. -> - Watch the [Making the case for CI/CD in your organization](https://about.gitlab.com/compare/github-actions-alternative/) -> webcast to learn the benefits of CI/CD and how to measure the results of CI/CD automation. +> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Learn how to [configure CI/CD](https://www.youtube.com/embed/opdLqwz6tcE). +> - [Make the case for CI/CD in your organization](https://about.gitlab.com/compare/github-actions-alternative/). > - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Learn how [Verizon reduced rebuilds](https://about.gitlab.com/blog/2019/02/14/verizon-customer-story/) > from 30 days to under 8 hours with GitLab. -## Introduction to CI/CD methodologies - -The continuous methodologies of software development are based on -automating the execution of scripts to minimize the chance of -introducing errors while developing applications. They require -less human intervention or even no intervention at all, from the -development of new code until its deployment. - -It involves continuously building, testing, and deploying code -changes at every small iteration, reducing the chance of developing -new code based on bugged or failed previous versions. - -There are three main approaches to this methodology, each of them -to be applied according to what best suits your strategy. - -### Continuous Integration +## Continuous Integration Consider an application that has its code stored in a Git repository in GitLab. Developers push code changes every day, multiple times a day. For every push to the repository, you can create a set of scripts to build and test your application -automatically, decreasing the chance of introducing errors to your app. +automatically. These scripts help decrease the chances that you introduce errors in your application. -This practice is known as [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration); -for every change submitted to an application - even to development branches - -it's built and tested automatically and continuously, ensuring the -introduced changes pass all tests, guidelines, and code compliance -standards you established for your app. +This practice is known as [Continuous Integration](https://en.wikipedia.org/wiki/Continuous_integration). +Each change submitted to an application, even to development branches, +is built and tested automatically and continuously. These tests ensure the +changes pass all tests, guidelines, and code compliance +standards you established for your application. -[GitLab itself](https://gitlab.com/gitlab-org/gitlab-foss) is an -example of using Continuous Integration as a software -development method. For every push to the project, there's a set -of scripts the code is checked against. +[GitLab itself](https://gitlab.com/gitlab-org/gitlab) is an +example of a project that uses Continuous Integration as a software +development method. For every push to the project, a set +of checks run against the code. -### Continuous Delivery +## Continuous Delivery [Continuous Delivery](https://continuousdelivery.com/) is a step -beyond Continuous Integration. Your application is not only -built and tested at every code change pushed to the codebase, -but, as an additional step, it's also deployed continuously, though -the deployments are triggered manually. +beyond Continuous Integration. Not only is your application +built and tested each time a code change is pushed to the codebase, +the application is also deployed continuously. However, with continuous +delivery, you trigger the deployments manually. -This method ensures the code is checked automatically but requires +Continuous Delivery checks the code automatically, but it requires human intervention to manually and strategically trigger the deployment of the changes. -### Continuous Deployment +## Continuous Deployment [Continuous Deployment](https://www.airpair.com/continuous-deployment/posts/continuous-deployment-for-practical-people) -is also a further step beyond Continuous Integration, similar to +is another step beyond Continuous Integration, similar to Continuous Delivery. The difference is that instead of deploying your -application manually, you set it to be deployed automatically. It does -not require human intervention at all to have your application -deployed. +application manually, you set it to be deployed automatically. +Human intervention is not required. -## Introduction to GitLab CI/CD +## GitLab CI/CD -GitLab CI/CD is a powerful tool built into GitLab that allows you -to apply all the continuous methods (Continuous Integration, -Delivery, and Deployment) to your software with no third-party -application or integration needed. +[GitLab CI/CD](../quick_start/index.md) is the part of GitLab that you use +for all of the continuous methods (Continuous Integration, +Delivery, and Deployment). With GitLab CI/CD, you can test, build, +and publish your software with no third-party application or integration needed. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Introduction to GitLab CI](https://www.youtube.com/watch?v=l5705U8s_nQ&t=397) from a recent GitLab meetup. +For an overview, see [Introduction to GitLab CI/CD](https://www.youtube.com/watch?v=l5705U8s_nQ&t=397) from an April 2020 GitLab meetup. -### Basic CI/CD workflow +### GitLab CI/CD workflow -Consider the following example for how GitLab CI/CD fits in a -common development workflow. +GitLab CI/CD fits in a common development workflow. -Assume that you have discussed a code implementation in an issue -and worked locally on your proposed changes. After you push your -commits to a feature branch in a remote repository in GitLab, -the CI/CD pipeline set for your project is triggered. By doing -so, GitLab CI/CD: +You can start by discussing a code implementation in an issue +and working locally on your proposed changes. Then you can push your +commits to a feature branch in a remote repository that's hosted in GitLab. +The push triggers the CI/CD pipeline for your project. Then, GitLab CI/CD: - Runs automated scripts (sequentially or in parallel) to: - - Build and test your app. - - Preview the changes per merge request with Review Apps, as you - would see in your `localhost`. + - Build and test your application. + - Preview the changes in a Review App, the same as you + would see on your `localhost`. -After you're happy with your implementation: +After the implementation works as expected: - Get your code reviewed and approved. - Merge the feature branch into the default branch. - GitLab CI/CD deploys your changes automatically to a production environment. -- And finally, you and your team can easily roll it back if something goes wrong. + +If something goes wrong, you can roll back your changes.  -GitLab CI/CD is capable of doing a lot more, but this workflow -exemplifies the ability of GitLab to track the entire process, -without the need for an external tool to deliver your software. -And, most usefully, you can visualize all the steps through -the GitLab UI. +This workflow shows the major steps in the GitLab process. +You don't need any external tools to deliver your software and +you can visualize all the steps in the GitLab UI. -#### A deeper look into the CI/CD basic workflow +### A deeper look into the CI/CD workflow -If we take a deeper look into the basic workflow, we can see +If you look deeper into the workflow, you can see the features available in GitLab at each stage of the DevOps -lifecycle, as shown in the illustration below. +lifecycle.  -If you look at the image from the left to the right, -you can see some of the features available in GitLab -according to each stage (Verify, Package, Release). - -1. **Verify**: - - Automatically build and test your application with Continuous Integration. - - Analyze your source code quality with [GitLab Code Quality](../../user/project/merge_requests/code_quality.md). - - Determine the browser performance impact of code changes with [Browser Performance Testing](../../user/project/merge_requests/browser_performance_testing.md). **(PREMIUM)** - - Determine the server performance impact of code changes with [Load Performance Testing](../../user/project/merge_requests/load_performance_testing.md). **(PREMIUM)** - - Perform a series of tests, such as [Container Scanning](../../user/application_security/container_scanning/index.md) **(ULTIMATE)**, [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) **(ULTIMATE)**, and [Unit tests](../unit_test_reports.md). - - Deploy your changes with [Review Apps](../review_apps/index.md) to preview the app changes on every branch. -1. **Package**: - - Store Docker images with the [Container Registry](../../user/packages/container_registry/index.md). - - Store packages with the [Package Registry](../../user/packages/package_registry/index.md). -1. **Release**: - - Continuous Deployment, automatically deploying your app to production. - - Continuous Delivery, manually click to deploy your app to production. - - Deploy static websites with [GitLab Pages](../../user/project/pages/index.md). - - Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature with [Canary Deployments](../../user/project/canary_deployments.md). - - Deploy your features behind [Feature Flags](../../operations/feature_flags.md). - - Add release notes to any Git tag with [GitLab Releases](../../user/project/releases/index.md). - - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). - - Deploy your application to a production environment in a Kubernetes cluster with [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy). - -With GitLab CI/CD you can also: - -- Easily set up your app's entire lifecycle with [Auto DevOps](../../topics/autodevops/index.md). -- Deploy your app to different [environments](../environments/index.md). -- Install your own [GitLab Runner](https://docs.gitlab.com/runner/). -- [Schedule pipelines](../pipelines/schedules.md). -- Check for app vulnerabilities with [Security Test reports](../../user/application_security/index.md). **(ULTIMATE)** - -To see all CI/CD features, navigate back to the [CI/CD index](../README.md). - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -Watch the video [GitLab CI Live Demo](https://youtu.be/l5705U8s_nQ?t=369) with a deeper overview of GitLab CI/CD. +[Get a deeper look at GitLab CI/CD](https://youtu.be/l5705U8s_nQ?t=369). diff --git a/doc/ci/jenkins/index.md b/doc/ci/jenkins/index.md deleted file mode 100644 index 3f720ef959e..00000000000 --- a/doc/ci/jenkins/index.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../migration/jenkins.md' ---- - -This document was moved to [another location](../migration/jenkins.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index c985a5b00ef..d1fe6db3ee4 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -133,12 +133,29 @@ In the pipeline, the result is a group named `build ruby` with three jobs:  -The jobs are be ordered by comparing the numbers from left to right. You +The jobs are ordered by comparing the numbers from left to right. You usually want the first number to be the index and the second number to be the total. [This regular expression](https://gitlab.com/gitlab-org/gitlab/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99) evaluates the job names: `\d+[\s:\/\\]+\d+\s*`. +### Improved job grouping + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52644) in GitLab 13.9. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It's enabled on GitLab.com. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](../../administration/feature_flags.md). **(FREE SELF)** + +Job grouping is evaluated with an improved regular expression to group jobs by name: + +- `([\b\s:]+((\[.*\])|(\d+[\s:\/\\]+\d+)))+\s*\z`. + +The new implementation removes one or more `: [...]`, `X Y`, `X/Y`, or `X\Y` sequences +from the **end** of job names only. + +Matching substrings occuring at the beginning or in the middle of build names are +no longer removed. + ## Specifying variables when running manual jobs > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30485) in GitLab 12.2. @@ -150,9 +167,9 @@ additional variables. To access this page, click on the **name** of the manual j the pipeline view, *not* the play (**{play}**) button. This is useful when you want to alter the execution of a job that uses -[custom environment variables](../variables/README.md#custom-environment-variables). +[custom CI/CD variables](../variables/README.md#custom-cicd-variables). Add a variable name (key) and value here to override the value defined in -[the UI or `.gitlab-ci.yml`](../variables/README.md#custom-environment-variables), +[the UI or `.gitlab-ci.yml`](../variables/README.md#custom-cicd-variables), for a single run of the manual job.  @@ -196,8 +213,8 @@ You can create [collapsible sections in job logs](#expand-and-collapse-job-log-s by manually outputting special codes that GitLab uses to determine what sections to collapse: -- Section start marker: `section_start:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` + `TEXT_OF_SECTION_HEADER` -- Section end marker: `section_end:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` +- Section start marker: `\e[0Ksection_start:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` + `TEXT_OF_SECTION_HEADER` +- Section end marker: `\e[0Ksection_end:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` You must add these codes to the script section of the CI configuration. For example, using `echo`: @@ -205,9 +222,9 @@ using `echo`: ```yaml job1: script: - - echo -e "section_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section" + - echo -e "\e[0Ksection_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section" - echo 'this line should be hidden when collapsed' - - echo -e "section_end:`date +%s`:my_first_section\r\e[0K" + - echo -e "\e[0Ksection_end:`date +%s`:my_first_section\r\e[0K" ``` In the example above: @@ -223,9 +240,9 @@ In the example above: Sample raw job log: ```plaintext -section_start:1560896352:my_first_section\r\e[0KHeader of the 1st collapsible section +\e[0Ksection_start:1560896352:my_first_section\r\e[0KHeader of the 1st collapsible section this line should be hidden when collapsed -section_end:1560896353:my_first_section\r\e[0K +\e[0Ksection_end:1560896353:my_first_section\r\e[0K ``` ### Pre-collapse sections @@ -236,7 +253,7 @@ You can make the job log automatically collapse collapsible sections by adding t Add `[collapsed=true]` after the section name and before the `\r`. The section end marker remains unchanged: -- Section start marker with `[collapsed=true]`: `section_start:UNIX_TIMESTAMP:SECTION_NAME[collapsed=true]\r\e[0K` + `TEXT_OF_SECTION_HEADER` +- Section start marker with `[collapsed=true]`: `\e[0Ksection_start:UNIX_TIMESTAMP:SECTION_NAME[collapsed=true]\r\e[0K` + `TEXT_OF_SECTION_HEADER` - Section end marker: `section_end:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` Add the updated section start text to the CI configuration. For example, @@ -245,7 +262,7 @@ using `echo`: ```yaml job1: script: - - echo -e "section_start:`date +%s`:my_first_section[collapsed=true]\r\e[0KHeader of the 1st collapsible section" + - echo -e "\e[0Ksection_start:`date +%s`:my_first_section[collapsed=true]\r\e[0KHeader of the 1st collapsible section" - echo 'this line should be hidden automatically after loading the job log' - - echo -e "section_end:`date +%s`:my_first_section\r\e[0K" + - echo -e "\e[0Ksection_end:`date +%s`:my_first_section\r\e[0K" ``` diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md deleted file mode 100644 index 2ae17df0933..00000000000 --- a/doc/ci/junit_test_reports.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'unit_test_reports.md' ---- - -This document was moved to [unit_test_reports](unit_test_reports.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index 35b4a2de8f8..aff18b0889f 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -114,6 +114,11 @@ For example, if your project contains a large number of tags that your CI jobs d you could add [`--no-tags`](https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---no-tags) to the extra flags to make your fetches faster and more compact. +Also in the case where you repository does _not_ contain a lot of +tags, `--no-tags` can [make a big difference in some +cases](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/746). +If your CI builds do not depend on Git tags it is worth trying. + See the [`GIT_FETCH_EXTRA_FLAGS` documentation](../runners/README.md#git-fetch-extra-flags) for more information. diff --git a/doc/ci/lint.md b/doc/ci/lint.md index ec3c128f076..3888a750d6a 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -1,11 +1,13 @@ --- -stage: none -group: unassigned +stage: Verify +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- <!-- markdownlint-disable MD044 --> +<!-- vale gitlab.Spelling = NO --> # Validate .gitlab-ci.yml syntax with the CI Lint tool <!-- markdownlint-enable MD044 --> +<!-- vale gitlab.Spelling = YES --> If you want to test the validity of your GitLab CI/CD configuration before committing the changes, you can use the CI Lint tool. This tool checks for syntax and logical diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index 999d15eac24..5c3cf6ec02e 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -166,9 +166,10 @@ Read the [documentation on Pipelines for Merged Results](pipelines_for_merged_re Read the [documentation on Merge Trains](pipelines_for_merged_results/merge_trains/index.md). -## Run pipelines in the parent project for merge requests from a forked project **(STARTER)** +## Run pipelines in the parent project for merge requests from a forked project **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217451) in GitLab 13.3. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217451) in GitLab 13.3. +> - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. By default, external contributors working from forks can't create pipelines in the parent project. When a pipeline for merge requests is triggered by a merge request diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index e5b9ad030d0..edfedbc2527 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -84,9 +84,10 @@ To enable merge trains for your project: 1. If you are on a self-managed GitLab instance, ensure the [feature flag](#merge-trains-feature-flag) is set correctly. 1. [Configure your CI/CD configuration file](../../index.md#configuring-pipelines-for-merge-requests) so that the pipeline or individual jobs run for merge requests. -1. Visit your project's **Settings > General** and expand **Merge requests** -1. Check **Enable merged results pipelines.** (if not enabled) -1. Check **Enable merge trains.** +1. Visit your project's **Settings > General** and expand **Merge requests**. +1. In the **Merge method** section, verify that **Merge commit** is selected. + You cannot use **Merge commit with semi-linear history** or **Fast-forward merge** with merge trains. +1. In the **Merge options** section, select **Enable merged results pipelines.** (if not already selected) and **Enable merge trains.** 1. Click **Save changes** In GitLab 13.5 and earlier, there is only one checkbox, named @@ -209,7 +210,7 @@ Trains, create a new pipeline for merged results when this error occurs: See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/35135) for more information. -### Merge Trains feature flag **(PREMIUM ONLY)** +### Merge Trains feature flag **(PREMIUM SELF)** In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/244831), you can [enable or disable merge trains in the project settings](#enable-merge-trains). diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index 5d217466f5c..0a44232efd3 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -15,7 +15,7 @@ comparison to see what's different. We have collected several resources that you may find useful before starting to migrate. -The [Quick Start Guide](../quick_start/README.md) is a good overview of how GitLab CI/CD works. You may also be interested in [Auto DevOps](../../topics/autodevops/index.md) which can be used to build, test, and deploy your applications with little to no configuration needed at all. +The [Quick Start Guide](../quick_start/index.md) is a good overview of how GitLab CI/CD works. You may also be interested in [Auto DevOps](../../topics/autodevops/index.md) which can be used to build, test, and deploy your applications with little to no configuration needed at all. For advanced CI/CD teams, [custom project templates](../../user/admin_area/custom_project_templates.md) can enable the reuse of pipeline configurations. @@ -265,14 +265,14 @@ test_async: ## Contexts and variables -CircleCI provides [Contexts](https://circleci.com/docs/2.0/contexts/) to securely pass environment variables across project pipelines. In GitLab, a [Group](../../user/group/index.md) can be created to assemble related projects together. At the group level, [variables](../variables/README.md#group-level-environment-variables) can be stored outside the individual projects, and securely passed into pipelines across multiple projects. +CircleCI provides [Contexts](https://circleci.com/docs/2.0/contexts/) to securely pass environment variables across project pipelines. In GitLab, a [Group](../../user/group/index.md) can be created to assemble related projects together. At the group level, [CI/CD variables](../variables/README.md#group-level-cicd-variables) can be stored outside the individual projects, and securely passed into pipelines across multiple projects. ## Orbs There are two GitLab issues open addressing CircleCI Orbs and how GitLab can achieve similar functionality. -- <https://gitlab.com/gitlab-com/Product/-/issues/1151> -- <https://gitlab.com/gitlab-org/gitlab/-/issues/195173> +- [issue #1151](https://gitlab.com/gitlab-com/Product/-/issues/1151) +- [issue #195173](https://gitlab.com/gitlab-org/gitlab/-/issues/195173) ## Build environments diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index 71d5e6eb859..1424868401e 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -15,11 +15,11 @@ before diving in. Think of this page as a "GitLab CI/CD for Jenkins Users" guide The following list of recommended steps was created after observing organizations that were able to quickly complete this migration: -1. Start by reading the GitLab CI/CD [Quick Start Guide](../quick_start/README.md) and [important product differences](#important-product-differences). +1. Start by reading the GitLab CI/CD [Quick Start Guide](../quick_start/index.md) and [important product differences](#important-product-differences). 1. Learn the importance of [managing the organizational transition](#managing-the-organizational-transition). 1. [Add runners](../runners/README.md) to your GitLab instance. 1. Educate and enable your developers to independently perform the following steps in their projects: - 1. Review the [Quick Start Guide](../quick_start/README.md) and [Pipeline Configuration Reference](../yaml/README.md). + 1. Review the [Quick Start Guide](../quick_start/index.md) and [Pipeline Configuration Reference](../yaml/README.md). 1. Use the [Jenkins Wrapper](#jenkinsfile-wrapper) to temporarily maintain fragile Jenkins jobs. 1. Migrate the build and CI jobs and configure them to show results directly in your merge requests. They can use [Auto DevOps](../../topics/autodevops/index.md) as a starting point, and [customize](../../topics/autodevops/customize.md) or [decompose](../../topics/autodevops/customize.md#using-components-of-auto-devops) the configuration as needed. 1. Add [Review Apps](../review_apps/index.md). @@ -79,7 +79,7 @@ There are some high level differences between the products worth mentioning: - from the [GitLab UI](../pipelines/index.md#run-a-pipeline-manually) - by [API call](../triggers/README.md) - by [webhook](../triggers/README.md#triggering-a-pipeline-from-a-webhook) - - by [ChatOps](../chatops/README.md) + - by [ChatOps](../chatops/index.md) - You can control which jobs run in which cases, depending on how they are triggered, with the [`rules` syntax](../yaml/README.md#rules). @@ -188,7 +188,7 @@ pdf: expire_in: 1 week ``` -Additionally, we have package management features like a built-in container, NPM, and Maven registry that you +Additionally, we have package management features like built-in container and package registries that you can leverage. You can see the complete list of packaging features in the [Packages & Registries](../../user/packages/index.md) documentation. @@ -209,7 +209,7 @@ as well as encourage inner sourcing. In self-managed GitLab instances, you can build an [Instance Template Repository](../../user/admin_area/settings/instance_template_repository.md). Development teams across the whole organization can select templates from a dropdown menu. -A group administrator is able to set a group to use as the source for the +A group maintainer or a group owner is able to set a group to use as the source for the [custom project templates](../../user/admin_area/custom_project_templates.md), which can be used by all projects in the group. An instance administrator can set a group as the source for [instance project templates](../../user/group/custom_project_templates.md), diff --git a/doc/ci/multi_project_pipeline_graphs.md b/doc/ci/multi_project_pipeline_graphs.md deleted file mode 100644 index 66efa0a7c35..00000000000 --- a/doc/ci/multi_project_pipeline_graphs.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'multi_project_pipelines.md' ---- - -This document was moved to [another location](multi_project_pipelines.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 006d6bda0e0..4c186b8a64e 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -68,7 +68,7 @@ outbound connections for upstream and downstream pipeline dependencies. When using: -- Variables or [`rules`](yaml/README.md#rulesif) to control job behavior, the value of +- CI/CD Variables or [`rules`](yaml/README.md#rulesif) to control job behavior, the value of the [`$CI_PIPELINE_SOURCE` predefined variable](variables/predefined_variables.md) is `pipeline` for multi-project pipeline triggered through the API with `CI_JOB_TOKEN`. - [`only/except`](yaml/README.md#onlyexcept-basic) to control job behavior, use the @@ -114,7 +114,7 @@ the `staging` job is marked as _failed_. When using: -- Variables or [`rules`](yaml/README.md#rulesif) to control job behavior, the value of +- CI/CD variables or [`rules`](yaml/README.md#rulesif) to control job behavior, the value of the [`$CI_PIPELINE_SOURCE` predefined variable](variables/predefined_variables.md) is `pipeline` for multi-project pipelines triggered with a bridge job (using the [`trigger:`](yaml/README.md#trigger) keyword). @@ -162,11 +162,11 @@ of the user that ran the trigger job in the upstream project. If the user does n have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See [pipeline security for protected branches](pipelines/index.md#pipeline-security-on-protected-branches). -### Passing variables to a downstream pipeline +### Passing CI/CD variables to a downstream pipeline #### With the `variables` keyword -Sometimes you might want to pass variables to a downstream pipeline. +Sometimes you might want to pass CI/CD variables to a downstream pipeline. You can do that using the `variables` keyword, just like you would when defining a regular job. @@ -183,7 +183,7 @@ staging: ``` The `ENVIRONMENT` variable is passed to every job defined in a downstream -pipeline. It is available as an environment variable when GitLab Runner picks a job. +pipeline. It is available as a variable when GitLab Runner picks a job. In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` @@ -220,7 +220,7 @@ the ones defined in the upstream project take precedence. #### With variable inheritance -You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](variables/README.md#inherit-environment-variables) and [cross project artifact downloads](yaml/README.md#cross-project-artifact-downloads-with-needs). +You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](variables/README.md#inherit-cicd-variables) and [cross project artifact downloads](yaml/README.md#cross-project-artifact-downloads-with-needs). In the upstream pipeline: @@ -261,7 +261,7 @@ test: ### Mirroring status from triggered pipeline > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11238) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Core in 12.8. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. You can mirror the pipeline status from the triggered pipeline to the source bridge job by using `strategy: depend`. For example: diff --git a/doc/ci/parent_child_pipelines.md b/doc/ci/parent_child_pipelines.md index d088d0d7e4d..bc1c02bc48c 100644 --- a/doc/ci/parent_child_pipelines.md +++ b/doc/ci/parent_child_pipelines.md @@ -156,7 +156,11 @@ build a matrix of targets and architectures. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM). -We also have an [example project using Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. You could use a similar process for other templating languages like [Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/). +<!-- vale gitlab.Spelling = NO --> +We also have an example project using +[Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) +which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. You could use a similar process for other templating languages like [Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/). +<!-- vale gitlab.Spelling = NO --> The artifact path is parsed by GitLab, not the runner, so the path must match the syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows @@ -176,7 +180,10 @@ pipelines, which was later increased to two. A parent pipeline can trigger many pipelines, and these child pipelines can trigger their own child pipelines. It's not possible to trigger another level of child pipelines. -## Pass variables to a child pipeline +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Nested Dynamic Pipelines](https://youtu.be/C5j3ju9je2M). + +## Pass CI/CD variables to a child pipeline -You can [pass variables to a downstream pipeline](multi_project_pipelines.md#passing-variables-to-a-downstream-pipeline) +You can [pass CI/CD variables to a downstream pipeline](multi_project_pipelines.md#passing-cicd-variables-to-a-downstream-pipeline) the same way as for multi-project pipelines. diff --git a/doc/ci/permissions/README.md b/doc/ci/permissions/README.md deleted file mode 100644 index 897d7b6ce51..00000000000 --- a/doc/ci/permissions/README.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: '../../user/permissions.md#gitlab-cicd-permissions' ---- - -This document was moved to [user/permissions.md](../../user/permissions.md#gitlab-cicd-permissions). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipeline_editor/img/pipeline_editor_commit_v13_8.png b/doc/ci/pipeline_editor/img/pipeline_editor_commit_v13_8.png Binary files differindex cc1f666f319..fcafc984ce4 100644 --- a/doc/ci/pipeline_editor/img/pipeline_editor_commit_v13_8.png +++ b/doc/ci/pipeline_editor/img/pipeline_editor_commit_v13_8.png diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index 61b8e289509..9f4a1afe2f9 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -5,13 +5,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Pipeline Editor **(CORE)** +# Pipeline Editor **(FREE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4540) in GitLab 13.8. > - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. > - It's enabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-pipeline-editor). **(CORE ONLY)** +> - It's recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-pipeline-editor). **(FREE SELF)** WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -25,11 +25,12 @@ From the pipeline editor page you can: - Do a deeper [lint](#lint-ci-configuration) of your configuration, that verifies it with any configuration added with the [`include`](../yaml/README.md#include) keyword. - See a [visualization](#visualize-ci-configuration) of the current configuration. +- View an [expanded](#view-expanded-configuration) version of your configuration. - [Commit](#commit-changes-to-ci-configuration) the changes to a specific branch. NOTE: -You must have already [created a CI/CD configuration file](../quick_start/README.md#create-a-gitlab-ciyml-file) -to use the editor. +You must already have [a `.gitlab-ci.yml` file](../quick_start/index.md#create-a-gitlab-ciyml-file) +on the default branch (usually `master`) of your project to use the editor. ## Validate CI configuration @@ -62,26 +63,29 @@ reflected in the CI lint. It displays the same results as the existing [CI Lint > - It was [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. > - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/290117) in GitLab 13.8. > - It's enabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-cicd-configuration-visualization). **(CORE ONLY)** +> - It's recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-cicd-configuration-visualization). **(FREE SELF)** WARNING: This feature might not be available to you. Check the **version history** note above for details. +It is not accessible if the [pipeline editor is disabled](#enable-or-disable-pipeline-editor). -To see a visualization of your `gitlab-ci.yml` configuration, navigate to **CI/CD > Editor** -and select the `visualization` tab. The visualization shows all stages and jobs. -[`needs`](../yaml/README.md#needs) relationships are displayed as lines connecting jobs together, showing the hierarchy of execution: +To view a visualization of your `gitlab-ci.yml` configuration, in your project, +go to **CI/CD > Editor**, and then select the **Visualize** tab. The +visualization shows all stages and jobs. Any [`needs`](../yaml/README.md#needs) +relationships are displayed as lines connecting jobs together, showing the +hierarchy of execution:  -Hovering on a job highlights its `needs` relationships: +Hover over a job to highlight its `needs` relationships:  If the configuration does not have any `needs` relationships, then no lines are drawn because each job depends only on the previous stage being completed successfully. -### Enable or disable CI/CD configuration visualization **(CORE ONLY)** +### Enable or disable CI/CD configuration visualization **(FREE SELF)** CI/CD configuration visualization is under development but ready for production use. It is deployed behind a feature flag that is **enabled by default**. @@ -100,6 +104,40 @@ To enable it: Feature.enable(:ci_config_visualization_tab) ``` +## View expanded configuration + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/246801) in GitLab 13.9. +> - It is [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-expanded-configuration). **(FREE SELF)** + +To view the fully expanded CI/CD configuration as one combined file, go to the +pipeline editor's **View merged YAML** tab. This tab displays an expanded configuration +where: + +- Configuration imported with [`include`](../yaml/README.md#include) is copied into the view. +- Jobs that use [`extends`](../yaml/README.md#extends) display with the + [extended configuration merged into the job](../yaml/README.md#merge-details). +- YAML anchors are [replaced with the linked configuration](../yaml/README.md#anchors). + +### Enable or disable expanded configuration **(FREE SELF)** + +Expanded CI/CD configuration is under development and not ready for production use. +It is deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to enable it. + +To enable it: + +```ruby +Feature.enable(:ci_config_visualization_tab) +``` + +To disable it: + +```ruby +Feature.disable(:ci_config_visualization_tab) +``` + ## Commit changes to CI configuration The commit form appears at the bottom of each tab in the editor so you can commit @@ -113,9 +151,9 @@ checkbox appears. Select it to start a new merge request after you commit the ch  -## Enable or disable pipeline editor **(CORE ONLY)** +## Enable or disable pipeline editor **(FREE SELF)** -The pipeline editor is under development and not ready for production use. It is +The pipeline editor is under development but ready for production use. It is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) can disable it. diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md deleted file mode 100644 index 090dbd3d347..00000000000 --- a/doc/ci/pipelines.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'pipelines/index.md' ---- - -This document was moved to [another location](pipelines/index.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipelines/img/job_artifacts_merge_request.png b/doc/ci/pipelines/img/job_artifacts_merge_request.png Binary files differindex fa1ed9acbf8..e87839ceeca 100644 --- a/doc/ci/pipelines/img/job_artifacts_merge_request.png +++ b/doc/ci/pipelines/img/job_artifacts_merge_request.png diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 7920a3bf7f3..6bc0d9bddd9 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -77,7 +77,7 @@ You can also configure specific aspects of your pipelines through the GitLab UI. - [Pipeline settings](settings.md) for each project. - [Pipeline schedules](schedules.md). -- [Custom CI/CD variables](../variables/README.md#custom-environment-variables). +- [Custom CI/CD variables](../variables/README.md#custom-cicd-variables). ### Ref Specs for Runners @@ -132,10 +132,6 @@ Pipelines can be manually executed, with predefined or manually-specified [varia You might do this if the results of a pipeline (for example, a code build) are required outside the normal operation of the pipeline. -[In GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/30101), -all global variables with descriptions defined in the `.gitlab-ci.yml` file are -displayed in the variable fields. - To execute a pipeline manually: 1. Navigate to your project's **CI/CD > Pipelines**. @@ -143,10 +139,33 @@ To execute a pipeline manually: 1. On the **Run Pipeline** page: 1. Select the branch to run the pipeline for in the **Create for** field. 1. Enter any [environment variables](../variables/README.md) required for the pipeline run. + You can set specific variables to have their [values prefilled in the form](#prefill-variables-in-manual-pipelines). 1. Click the **Create pipeline** button. The pipeline now executes the jobs as configured. +#### Prefill variables in manual pipelines + +> [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/30101) GitLab 13.7. + +You can use the [`value` and `description`](../yaml/README.md#prefill-variables-in-manual-pipelines) +keywords to define [variables](../variables/README.md) that are prefilled when running +a pipeline manually. + +In pipelines triggered manually, the **Run pipelines** page displays all variables +with a `description` and `value` defined in the `.gitlab-ci.yml` file. The values +can then be modified if needed, which overrides the value for that single pipeline run. + +The description is displayed below the variable. It can be used to explain what +the variable is used for, what the acceptable values are, and so on: + +```yaml +variables: + DEPLOY_ENVIRONMENT: + value: "staging" # Deploy to staging by default + description: "The deployment target. Change this variable to 'canary' or 'production' if needed." +``` + ### Run a pipeline by using a URL query string > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24146) in GitLab 12.5. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 4c77a578aa4..cf8e4e71534 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -150,7 +150,7 @@ in merge requests. For more information, see > - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in GitLab 13.2. > - Requires GitLab Runner 11.5 and above. -The `codequality` report collects [CodeQuality issues](../../user/project/merge_requests/code_quality.md) +The `codequality` report collects [Code Quality issues](../../user/project/merge_requests/code_quality.md) as artifacts. The collected Code Quality report uploads to GitLab as an artifact and is summarized in merge requests. @@ -168,9 +168,11 @@ The collected SAST report uploads to GitLab as an artifact and is summarized in merge requests and the pipeline view. It's also used to provide data for security dashboards. -#### `artifacts:reports:secret_detection` **(ULTIMATE)** +#### `artifacts:reports:secret_detection` > - Introduced in GitLab 13.1. +> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/222788) in GitLab + 13.3. > - Requires GitLab Runner 11.5 and above. The `secret-detection` report collects [detected secrets](../../user/application_security/secret_detection/index.md) @@ -482,6 +484,12 @@ a project, you can disable this behavior to save space: 1. Navigate to **Settings > CI/CD > Artifacts**. 1. Uncheck **Keep artifacts from most recent successful jobs**. +You can disable this behavior for all projects on a self-managed instance in the +[instance's CI/CD settings](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). **(CORE ONLY)** + +When you disable the feature, the latest artifacts do not immediately expire. +A new pipeline must run before the latest artifacts can expire and be deleted. + ## Troubleshooting ### Error message `No files to upload` @@ -490,7 +498,7 @@ This is often preceded by other errors or warnings that specify the filename and generated in the first place. Please check the entire job log for such messages. If you find no helpful messages, please retry the failed job after activating -[CI debug logging](../variables/README.md#debug-logging). +[CI/CD debug logging](../variables/README.md#debug-logging). This provides useful information to investigate further. <!-- ## Troubleshooting diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index de78b8558c4..faebf40462e 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -20,7 +20,7 @@ to use pipeline features that improve efficiency right away, and get a faster so development lifecycle earlier. First ensure you are familiar with [GitLab CI/CD fundamentals](../introduction/index.md) -and understand the [quick start guide](../quick_start/README.md). +and understand the [quick start guide](../quick_start/index.md). ## Identify bottlenecks and common failures @@ -161,7 +161,7 @@ Try to find which jobs don't need to run in all situations, and use pipeline con to stop them from running: - Use the [`interruptible`](../yaml/README.md#interruptible) keyword to stop old pipelines - when they are superceded by a newer pipeline. + when they are superseded by a newer pipeline. - Use [`rules`](../yaml/README.md#rules) to skip tests that aren't needed. For example, skip backend tests when only the frontend code is changed. - Run non-essential [scheduled pipelines](schedules.md) less frequently. @@ -235,7 +235,7 @@ Methods to reduce Docker image size: to analyze and shrink images. To simplify Docker image management, you can create a dedicated group for managing -[Docker images](../docker/README.md) and test, build and publish them with CI/CD pipelines. +[Docker images](../docker/index.md) and test, build and publish them with CI/CD pipelines. ## Test, document, and learn diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index cddfcb754ec..a8e310c1f0d 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -6,7 +6,7 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/schedules. type: reference, howto --- -# Pipeline schedules +# Pipeline schedules **(FREE)** > - Introduced in GitLab 9.1 as [Trigger Schedule](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10533). > - [Renamed to Pipeline Schedule](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10853) in GitLab 9.2. @@ -83,7 +83,7 @@ job: - make build ``` -### Advanced configuration +### Advanced configuration **(FREE SELF)** The pipelines are not executed exactly on schedule because schedules are handled by Sidekiq, which runs according to its interval. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 5a758bccd62..32221b78039 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -63,12 +63,12 @@ if the job surpasses the threshold, it is marked as failed. Project defined timeout (either specific timeout set by user or the default 60 minutes timeout) may be [overridden for runners](../runners/README.md#set-maximum-job-timeout-for-a-runner). -## Maximum artifacts size **(CORE ONLY)** +## Maximum artifacts size **(FREE SELF)** For information about setting a maximum artifact size for a project, see [Maximum artifacts size](../../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size). -## Custom CI configuration path +## Custom CI/CD configuration path > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12509) in GitLab 9.4. > - [Support for external `.gitlab-ci.yml` locations](https://gitlab.com/gitlab-org/gitlab/-/issues/14376) introduced in GitLab 12.6. @@ -80,7 +80,7 @@ To customize the path: 1. Go to the project's **Settings > CI / CD**. 1. Expand the **General pipelines** section. -1. Provide a value in the **Custom CI configuration path** field. +1. Provide a value in the **CI/CD configuration file** field. 1. Click **Save changes**. If the CI configuration is stored within the repository in a non-default @@ -131,8 +131,23 @@ averaged.  -A few examples of known coverage tools for a variety of languages can be found -in the pipelines settings page. +<!-- vale gitlab.Spelling = NO --> + +| Coverage Tool | Sample regular expression | +|------------------------------------------------|---------------------------| +| Simplecov (Ruby) | `\(\d+.\d+\%\) covered` | +| pytest-cov (Python) | `^TOTAL.+?(\d+\%)$` | +| Scoverage (Scala) | `Statement coverage[A-Za-z\.*]\s*:\s*([^%]+)` | +| `phpunit --coverage-text --colors=never` (PHP) | `^\s*Lines:\s*\d+.\d+\%` | +| gcovr (C/C++) | `^TOTAL.*\s+(\d+\%)$` | +| `tap --coverage-report=text-summary` (NodeJS) | `^Statements\s*:\s*([^%]+)` | +| `nyc npm test` (NodeJS) | `All files[^|]*\|[^|]*\s+([\d\.]+)` | +| excoveralls (Elixir) | `\[TOTAL\]\s+(\d+\.\d+)%` | +| `mix test --cover` (Elixir) | `\d+.\d+\%\s+\|\s+Total` | +| JaCoCo (Java/Kotlin) | `Total.*?([0-9]{1,3})%` | +| `go test -cover` (Go) | `coverage: \d+.\d+% of statements` | + +<!-- vale gitlab.Spelling = YES --> ### Code Coverage history @@ -143,7 +158,7 @@ To see the evolution of your project code coverage over time, you can view a graph or download a CSV file with this data. From your project: 1. Go to **{chart}** **Project Analytics > Repository** to see the historic data for each job listed in the dropdown above the graph. -1. If you want a CSV file of that data, click **Download raw data (.csv)** +1. If you want a CSV file of that data, click **Download raw data (`.csv`)**  @@ -198,7 +213,7 @@ If **Public pipelines** is disabled: - For **private** projects, only project members (reporter or higher) can view the pipelines or access the related features. -## Auto-cancel pending pipelines +## Auto-cancel redundant pipelines > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9362) in GitLab 9.1. @@ -206,7 +221,7 @@ You can set pending or running pipelines to cancel automatically when a new pipe 1. Go to **Settings > CI / CD**. 1. Expand **General Pipelines**. -1. Check the **Auto-cancel redundant, pending pipelines** checkbox. +1. Check the **Auto-cancel redundant pipelines** checkbox. 1. Click **Save changes**. Use the [`interruptible`](../yaml/README.md#interruptible) keyword to indicate if a @@ -336,9 +351,9 @@ https://gitlab.com/gitlab-org/gitlab/badges/master/coverage.svg?job=karma&key_te  -## Environment Variables +## CI/CD Variables -[Environment variables](../variables/README.md#gitlab-cicd-environment-variables) can be set in an environment to be available to a runner. +[CI/CD variables](../variables/README.md) can be set to be available to a runner. <!-- ## Troubleshooting diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index e9c85353db3..c94d6e3ea80 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,157 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'index.md' --- -# Get started with GitLab CI/CD +This document was moved to [another location](index.md). -Use this document to get started with -GitLab [continuous integration](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/). - -Before you start, make sure you have: - -- A project in GitLab that you would like to use CI/CD for. -- Maintainer or owner access for the project. - -If you are migrating from another CI/CD tool, view this documentation: - -- [Migrate from CircleCI](../migration/circleci.md). -- [Migrate from Jenkins](../migration/jenkins.md). - -## CI/CD process overview - -To use GitLab CI/CD: - -1. [Ensure you have runners available](#ensure-you-have-runners-available) to run your jobs. - If you don't have a runner, [install GitLab Runner](https://docs.gitlab.com/runner/install/) - and [register a runner](https://docs.gitlab.com/runner/register/) for your instance, project, or group. -1. [Create a `.gitlab-ci.yml` file](#create-a-gitlab-ciyml-file) - at the root of your repository. This file is where you define your CI/CD jobs. - -When you commit the file to your repository, the runner runs your jobs. -The job results [are displayed in a pipeline](#view-the-status-of-your-pipeline-and-jobs). - -### Ensure you have runners available - -In GitLab, runners are agents that run your CI/CD jobs. - -You might already have runners available for your project, including -[shared runners](../runners/README.md#shared-runners), which are -available to all projects in your GitLab instance. - -To view available runners: - -- Go to **Settings > CI/CD** and expand **Runners**. - -As long as you have at least one runner that's active, with a green circle next to it, -you have a runner available to process your jobs. - -If no runners are listed on the **Runners** page in the UI, you or an administrator -must [install GitLab Runner](https://docs.gitlab.com/runner/install/) and -[register](https://docs.gitlab.com/runner/register/) at least one runner. - -If you are testing CI/CD, you can install GitLab Runner and register runners on your local machine. -When your CI/CD jobs run, they run on your local machine. - -### Create a `.gitlab-ci.yml` file - -The `.gitlab-ci.yml` file is a [YAML](https://en.wikipedia.org/wiki/YAML) file where -you configure specific instructions for GitLab CI/CD. - -In this file, you define: - -- The structure and order of jobs that the runner should execute. -- The decisions the runner should make when specific conditions are encountered. - -For example, you might want to run a suite of tests when you commit to -any branch except `master`. When you commit to `master`, you want -to run the same suite, but also publish your application. - -All of this is defined in the `.gitlab-ci.yml` file. - -To create a `.gitlab-ci.yml` file: - -1. Go to **Project overview > Details**. -1. Above the file list, select the branch you want to commit to, - click the plus icon, then select **New file**: - -  - -1. For the **Filename**, type `.gitlab-ci.yml` and in the larger window, - paste this sample code: - - ```yaml - build-job: - stage: build - script: - - echo "Hello, $GITLAB_USER_LOGIN!" - - test-job1: - stage: test - script: - - echo "This job tests something" - - test-job2: - stage: test - script: - - echo "This job tests something, but takes more time than test-job1." - - echo "After the echo commands complete, it runs the sleep command for 20 seconds" - - echo "which simulates a test that runs 20 seconds longer than test-job1" - - sleep 20 - - deploy-prod: - stage: deploy - script: - - echo "This job deploys something from the $CI_COMMIT_BRANCH branch." - ``` - - `$GITLAB_USER_LOGIN` and `$CI_COMMIT_BRANCH` are - [predefined variables](../variables/predefined_variables.md) - that populate when the job runs. - -1. Click **Commit changes**. - -The pipeline starts when the commit is committed. - -#### `.gitlab-ci.yml` tips - -- If you want the runner to use a Docker image to run the jobs, edit the `.gitlab-ci.yml` file - to include your image name: - - ```yaml - default: - image: ruby:2.7.2 - ``` - - This command tells the runner to use a Ruby image from Docker Hub. - -- To validate your `.gitlab-ci.yml` file, use the - [CI Lint tool](../lint.md), which is available in every project. -- You can also use [CI/CD configuration visualization](../pipeline_editor/index.md#visualize-ci-configuration) to - view a graphical representation of your `.gitlab-ci.yml` file. -- For the complete `.gitlab-ci.yml` syntax, see - [the `.gitlab-ci.yml` reference topic](../yaml/README.md). - -### View the status of your pipeline and jobs - -When you committed your changes, a pipeline started. - -To view your pipeline: - -- Go **CI/CD > Pipelines**. - - A pipeline with three stages should be displayed: - -  - -- To view a visual representation of your pipeline, click the pipeline ID. - -  - -- To view details of a job, click the job name, for example, `deploy-prod`. - -  - -If the job status is `stuck`, check to ensure a runner is probably configured for the project. +<!-- This redirect file can be deleted after 2021-05-01. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md new file mode 100644 index 00000000000..711bf0c0e7a --- /dev/null +++ b/doc/ci/quick_start/index.md @@ -0,0 +1,157 @@ +--- +stage: Verify +group: Continuous Integration +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# Get started with GitLab CI/CD **(FREE)** + +Use this document to get started with +GitLab [continuous integration](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/). + +Before you start, make sure you have: + +- A project in GitLab that you would like to use CI/CD for. +- Maintainer or owner access for the project. + +If you are migrating from another CI/CD tool, view this documentation: + +- [Migrate from CircleCI](../migration/circleci.md). +- [Migrate from Jenkins](../migration/jenkins.md). + +## CI/CD process overview + +To use GitLab CI/CD: + +1. [Ensure you have runners available](#ensure-you-have-runners-available) to run your jobs. + If you don't have a runner, [install GitLab Runner](https://docs.gitlab.com/runner/install/) + and [register a runner](https://docs.gitlab.com/runner/register/) for your instance, project, or group. +1. [Create a `.gitlab-ci.yml` file](#create-a-gitlab-ciyml-file) + at the root of your repository. This file is where you define your CI/CD jobs. + +When you commit the file to your repository, the runner runs your jobs. +The job results [are displayed in a pipeline](#view-the-status-of-your-pipeline-and-jobs). + +### Ensure you have runners available + +In GitLab, runners are agents that run your CI/CD jobs. + +You might already have runners available for your project, including +[shared runners](../runners/README.md#shared-runners), which are +available to all projects in your GitLab instance. + +To view available runners: + +- Go to **Settings > CI/CD** and expand **Runners**. + +As long as you have at least one runner that's active, with a green circle next to it, +you have a runner available to process your jobs. + +If no runners are listed on the **Runners** page in the UI, you or an administrator +must [install GitLab Runner](https://docs.gitlab.com/runner/install/) and +[register](https://docs.gitlab.com/runner/register/) at least one runner. + +If you are testing CI/CD, you can install GitLab Runner and register runners on your local machine. +When your CI/CD jobs run, they run on your local machine. + +### Create a `.gitlab-ci.yml` file + +The `.gitlab-ci.yml` file is a [YAML](https://en.wikipedia.org/wiki/YAML) file where +you configure specific instructions for GitLab CI/CD. + +In this file, you define: + +- The structure and order of jobs that the runner should execute. +- The decisions the runner should make when specific conditions are encountered. + +For example, you might want to run a suite of tests when you commit to +any branch except `master`. When you commit to `master`, you want +to run the same suite, but also publish your application. + +All of this is defined in the `.gitlab-ci.yml` file. + +To create a `.gitlab-ci.yml` file: + +1. Go to **Project overview > Details**. +1. Above the file list, select the branch you want to commit to, + click the plus icon, then select **New file**: + +  + +1. For the **Filename**, type `.gitlab-ci.yml` and in the larger window, + paste this sample code: + + ```yaml + build-job: + stage: build + script: + - echo "Hello, $GITLAB_USER_LOGIN!" + + test-job1: + stage: test + script: + - echo "This job tests something" + + test-job2: + stage: test + script: + - echo "This job tests something, but takes more time than test-job1." + - echo "After the echo commands complete, it runs the sleep command for 20 seconds" + - echo "which simulates a test that runs 20 seconds longer than test-job1" + - sleep 20 + + deploy-prod: + stage: deploy + script: + - echo "This job deploys something from the $CI_COMMIT_BRANCH branch." + ``` + + `$GITLAB_USER_LOGIN` and `$CI_COMMIT_BRANCH` are + [predefined variables](../variables/predefined_variables.md) + that populate when the job runs. + +1. Click **Commit changes**. + +The pipeline starts when the commit is committed. + +#### `.gitlab-ci.yml` tips + +- If you want the runner to use a Docker image to run the jobs, edit the `.gitlab-ci.yml` file + to include your image name: + + ```yaml + default: + image: ruby:2.7.2 + ``` + + This command tells the runner to use a Ruby image from Docker Hub. + +- To validate your `.gitlab-ci.yml` file, use the + [CI Lint tool](../lint.md), which is available in every project. +- You can also use [CI/CD configuration visualization](../pipeline_editor/index.md#visualize-ci-configuration) to + view a graphical representation of your `.gitlab-ci.yml` file. +- For the complete `.gitlab-ci.yml` syntax, see + [the `.gitlab-ci.yml` reference topic](../yaml/README.md). + +### View the status of your pipeline and jobs + +When you committed your changes, a pipeline started. + +To view your pipeline: + +- Go **CI/CD > Pipelines**. + + A pipeline with three stages should be displayed: + +  + +- To view a visual representation of your pipeline, click the pipeline ID. + +  + +- To view details of a job, click the job name, for example, `deploy-prod`. + +  + +If the job status is `stuck`, check to ensure a runner is probably configured for the project. diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 3512b77e4e2..9de6a1162bd 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -58,7 +58,7 @@ The process of configuring Review Apps is as follows: 1. Set up the infrastructure to host and deploy the Review Apps (check the [examples](#review-apps-examples) below). 1. [Install](https://docs.gitlab.com/runner/install/) and [configure](https://docs.gitlab.com/runner/commands/) a runner to do deployment. -1. Set up a job in `.gitlab-ci.yml` that uses the [predefined CI environment variable](../variables/README.md) `${CI_COMMIT_REF_NAME}` +1. Set up a job in `.gitlab-ci.yml` that uses the [predefined CI/CD variable](../variables/README.md) `${CI_COMMIT_REF_NAME}` to create dynamic environments and restrict it to run only on branches. Alternatively, you can get a YML template for this job by [enabling review apps](#enable-review-apps-button) for your project. 1. Optionally, set a job that [manually stops](../environments/index.md#stopping-an-environment) the Review Apps. @@ -180,18 +180,19 @@ After you have the route mapping set up, it takes effect in the following locati - In the diff for a merge request, comparison, or commit. -  +  - In the blob file view. -  +  -## Visual Reviews **(STARTER)** +## Visual Reviews **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10761) in GitLab Starter 12.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10761) in GitLab 12.0. +> - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. > - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. > - It's enabled on GitLab.com. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-visual-reviews). **(STARTER ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-visual-reviews). **(PREMIUM SELF)** With Visual Reviews, members of any team (Product, Design, Quality, and so on) can provide feedback comments through a form in your review apps. The comments are added to the merge request that triggered the review app. @@ -242,7 +243,7 @@ looks for a project with code hosted in a project on GitLab.com: </script> ``` -Ideally, you should use [environment variables](../variables/predefined_variables.md) +Ideally, you should use [CI/CD variables](../variables/predefined_variables.md) to replace those values at runtime when each review app is created: - `data-project-id` is the project ID, which can be found by the `CI_PROJECT_ID` @@ -288,7 +289,7 @@ can supply the ID by either:​​ - Dynamically adding the `data-merge-request-id` value during the build of the app. - Supplying it manually through the visual review form in the app. -### Enable or disable Visual Reviews **(STARTER ONLY)** +### Enable or disable Visual Reviews **(PREMIUM SELF)** Visual Reviews is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index d6ea4d83825..bf7552ad609 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -37,10 +37,10 @@ multiple projects. If you are using a self-managed instance of GitLab: -- Your administrator can install and register shared runners by [following the documentation](https://docs.gitlab.com/runner/install/index.html). - <!-- going to your project's--> - <!-- **Settings > CI / CD**, expanding the **Runners** section, and clicking **Show runner installation instructions**.--> - <!-- These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html).--> +- Your administrator can install and register shared runners by + going to your project's **Settings > CI / CD**, expanding the **Runners** section, + and clicking **Show runner installation instructions**. + These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html). - The administrator can also configure a maximum number of shared runner [pipeline minutes for each group](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota). @@ -136,10 +136,16 @@ Shared runners are automatically disabled for a project: To disable shared runners for a group: 1. Go to the group's **Settings > CI/CD** and expand the **Runners** section. -1. In the **Shared runners** area, click **Enable shared runners for this group**. +1. In the **Shared runners** area, turn off the **Enable shared runners for this group** toggle. 1. Optionally, to allow shared runners to be enabled for individual projects or subgroups, click **Allow projects and subgroups to override the group setting**. +NOTE: +To re-enable the shared runners for a group, turn on the +**Enable shared runners for this group** toggle. +Then, an owner or maintainer must explicitly change this setting +for each project subgroup or project. + ### Group runners Use *Group runners* when you want all projects in a group @@ -274,7 +280,7 @@ On GitLab.com, you cannot override the job timeout for shared runners and must u To set the maximum job timeout: 1. In a project, go to **Settings > CI/CD > Runners**. -1. Select your specific runner to edit the settings. +1. Select your specific runner to edit the settings. 1. Enter a value under **Maximum job timeout**. 1. Select **Save changes**. @@ -805,3 +811,38 @@ You can set them globally or per-job in the [`variables`](../yaml/README.md#vari ## System calls not available on GitLab.com shared runners GitLab.com shared runners run on CoreOS. This means that you cannot use some system calls, like `getlogin`, from the C standard library. + +## Artifact and cache settings + +> Introduced in GitLab Runner 13.9. + +Artifact and cache settings control the compression ratio of artifacts and caches. +Use these settings to specify the size of the archive produced by a job. + +- On a slow network, uploads might be faster for smaller archives. +- On a fast network where bandwidth and storage are not a concern, uploads might be faster using the fastest compression ratio, despite the archive produced being larger. + +For [GitLab Pages](../../user/project/pages/index.md) to serve +[HTTP Range requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests), artifacts +should use the `ARTIFACT_COMPRESSION_LEVEL: fastest` setting, as only uncompressed zip archives +support this feature. + +A meter can also be enabled to provide the rate of transfer for uploads and downloads. + +```yaml +variables: + # output upload and download progress every 2 seconds + TRANSFER_METER_FREQUENCY: "2s" + + # Use fast compression for artifacts, resulting in larger archives + ARTIFACT_COMPRESSION_LEVEL: "fast" + + # Use no compression for caches + CACHE_COMPRESSION_LEVEL: "fastest" +``` + +| Variable | Description | +|---------------------------------|--------------------------------------------------------| +| `TRANSFER_METER_FREQUENCY` | Specify how often to print the meter's transfer rate. It can be set to a duration (for example, `1s` or `1m30s`). A duration of `0` disables the meter (default). When a value is set, the pipeline shows a progress meter for artifact and cache uploads and downloads. | +| `ARTIFACT_COMPRESSION_LEVEL` | To adjust compression ratio, set to `fastest`, `fast`, `default`, `slow`, or `slowest`. This setting works with the Fastzip archiver only, so the GitLab Runner feature flag [`FF_USE_FASTZIP`](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) must also be enabled. | +| `CACHE_COMPRESSION_LEVEL` | To adjust compression ratio, set to `fastest`, `fast`, `default`, `slow`, or `slowest`. This setting works with the Fastzip archiver only, so the GitLab Runner feature flag [`FF_USE_FASTZIP`](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) must also be enabled. | diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index f05812f77f7..eac72bc7351 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -13,7 +13,7 @@ Secrets represent sensitive information your CI job needs to complete work. This sensitive information can be items like API tokens, database credentials, or private keys. Secrets are sourced from your secrets provider. -Unlike CI variables, which are always presented to a job, secrets must be explicitly +Unlike CI/CD variables, which are always presented to a job, secrets must be explicitly required by a job. Read [GitLab CI/CD pipeline configuration reference](../yaml/README.md#secrets) for more information about the syntax. @@ -80,7 +80,7 @@ To configure your Vault server: 1. Configure roles on your Vault server, restricting roles to a project or namespace, as described in [Configure Vault server roles](#configure-vault-server-roles) on this page. -1. [Create the following CI variables](../variables/README.md#custom-environment-variables) +1. [Create the following CI/CD variables](../variables/README.md#custom-cicd-variables) to provide details about your Vault server: - `VAULT_SERVER_URL` - The URL of your Vault server, such as `https://vault.example.com:8200`. Required. @@ -113,8 +113,8 @@ In this example: - `ops` - The path where the secrets engine is mounted. After GitLab fetches the secret from Vault, the value is saved in a temporary file. -The path to this file is stored in environment variable named `DATABASE_PASSWORD`, -similar to [CI variables of type `file`](../variables/README.md#custom-environment-variables-of-type-file). +The path to this file is stored in a CI/CD variable named `DATABASE_PASSWORD`, +similar to [variables of type `file`](../variables/README.md#custom-cicd-variables-of-type-file). For more information about the supported syntax, read the [`.gitlab-ci.yml` reference](../yaml/README.md#secretsvault). diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 71c2be70de3..c94d6e3ea80 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,21 +1,8 @@ --- -stage: Verify -group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -comments: false -type: index +redirect_to: 'index.md' --- -# GitLab CI services examples +This document was moved to [another location](index.md). -The [`services`](../docker/using_docker_images.md#what-is-a-service) -keyword defines a Docker image that runs during a `job` linked to the -Docker image that the image keyword defines. This allows you to access -the service image during build time. - -The service image can run any application, but the most common use -case is to run a database container, for example: - -- [Using MySQL](mysql.md) -- [Using PostgreSQL](postgres.md) -- [Using Redis](redis.md) +<!-- This redirect file can be deleted after 2021-05-01. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md deleted file mode 100644 index e4653cdc9e2..00000000000 --- a/doc/ci/services/docker-services.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -redirect_to: 'README.md' ---- - -This document was moved to [another location](README.md). - -<!-- This redirect file can be deleted after February 1, 2021. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md new file mode 100644 index 00000000000..71c2be70de3 --- /dev/null +++ b/doc/ci/services/index.md @@ -0,0 +1,21 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +comments: false +type: index +--- + +# GitLab CI services examples + +The [`services`](../docker/using_docker_images.md#what-is-a-service) +keyword defines a Docker image that runs during a `job` linked to the +Docker image that the image keyword defines. This allows you to access +the service image during build time. + +The service image can run any application, but the most common use +case is to run a database container, for example: + +- [Using MySQL](mysql.md) +- [Using PostgreSQL](postgres.md) +- [Using Redis](redis.md) diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 1595907184e..5bd034cbf97 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -14,7 +14,7 @@ need it for your tests to run. If you want to use a MySQL container, you can use [GitLab Runner](../runners/README.md) with the Docker executor. -1. [Create variables](../variables/README.md#create-a-custom-variable-in-the-ui) for your +1. [Create CI/CD variables](../variables/README.md#create-a-custom-variable-in-the-ui) for your MySQL database and password by going to **Settings > CI/CD**, expanding **Variables**, and clicking **Add Variable**. diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index d37875e1e05..16576069583 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -31,7 +31,7 @@ variables: To set values for the `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD` and `POSTGRES_HOST_AUTH_METHOD`, -[assign them to a variable in the user interface](../variables/README.md#create-a-custom-variable-in-the-ui), +[assign them to a CI/CD variable in the user interface](../variables/README.md#create-a-custom-variable-in-the-ui), then assign that variable to the corresponding variable in your `.gitlab-ci.yml` file. diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index a5410d53a95..c94d6e3ea80 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -1,212 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: 'index.md' --- -# Using SSH keys with GitLab CI/CD +This document was moved to [another location](index.md). -GitLab currently doesn't have built-in support for managing SSH keys in a build -environment (where the GitLab Runner runs). - -The SSH keys can be useful when: - -1. You want to checkout internal submodules -1. You want to download private packages using your package manager (e.g., Bundler) -1. You want to deploy your application to your own server, or, for example, Heroku -1. You want to execute SSH commands from the build environment to a remote server -1. You want to rsync files from the build environment to a remote server - -If anything of the above rings a bell, then you most likely need an SSH key. - -The most widely supported method is to inject an SSH key into your build -environment by extending your `.gitlab-ci.yml`, and it's a solution which works -with any type of [executor](https://docs.gitlab.com/runner/executors/) -(Docker, shell, etc.). - -## How it works - -1. Create a new SSH key pair locally with [`ssh-keygen`](https://linux.die.net/man/1/ssh-keygen) -1. Add the private key as a [variable](../variables/README.md) to - your project -1. Run the [`ssh-agent`](https://linux.die.net/man/1/ssh-agent) during job to load - the private key. -1. Copy the public key to the servers you want to have access to (usually in - `~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys) - if you are accessing a private GitLab repository. - -The private key is displayed in the job log, unless you enable -[debug logging](../variables/README.md#debug-logging). You might also want to -check the [visibility of your pipelines](../pipelines/settings.md#visibility-of-pipelines). - -## SSH keys when using the Docker executor - -When your CI/CD jobs run inside Docker containers (meaning the environment is -contained) and you want to deploy your code in a private server, you need a way -to access it. This is where an SSH key pair comes in handy. - -1. You first need to create an SSH key pair. For more information, follow - the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair). - **Do not** add a passphrase to the SSH key, or the `before_script` will - prompt for it. - -1. Create a new [variable](../variables/README.md#gitlab-cicd-environment-variables). - As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste - the content of your _private_ key that you created earlier. - -1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following - example, a Debian based image is assumed. Edit to your needs: - - ```yaml - before_script: - ## - ## Install ssh-agent if not already installed, it is required by Docker. - ## (change apt-get to yum if you use an RPM-based image) - ## - - 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )' - - ## - ## Run ssh-agent (inside the build environment) - ## - - eval $(ssh-agent -s) - - ## - ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store - ## We're using tr to fix line endings which makes ed25519 keys work - ## without extra base64 encoding. - ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556 - ## - - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - - - ## - ## Create the SSH directory and give it the right permissions - ## - - mkdir -p ~/.ssh - - chmod 700 ~/.ssh - - ## - ## Optionally, if you will be using any Git commands, set the user name and - ## and email. - ## - # - git config --global user.email "user@example.com" - # - git config --global user.name "User name" - ``` - - The [`before_script`](../yaml/README.md#before_script) can be set globally - or per-job. - -1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys). - -1. As a final step, add the _public_ key from the one you created in the first - step to the services that you want to have an access to from within the build - environment. If you are accessing a private GitLab repository you need to add - it as a [deploy key](../../ssh/README.md#deploy-keys). - -That's it! You can now have access to private servers or repositories in your -build environment. - -## SSH keys when using the Shell executor - -If you are using the Shell executor and not Docker, it is easier to set up an -SSH key. - -You can generate the SSH key from the machine that GitLab Runner is installed -on, and use that key for all projects that are run on this machine. - -1. First, log in to the server that runs your jobs. - -1. Then, from the terminal, log in as the `gitlab-runner` user: - - ```shell - sudo su - gitlab-runner - ``` - -1. Generate the SSH key pair as described in the instructions to - [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair). - **Do not** add a passphrase to the SSH key, or the `before_script` will - prompt for it. - -1. As a final step, add the _public_ key from the one you created earlier to the - services that you want to have an access to from within the build environment. - If you are accessing a private GitLab repository you need to add it as a - [deploy key](../../ssh/README.md#deploy-keys). - -After generating the key, try to sign in to the remote server to accept the -fingerprint: - -```shell -ssh example.com -``` - -For accessing repositories on GitLab.com, you would use `git@gitlab.com`. - -## Verifying the SSH host keys - -It is a good practice to check the private server's own public key to make sure -you are not being targeted by a man-in-the-middle attack. If anything -suspicious happens, you notice it because the job fails (the SSH -connection fails when the public keys don't match). - -To find out the host keys of your server, run the `ssh-keyscan` command from a -trusted network (ideally, from the private server itself): - -```shell -## Use the domain name -ssh-keyscan example.com - -## Or use an IP -ssh-keyscan 1.2.3.4 -``` - -Create a new [variable](../variables/README.md#gitlab-cicd-environment-variables) with -`SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`. - -If you need to connect to multiple servers, all the server host keys -need to be collected in the **Value** of the variable, one key per line. - -NOTE: -By using a variable instead of `ssh-keyscan` directly inside -`.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml` -if the host domain name changes for some reason. Also, the values are predefined -by you, meaning that if the host keys suddenly change, the CI/CD job doesn't fail, -so there's something wrong with the server or the network. - -Now that the `SSH_KNOWN_HOSTS` variable is created, in addition to the -[content of `.gitlab-ci.yml`](#ssh-keys-when-using-the-docker-executor) -above, here's what more you need to add: - -```yaml -before_script: - ## - ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the - ## following two lines. - ## - - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts - - chmod 644 ~/.ssh/known_hosts - - ## - ## Alternatively, use ssh-keyscan to scan the keys of your private server. - ## Replace example.com with your private server's domain name. Repeat that - ## command if you have more than one server to connect to. - ## - # - ssh-keyscan example.com >> ~/.ssh/known_hosts - # - chmod 644 ~/.ssh/known_hosts - - ## - ## You can optionally disable host key checking. Be aware that by adding that - ## you are susceptible to man-in-the-middle attacks. - ## WARNING: Use this only with the Docker executor, if you use it with shell - ## you will overwrite your user's SSH config. - ## - # - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config' -``` - -## Example project - -We have set up an [Example SSH Project](https://gitlab.com/gitlab-examples/ssh-private-key/) for your convenience -that runs on [GitLab.com](https://gitlab.com) using our publicly available -[shared runners](../runners/README.md). - -Want to hack on it? Simply fork it, commit and push your changes. Within a few -moments the changes is picked by a public runner and the job starts. +<!-- This redirect file can be deleted after 2021-05-01. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/ssh_keys/index.md b/doc/ci/ssh_keys/index.md new file mode 100644 index 00000000000..2cdef176a22 --- /dev/null +++ b/doc/ci/ssh_keys/index.md @@ -0,0 +1,212 @@ +--- +stage: Verify +group: Continuous Integration +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: tutorial +--- + +# Using SSH keys with GitLab CI/CD + +GitLab currently doesn't have built-in support for managing SSH keys in a build +environment (where the GitLab Runner runs). + +The SSH keys can be useful when: + +1. You want to checkout internal submodules +1. You want to download private packages using your package manager (e.g., Bundler) +1. You want to deploy your application to your own server, or, for example, Heroku +1. You want to execute SSH commands from the build environment to a remote server +1. You want to rsync files from the build environment to a remote server + +If anything of the above rings a bell, then you most likely need an SSH key. + +The most widely supported method is to inject an SSH key into your build +environment by extending your `.gitlab-ci.yml`, and it's a solution which works +with any type of [executor](https://docs.gitlab.com/runner/executors/) +(Docker, shell, etc.). + +## How it works + +1. Create a new SSH key pair locally with [`ssh-keygen`](https://linux.die.net/man/1/ssh-keygen) +1. Add the private key as a [variable](../variables/README.md) to + your project +1. Run the [`ssh-agent`](https://linux.die.net/man/1/ssh-agent) during job to load + the private key. +1. Copy the public key to the servers you want to have access to (usually in + `~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys) + if you are accessing a private GitLab repository. + +The private key is displayed in the job log, unless you enable +[debug logging](../variables/README.md#debug-logging). You might also want to +check the [visibility of your pipelines](../pipelines/settings.md#visibility-of-pipelines). + +## SSH keys when using the Docker executor + +When your CI/CD jobs run inside Docker containers (meaning the environment is +contained) and you want to deploy your code in a private server, you need a way +to access it. This is where an SSH key pair comes in handy. + +1. You first need to create an SSH key pair. For more information, follow + the instructions to [generate an SSH key](../../ssh/README.md#generate-an-ssh-key-pair). + **Do not** add a passphrase to the SSH key, or the `before_script` will + prompt for it. + +1. Create a new [CI/CD variable](../variables/README.md). + As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste + the content of your _private_ key that you created earlier. + +1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following + example, a Debian based image is assumed. Edit to your needs: + + ```yaml + before_script: + ## + ## Install ssh-agent if not already installed, it is required by Docker. + ## (change apt-get to yum if you use an RPM-based image) + ## + - 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )' + + ## + ## Run ssh-agent (inside the build environment) + ## + - eval $(ssh-agent -s) + + ## + ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store + ## We're using tr to fix line endings which makes ed25519 keys work + ## without extra base64 encoding. + ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556 + ## + - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - + + ## + ## Create the SSH directory and give it the right permissions + ## + - mkdir -p ~/.ssh + - chmod 700 ~/.ssh + + ## + ## Optionally, if you will be using any Git commands, set the user name and + ## and email. + ## + # - git config --global user.email "user@example.com" + # - git config --global user.name "User name" + ``` + + The [`before_script`](../yaml/README.md#before_script) can be set globally + or per-job. + +1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys). + +1. As a final step, add the _public_ key from the one you created in the first + step to the services that you want to have an access to from within the build + environment. If you are accessing a private GitLab repository you need to add + it as a [deploy key](../../ssh/README.md#deploy-keys). + +That's it! You can now have access to private servers or repositories in your +build environment. + +## SSH keys when using the Shell executor + +If you are using the Shell executor and not Docker, it is easier to set up an +SSH key. + +You can generate the SSH key from the machine that GitLab Runner is installed +on, and use that key for all projects that are run on this machine. + +1. First, log in to the server that runs your jobs. + +1. Then, from the terminal, log in as the `gitlab-runner` user: + + ```shell + sudo su - gitlab-runner + ``` + +1. Generate the SSH key pair as described in the instructions to + [generate an SSH key](../../ssh/README.md#generate-an-ssh-key-pair). + **Do not** add a passphrase to the SSH key, or the `before_script` will + prompt for it. + +1. As a final step, add the _public_ key from the one you created earlier to the + services that you want to have an access to from within the build environment. + If you are accessing a private GitLab repository you need to add it as a + [deploy key](../../ssh/README.md#deploy-keys). + +After generating the key, try to sign in to the remote server to accept the +fingerprint: + +```shell +ssh example.com +``` + +For accessing repositories on GitLab.com, you would use `git@gitlab.com`. + +## Verifying the SSH host keys + +It is a good practice to check the private server's own public key to make sure +you are not being targeted by a man-in-the-middle attack. If anything +suspicious happens, you notice it because the job fails (the SSH +connection fails when the public keys don't match). + +To find out the host keys of your server, run the `ssh-keyscan` command from a +trusted network (ideally, from the private server itself): + +```shell +## Use the domain name +ssh-keyscan example.com + +## Or use an IP +ssh-keyscan 1.2.3.4 +``` + +Create a new [CI/CD variable](../variables/README.md) with +`SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`. + +If you need to connect to multiple servers, all the server host keys +need to be collected in the **Value** of the variable, one key per line. + +NOTE: +By using a variable instead of `ssh-keyscan` directly inside +`.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml` +if the host domain name changes for some reason. Also, the values are predefined +by you, meaning that if the host keys suddenly change, the CI/CD job doesn't fail, +so there's something wrong with the server or the network. + +Now that the `SSH_KNOWN_HOSTS` variable is created, in addition to the +[content of `.gitlab-ci.yml`](#ssh-keys-when-using-the-docker-executor) +above, here's what more you need to add: + +```yaml +before_script: + ## + ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the + ## following two lines. + ## + - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts + - chmod 644 ~/.ssh/known_hosts + + ## + ## Alternatively, use ssh-keyscan to scan the keys of your private server. + ## Replace example.com with your private server's domain name. Repeat that + ## command if you have more than one server to connect to. + ## + # - ssh-keyscan example.com >> ~/.ssh/known_hosts + # - chmod 644 ~/.ssh/known_hosts + + ## + ## You can optionally disable host key checking. Be aware that by adding that + ## you are susceptible to man-in-the-middle attacks. + ## WARNING: Use this only with the Docker executor, if you use it with shell + ## you will overwrite your user's SSH config. + ## + # - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config' +``` + +## Example project + +We have set up an [Example SSH Project](https://gitlab.com/gitlab-examples/ssh-private-key/) for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes is picked by a public runner and the job starts. diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 2e7ec58f048..b4cea48a362 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Triggering pipelines through the API **(CORE)** +# Triggering pipelines through the API **(FREE)** Triggers can be used to force a pipeline rerun of a specific `ref` (branch or tag) with an API call. @@ -17,7 +17,7 @@ The following methods of authentication are supported: - [Trigger token](#trigger-token) - [CI job token](#ci-job-token) -If using the `$CI_PIPELINE_SOURCE` [predefined environment variable](../variables/predefined_variables.md) +If using the `$CI_PIPELINE_SOURCE` [predefined CI/CD variable](../variables/predefined_variables.md) to limit which jobs run in a pipeline, the value could be either `pipeline` or `trigger`, depending on which trigger method is used. @@ -35,12 +35,12 @@ A unique trigger token can be obtained when [adding a new trigger](#adding-a-new WARNING: Passing plain text tokens in public projects is a security issue. Potential attackers can impersonate the user that exposed their trigger token publicly in -their `.gitlab-ci.yml` file. Use [variables](../variables/README.md#gitlab-cicd-environment-variables) +their `.gitlab-ci.yml` file. Use [CI/CD variables](../variables/README.md) to protect trigger tokens. ### CI job token -You can use the `CI_JOB_TOKEN` [variable](../variables/README.md#predefined-environment-variables) (used to authenticate +You can use the `CI_JOB_TOKEN` [CI/CD variable](../variables/README.md#predefined-cicd-variables) (used to authenticate with the [GitLab Container Registry](../../user/packages/container_registry/index.md)) in the following cases. #### When used with multi-project pipelines @@ -53,7 +53,7 @@ and it creates a dependent pipeline relation visible on the [pipeline graph](../multi_project_pipelines.md). For example: ```yaml -build_docs: +trigger_pipeline: stage: deploy script: - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=master "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" @@ -126,16 +126,14 @@ branches or tags. The `:id` of a project can be found by [querying the API](../../api/projects.md) or by visiting the **CI/CD** settings page which provides self-explanatory examples. -When a rerun of a pipeline is triggered, the information is exposed in the GitLab -UI under the **Jobs** page and the jobs are marked as triggered 'by API'. +When a rerun of a pipeline is triggered, jobs are marked as triggered `by API` in +**CI/CD > Jobs**. - - -You can see which trigger caused the rebuild by visiting the single job page. +You can see which trigger caused a job to run by visiting the single job page. A part of the trigger's token is exposed in the UI as you can see from the image below. - + By using cURL you can trigger a pipeline rerun with minimal effort, for example: @@ -146,7 +144,7 @@ curl --request POST \ "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" ``` -In this case, the project with ID `9` gets rebuilt on `master` branch. +In this case, the pipeline for the project with ID `9` runs on the `master` branch. Alternatively, you can pass the `token` and `ref` arguments in the query string: @@ -156,12 +154,12 @@ curl --request POST \ ``` You can also benefit by using triggers in your `.gitlab-ci.yml`. Let's say that -you have two projects, A and B, and you want to trigger a rebuild on the `master` +you have two projects, A and B, and you want to trigger a pipeline on the `master` branch of project B whenever a tag on project A is created. This is the job you need to add in project A's `.gitlab-ci.yml`: ```yaml -build_docs: +trigger_pipeline: stage: deploy script: - 'curl --request POST --form token=TOKEN --form ref=master "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline"' @@ -170,7 +168,7 @@ build_docs: ``` This means that whenever a new tag is pushed on project A, the job runs and the -`build_docs` job is executed, triggering a rebuild of project B. The +`trigger_pipeline` job is executed, triggering the pipeline for project B. The `stage: deploy` ensures that this job runs only after all jobs with `stage: test` complete successfully. @@ -187,6 +185,41 @@ You should pass `ref` as part of the URL, to take precedence over `ref` from the webhook body that designates the branch ref that fired the trigger in the source repository. Be sure to URL-encode `ref` if it contains slashes. +### Using webhook payload in the triggered pipeline + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31197) in GitLab 13.9. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-the-trigger_payload-variable). **(FREE SELF)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +If you trigger a pipeline by using a webhook, you can access the webhook payload with +the `TRIGGER_PAYLOAD` [predefined CI/CD variable](../variables/predefined_variables.md). +The payload is exposed as a [file-type variable](../variables/README.md#custom-cicd-variables-of-type-file), +so you can access the data with `cat $TRIGGER_PAYLOAD` or a similar command. + +#### Enable or disable the `TRIGGER_PAYLOAD` variable + +The `TRIGGER_PAYLOAD` CI/CD variable is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:ci_trigger_payload_into_pipeline) +``` + +To disable it: + +```ruby +Feature.disable(:ci_trigger_payload_into_pipeline) +``` + ## Making use of trigger variables You can pass any number of arbitrary variables in the trigger API call and they @@ -204,7 +237,7 @@ This information is also exposed in the UI. Please note that _values_ are only v Using trigger variables can be proven useful for a variety of reasons: - Identifiable jobs. Since the variable is exposed in the UI you can know - why the rebuild was triggered if you pass a variable that explains the + why the pipeline was triggered if you pass a variable that explains the purpose. - Conditional job processing. You can have conditional jobs that run whenever a certain variable is present. @@ -236,7 +269,7 @@ upload_package: - if [ -n "${UPLOAD_TO_S3}" ]; then make upload; fi ``` -You can then trigger a rebuild while you pass the `UPLOAD_TO_S3` variable +You can then trigger a pipeline while you pass the `UPLOAD_TO_S3` variable and the script of the `upload_package` job is run: ```shell @@ -247,7 +280,7 @@ curl --request POST \ "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" ``` -Trigger variables have the [highest priority](../variables/README.md#priority-of-environment-variables) +Trigger variables have the [highest priority](../variables/README.md#priority-of-cicd-variables) of all types of variables. ## Using cron to trigger nightly pipelines diff --git a/doc/ci/triggers/img/builds_page.png b/doc/ci/triggers/img/builds_page.png Binary files differdeleted file mode 100644 index 14d73b140f4..00000000000 --- a/doc/ci/triggers/img/builds_page.png +++ /dev/null diff --git a/doc/ci/triggers/img/trigger_single_build.png b/doc/ci/triggers/img/trigger_single_job.png Binary files differindex b760782afdc..b760782afdc 100644 --- a/doc/ci/triggers/img/trigger_single_build.png +++ b/doc/ci/triggers/img/trigger_single_job.png diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 9cda1f14b01..cddcf76236a 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -222,6 +222,29 @@ This also applies if the pipeline has not been created yet, or if you are waitin for an external CI service. If you don't use pipelines for your project, then you should disable **Pipelines must succeed** so you can accept merge requests. +### "The pipeline for this merge request did not complete. Push a new commit to fix the failure or check the troubleshooting documentation to see other possible actions." message + +This message is shown if the [merge request pipeline](merge_request_pipelines/index.md), +[merged results pipeline](merge_request_pipelines/pipelines_for_merged_results/index.md), +or [merge train pipeline](merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md) +has failed or been canceled. + +If a merge request pipeline or merged result pipeline was canceled or failed, you can: + +- Re-run the entire pipeline by clicking **Run pipeline** in the pipeline tab in the merge request. +- [Retry only the jobs that failed](pipelines/index.md#view-pipelines). If you re-run the entire pipeline, this is not necessary. +- Push a new commit to fix the failure. + +If the merge train pipeline has failed, you can: + +- Check the failure and determine if you can use the [`/merge` quick action](../user/project/quick_actions.md) to immediately add the merge request to the train again. +- Re-run the entire pipeline by clicking **Run pipeline** in the pipeline tab in the merge request, then add the merge request to the train again. +- Push a commit to fix the failure, then add the merge request to the train again. + +If the merge train pipeline was canceled before the merge request was merged, without a failure, you can: + +- Add it to the train again. + ## Pipeline warnings Pipeline configuration warnings are shown when you: @@ -239,6 +262,23 @@ To [prevent duplicate pipelines](yaml/README.md#prevent-duplicate-pipelines), us [`workflow: rules`](yaml/README.md#workflowrules) or rewrite your rules to control which pipelines can run. +### Console workaround if job using resource_group gets stuck + +```ruby +# find resource group by name +resource_group = Project.find_by_full_path('...').resource_groups.find_by(key: 'the-group-name') +busy_resources = resource_group.resources.where('build_id IS NOT NULL') + +# identify which builds are occupying the resource +# (I think it should be 1 as of today) +busy_resources.pluck(:build_id) + +# it's good to check why this build is holding the resource. +# Is it stuck? Has it been forcefully dropped by the system? +# free up busy resources +busy_resources.update_all(build_id: nil) +``` + ## How to get help If you are unable to resolve pipeline issues, you can get help from: diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index 1fec1f77bc3..ee060f33d01 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -67,8 +67,9 @@ execution time and the error output. ### Number of recent failures -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in GitLab 13.7. +> - [Introduced in Merge Requests](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in GitLab 13.7. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268249) in GitLab 13.8. +> - [Introduced in Test Reports](https://gitlab.com/gitlab-org/gitlab/-/issues/235525) in GitLab 13.9. If a test failed in the project's default branch in the last 14 days, a message like `Failed {n} time(s) in {default_branch} in the last 14 days` is displayed for that test. @@ -290,6 +291,22 @@ javascript: - junit.xml ``` +### Flutter / Dart example + +This example `.gitlab-ci.yml` file uses the [JUnit Report](https://pub.dev/packages/junitreport) package to convert the `flutter test` output into JUnit report XML format: + +```yaml +test: + stage: test + script: + - flutter test --machine | tojunit -o report.xml + artifacts: + when: always + reports: + junit: + - report.xml +``` + ## Viewing Unit test reports on GitLab > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24792) in GitLab 12.5 behind a feature flag (`junit_pipeline_view`), disabled by default. @@ -310,7 +327,12 @@ You can also retrieve the reports via the [GitLab API](../api/pipelines.md#get-a > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202114) in GitLab 13.0. > - It's deployed behind a feature flag, disabled by default. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enabling-the-junit-screenshots-feature). **(CORE ONLY)** +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enabling-the-junit-screenshots-feature). **(FREE SELF)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +When [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/6061) is complete, the attached file will be visible on the pipeline details page. If JUnit report format XML files contain an `attachment` tag, GitLab parses the attachment. @@ -322,9 +344,7 @@ Upload your screenshots as [artifacts](pipelines/job_artifacts.md#artifactsrepor </testcase> ``` -When [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/6061) is complete, the attached file will be visible on the pipeline details page. - -### Enabling the JUnit screenshots feature **(CORE ONLY)** +### Enabling the JUnit screenshots feature **(FREE SELF)** This feature comes with the `:junit_pipeline_screenshots_view` feature flag disabled by default. diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 5fca8e8c2b7..b6228e26175 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -5,20 +5,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD environment variables +# GitLab CI/CD variables -An environment variable is a dynamically-named value that can -affect the way running processes behave on an operating -system. +CI/CD variables are part of the environment in which [pipelines](../pipelines/index.md) +and jobs run. For example, you could: -Environment variables are part of the environment in which a process runs. -For example, a running process could: - -- Use the value of a `TEMP` environment variable to know the correct location - to store temporary files. +- Use the value of a `TEMP` variable to know the correct location to store temporary files. - Use a `DATABASE_URL` variable for the URL to a database that can be reused in different scripts. -Variables are useful for customizing your jobs in GitLab CI/CD. +Variables can be used to customize your jobs in [GitLab CI/CD](../README.md). When you use variables, you don't have to hard-code values. > For more information about advanced use of GitLab CI/CD: @@ -28,28 +23,27 @@ When you use variables, you don't have to hard-code values. > - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Learn how the Cloud Native Computing Foundation (CNCF) [eliminates the complexity](https://about.gitlab.com/customers/cncf/) > of managing projects across many cloud providers with GitLab CI/CD. -## Predefined environment variables +## Predefined CI/CD variables -GitLab CI/CD has a [default set of predefined variables](predefined_variables.md) +GitLab CI/CD has a [default set of predefined CI/CD variables](predefined_variables.md) that you can use without any additional specification. You can call issue numbers, user names, branch names, pipeline and commit IDs, and much more. -Predefined environment variables are provided by GitLab -for the local environment of the runner. +Predefined variables are provided by GitLab for the local environment of the runner. GitLab reads the `.gitlab-ci.yml` file and sends the information to the runner, where the variables are exposed. The runner then runs the script commands. -### Use predefined environment variables +### Use predefined CI/CD variables -You can choose one of the existing predefined variables +You can choose one of the existing predefined CI/CD variables to be output by the runner. This example shows how to output a job's stage by using the predefined variable `CI_JOB_STAGE`. In your `.gitlab-ci.yml` file, call the variable from your script. Ensure -you use the correct [syntax](#syntax-of-environment-variables-in-job-scripts). +you use the correct [syntax](#syntax-of-cicd-variables-in-job-scripts). ```yaml test_variable: @@ -63,14 +57,15 @@ job `test_variable`, which is `test`:  -## Custom environment variables +## Custom CI/CD variables -When you need a specific custom environment variable, you can +When you need a specific custom variable, you can [set it up in the UI](#create-a-custom-variable-in-the-ui), in [the API](../../api/project_level_variables.md), or directly [in the `.gitlab-ci.yml` file](#create-a-custom-variable-in-gitlab-ciyml). The variables are used by the runner any time the pipeline runs. -You can also [override variable values manually for a specific pipeline](../jobs/index.md#specifying-variables-when-running-manual-jobs). +You can also [override variable values manually for a specific pipeline](../jobs/index.md#specifying-variables-when-running-manual-jobs), +or have them [prefilled in manual pipelines](../pipelines/index.md#prefill-variables-in-manual-pipelines). There are two types of variables: **Variable** and **File**. You cannot set types in the `.gitlab-ci.yml` file, but you can set them in the UI and API. @@ -96,7 +91,7 @@ For more details, see [`.gitlab-ci.yml` defined variables](#gitlab-ciyml-defined ### Create a custom variable in the UI -From within the UI, you can add or update custom environment variables: +From the UI, you can add or update custom variables: 1. Go to your project's **Settings > CI/CD** and expand the **Variables** section. 1. Click the **Add Variable** button. In the **Add variable** modal, fill in the details: @@ -104,7 +99,7 @@ From within the UI, you can add or update custom environment variables: - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: No limitations. - **Type**: `File` or `Variable`. - - **Environment scope**: `All`, or specific environments. + - **Environment scope**: `All`, or specific [environments](../environments/index.md). - **Protect variable** (Optional): If selected, the variable is only available in pipelines that run on protected branches or tags. - **Mask variable** (Optional): If selected, the variable's **Value** is masked in job logs. The variable fails to save if the value does not meet the [masking requirements](#masked-variable-requirements). @@ -145,7 +140,7 @@ build: - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" ``` -### Custom environment variables of type Variable +### Custom CI/CD variables of type Variable > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/46806) in GitLab 11.11. @@ -155,7 +150,7 @@ that uses the key for the name and the value for the value. There are [some predefined variables](#custom-variables-validated-by-gitlab) of this type, which may be further validated. They appear when you add or update a variable in the UI. -### Custom environment variables of type File +### Custom CI/CD variables of type File > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/46806) in GitLab 11.11. @@ -207,10 +202,11 @@ The value of the variable must: - Be in a single line. - Be at least 8 characters long. -- Not be a predefined or custom environment variable. -- Consist only of characters from the Base64 alphabet (RFC4648). - [In GitLab 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63043) - and newer, `@` and `:` are also valid values. +- Not be a predefined or custom CI/CD variable. +- Consist only of: + - Characters from the Base64 alphabet (RFC4648). + - The `@` and `:` characters ([In GitLab 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63043) and later). + - The `.` character ([In GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29022) and later). You can't mask variables that don't meet these requirements. @@ -247,7 +243,7 @@ WARNING: When you store credentials, there are security implications. If you are using AWS keys, for example, follow their [best practices](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). -## Syntax of environment variables in job scripts +## Syntax of CI/CD variables in job scripts All variables are set as environment variables in the build environment, and they are accessible with normal methods that are used to access such variables. @@ -263,7 +259,7 @@ To access environment variables, use the syntax for your runner's [shell](https: ### Bash -To access environment variables in **bash**, prefix the variable name with (`$`): +To access environment variables in **bash**, prefix the CI/CD variable name with (`$`): ```yaml job_name: @@ -274,8 +270,8 @@ job_name: ### PowerShell To access variables in a **Windows PowerShell** environment, including system set -environment variables, prefix the variable name with (`$env:`). Environment variables -set by GitLab CI can also be accessed by prefixing the variable name with (`$`) with +environment variables, prefix the variable name with (`$env:`). GitLab CI/CD variables +can also be accessed by prefixing the variable name with (`$`) with [GitLab Runner 1.0.0](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/68) and later. @@ -371,9 +367,8 @@ export GITLAB_USER_ID="42" ## `.gitlab-ci.yml` defined variables -You can add variables that are set in the build environment to `.gitlab-ci.yml`. -These variables are saved in the repository, and they -are meant to store non-sensitive project configuration, like `RAILS_ENV` or +You can add CI/CD variables to `.gitlab-ci.yml`. These variables are saved in the repository, +and they are meant to store non-sensitive project configuration, like `RAILS_ENV` or `DATABASE_URL`. For example, if you set the variable below globally (not inside a job), it is @@ -406,13 +401,20 @@ script: - 'eval $LS_CMD' # will execute 'ls -al $TMP_DIR' ``` -## Group-level environment variables +Use the [`value` and `description`](../yaml/README.md#prefill-variables-in-manual-pipelines) +keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) +when [running a pipeline manually](../pipelines/index.md#run-a-pipeline-manually): + +## Group-level CI/CD variables > Introduced in GitLab 9.4. -You can define per-project or per-group variables that are set in the pipeline environment. Group-level variables are stored out of the repository (not in `.gitlab-ci.yml`). They are securely passed to GitLab Runner, which makes them available during a pipeline run. +You can define per-project or per-group variables that are set in the pipeline environment. +Group-level variables are stored out of the repository (not in `.gitlab-ci.yml`). +They are securely passed to GitLab Runner, which makes them available during a pipeline run. -We recommend using group environment variables to store secrets (like passwords, SSH keys, and credentials) for Premium users who: +We recommend using group variables to store secrets (like passwords, SSH keys, and +credentials) for users who: - Do **not** use an external key store. - Use the GitLab [integration with HashiCorp Vault](../secrets/index.md). @@ -430,7 +432,7 @@ After you set them, they are available for all subsequent pipelines. Any group-l  -## Instance-level CI/CD environment variables +## Instance-level CI/CD variables > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14108) in GitLab 13.0. @@ -471,12 +473,12 @@ To enable it: Feature.enable(:instance_variables_ui) ``` -## Inherit environment variables +## Inherit CI/CD variables > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22638) in GitLab 13.0 behind a disabled [feature flag](../../administration/feature_flags.md): `ci_dependency_variables`. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217834) in GitLab 13.1. -You can inherit environment variables from dependent jobs. +You can inherit CI/CD variables from dependent jobs. This feature makes use of the [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) report feature. @@ -519,7 +521,7 @@ deploy: artifacts: true ``` -## Priority of environment variables +## Priority of CI/CD variables Variables of different types can take precedence over other variables, depending on where they are defined. @@ -528,14 +530,14 @@ The order of precedence for variables is (from highest to lowest): 1. [Trigger variables](../triggers/README.md#making-use-of-trigger-variables), [scheduled pipeline variables](../pipelines/schedules.md#using-variables), and [manual pipeline run variables](#override-a-variable-by-manually-running-a-pipeline). -1. Project-level [variables](#custom-environment-variables) or [protected variables](#protect-a-custom-variable). -1. Group-level [variables](#group-level-environment-variables) or [protected variables](#protect-a-custom-variable). -1. Instance-level [variables](#instance-level-cicd-environment-variables) or [protected variables](#protect-a-custom-variable). -1. [Inherited environment variables](#inherit-environment-variables). +1. Project-level [variables](#custom-cicd-variables) or [protected variables](#protect-a-custom-variable). +1. Group-level [variables](#group-level-cicd-variables) or [protected variables](#protect-a-custom-variable). +1. Instance-level [variables](#instance-level-cicd-variables) or [protected variables](#protect-a-custom-variable). +1. [Inherited CI/CD variables](#inherit-cicd-variables). 1. YAML-defined [job-level variables](../yaml/README.md#variables). 1. YAML-defined [global variables](../yaml/README.md#variables). -1. [Deployment variables](#deployment-environment-variables). -1. [Predefined environment variables](predefined_variables.md). +1. [Deployment variables](#deployment-variables). +1. [Predefined CI/CD variables](predefined_variables.md). For example, if you define: @@ -549,7 +551,7 @@ variables take precedence over those defined in `.gitlab-ci.yml`. Variable names are limited by the underlying shell used to execute scripts (see [available shells](https://docs.gitlab.com/runner/shells/index.html). Each shell has its own unique set of reserved variable names. -Keep in mind the [scope of environment variables](where_variables_can_be_used.md) to ensure a variable is defined in the scope in which you wish to use it. +Keep in mind the [scope of CI/CD variables](where_variables_can_be_used.md) to ensure a variable is defined in the scope in which you wish to use it. ## Where variables can be used @@ -557,14 +559,14 @@ Keep in mind the [scope of environment variables](where_variables_can_be_used.md ## Advanced use -### Limit the environment scopes of environment variables +### Limit the environment scopes of CI/CD variables You can limit the environment scope of a variable by [defining which environments](../environments/index.md) it can be available for. To learn more about scoping environments, see [Scoping environments with specs](../environments/index.md#scoping-environments-with-specs). -### Deployment environment variables +### Deployment variables > Introduced in GitLab 8.15. @@ -582,11 +584,10 @@ An example integration that defines deployment variables is the > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49056) in GitLab 11.7. You can configure [Auto DevOps](../../topics/autodevops/index.md) to -pass CI variables to the running application by prefixing the key of the +pass CI/CD variables to the running application by prefixing the key of the variable with `K8S_SECRET_`. -These [prefixed -variables](../../topics/autodevops/customize.md#application-secret-variables) are +These [prefixed variables](../../topics/autodevops/customize.md#application-secret-variables) are then available as environment variables on the running application container. @@ -603,9 +604,9 @@ You can override the value of a variable when: 1. Manually playing a job via the UI. 1. Using [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). 1. Manually triggering pipelines with [the API](../triggers/README.md#making-use-of-trigger-variables). -1. Passing variables to a [downstream pipeline](../multi_project_pipelines.md#passing-variables-to-a-downstream-pipeline). +1. Passing variables to a [downstream pipeline](../multi_project_pipelines.md#passing-cicd-variables-to-a-downstream-pipeline). -These pipeline variables declared in these events take [priority over other variables](#priority-of-environment-variables). +These pipeline variables declared in these events take [priority over other variables](#priority-of-cicd-variables). #### Restrict who can override variables @@ -618,7 +619,7 @@ variables, an `Insufficient permissions to set pipeline variables` error occurs. The setting is `disabled` by default. -If you [store your CI configurations in a different repository](../../ci/pipelines/settings.md#custom-ci-configuration-path), +If you [store your CI/CD configurations in a different repository](../../ci/pipelines/settings.md#custom-cicd-configuration-path), use this setting for strict control over all aspects of the environment the pipeline runs in. @@ -642,7 +643,7 @@ value for this specific pipeline.  -## Environment variables expressions +## CI/CD variable expressions > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37397) in GitLab 10.7 for [the `only` and `except` CI keywords](../yaml/README.md#onlyexcept-advanced) > - [Expanded](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3 with [the `rules` keyword](../yaml/README.md#rules) @@ -676,7 +677,7 @@ when `except` is being used, a job is not created. This follows the usual rules for [`only` / `except` policies](../yaml/README.md#onlyexcept-advanced). -### Syntax of environment variable expressions +### Syntax of CI/CD variable expressions Below you can find supported syntax reference. @@ -777,7 +778,7 @@ so `&&` is evaluated before `||`. > - It's deployed behind a feature flag, enabled by default. > - It's enabled on GitLab.com. > - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-parenthesis-support-for-variables). **(CORE ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-parenthesis-support-for-variables). **(FREE SELF)** It is possible to use parentheses to group conditions. Parentheses have the highest precedence of all operators. Expressions enclosed in parentheses are evaluated first, @@ -793,7 +794,7 @@ Examples: - `($VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/) && $VARIABLE3` - `$CI_COMMIT_BRANCH == "my-branch" || (($VARIABLE1 == "thing" || $VARIABLE2 == "thing") && $VARIABLE3)` -##### Enable or disable parenthesis support for variables **(CORE ONLY)** +##### Enable or disable parenthesis support for variables **(FREE SELF)** The feature is deployed behind a feature flag that is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) @@ -1076,7 +1077,7 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ## Video walkthrough of a working example -The [Managing the Complex Configuration Data Management Monster Using GitLab](https://www.youtube.com/watch?v=v4ZOJ96hAck) video is a walkthrough of the [Complex Config Data Monorepo](https://gitlab.com/guided-explorations/config-data-top-scope/config-data-subscope/config-data-monorepo) working example project. It explains how multiple levels of group CI/CD variables can be combined with environment-scoped project variables for complex configuration of application builds or deployments. +The [Managing the Complex Configuration Data Management Monster Using GitLab](https://www.youtube.com/watch?v=v4ZOJ96hAck) video is a walkthrough of the [Complex Configuration Data Monorepo](https://gitlab.com/guided-explorations/config-data-top-scope/config-data-subscope/config-data-monorepo) working example project. It explains how multiple levels of group CI/CD variables can be combined with environment-scoped project variables for complex configuration of application builds or deployments. The example can be copied to your own group or instance for testing. More details on what other GitLab CI patterns are demonstrated are available at the project page. diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 701fe33b53f..8d2df82a212 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -5,25 +5,25 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Predefined environment variables reference +# Predefined variables reference For an introduction on this subject, read through the -[getting started with environment variables](README.md) document. +[CI/CD variables](README.md) document. -Some of the predefined environment variables are available only if a minimum +Some of the predefined variables are available only if a minimum version of [GitLab Runner](https://docs.gitlab.com/runner/) is used. Consult the table below to find the version of GitLab Runner that's required. You can add a command to your `.gitlab-ci.yml` file to [output the values of all variables available for a job](README.md#list-all-environment-variables). -Kubernetes-specific environment variables are detailed in the +Kubernetes-specific variables are detailed in the [Kubernetes deployment variables](../../user/project/clusters/index.md#deployment-variables) section. | Variable | GitLab | Runner | Description | |-----------------------------------------------|--------|--------|-------------| -| `CHAT_CHANNEL` | 10.6 | all | Source chat channel which triggered the [ChatOps](../chatops/README.md) command. | -| `CHAT_INPUT` | 10.6 | all | Additional arguments passed in the [ChatOps](../chatops/README.md) command. | +| `CHAT_CHANNEL` | 10.6 | all | Source chat channel which triggered the [ChatOps](../chatops/index.md) command. | +| `CHAT_INPUT` | 10.6 | all | Additional arguments passed in the [ChatOps](../chatops/index.md) command. | | `CI` | all | 0.4 | Mark that job is executed in CI environment. | | `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL. | | `CI_BUILDS_DIR` | all | 11.10 | Top-level directory where builds are executed. | @@ -145,3 +145,4 @@ Kubernetes-specific environment variables are detailed in the | `GITLAB_USER_ID` | 8.12 | all | The ID of the user who started the job. | | `GITLAB_USER_LOGIN` | 10.0 | all | The login username of the user who started the job. | | `GITLAB_USER_NAME` | 10.0 | all | The real name of the user who started the job. | +| `TRIGGER_PAYLOAD` | 13.9 | all | This variable is available when a pipeline is [triggered with a webhook](../triggers/README.md#using-webhook-payload-in-the-triggered-pipeline) | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index e84714f2a46..d057fe9c4cc 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -124,7 +124,7 @@ They are: - Script execution shell. - Not supported: - For definitions where the ["Expansion place"](#gitlab-ciyml-file) is GitLab. - - In the `only` and `except` [variables expressions](README.md#environment-variables-expressions). + - In the `only` and `except` [variables expressions](README.md#cicd-variable-expressions). Some of the persisted variables contain tokens and cannot be used by some definitions due to security reasons. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 19b8f0f1c89..2abfbd3a5b4 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -5,11 +5,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD pipeline configuration reference +<!-- markdownlint-disable MD044 --> +<!-- vale gitlab.Spelling = NO --> +# Keyword reference for the .gitlab-ci.yml file +<!-- vale gitlab.Spelling = YES --> +<!-- markdownlint-enable MD044 --> This document lists the configuration options for your GitLab `.gitlab-ci.yml` file. -- For a quick introduction to GitLab CI/CD, follow the [quick start guide](../quick_start/README.md). +- For a quick introduction to GitLab CI/CD, follow the [quick start guide](../quick_start/index.md). - For a collection of examples, see [GitLab CI/CD Examples](../examples/README.md). - To view a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml). @@ -26,7 +30,7 @@ The keywords available for jobs are: |:---------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [`script`](#script) | Shell script that is executed by a runner. | | [`after_script`](#after_script) | Override a set of commands that are executed after job. | -| [`allow_failure`](#allow_failure) | Allow job to fail. A failed job does not cause the pipeline to fail. | +| [`allow_failure`](#allow_failure) | Allow job to fail. A failed job does not cause the pipeline to fail. | | [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, and `artifacts:reports`. | | [`before_script`](#before_script) | Override a set of commands that are executed before job. | | [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, `cache:when`, and `cache:policy`. | @@ -44,7 +48,7 @@ The keywords available for jobs are: | [`release`](#release) | Instructs the runner to generate a [Release](../../user/project/releases/index.md) object. | | [`resource_group`](#resource_group) | Limit job concurrency. | | [`retry`](#retry) | When and how many times a job can be auto-retried in case of a failure. | -| [`rules`](#rules) | List of conditions to evaluate and determine selected attributes of a job, and whether or not it's created. May not be used alongside `only`/`except`. | +| [`rules`](#rules) | List of conditions to evaluate and determine selected attributes of a job, and whether or not it's created. | | [`services`](#services) | Use Docker services images. Also available: `services:name`, `services:alias`, `services:entrypoint`, and `services:command`. | | [`stage`](#stage) | Defines a job stage (default: `test`). | | [`tags`](#tags) | List of tags that are used to select a runner. | @@ -55,8 +59,8 @@ The keywords available for jobs are: ### Unavailable names for jobs -Each job must have a unique name, but there are a few **reserved `keywords` that -can't be used as job names**: +Each job must have a unique name, but there are a few reserved `keywords` that +can't be used as job names: - `image` - `services` @@ -70,7 +74,7 @@ can't be used as job names**: ### Reserved keywords -If you get a validation error when using specific values (for example, `true` or `false`), try to: +If you get a validation error when you use specific values (for example, `true` or `false`), try to: - Quote them. - Change them to a different form. For example, `/bin/true`. @@ -223,7 +227,7 @@ stages: If any job fails, the pipeline is marked as `failed` and jobs in later stages do not start. Jobs in the current stage are not stopped and continue to run. -If no `stages` are defined in `.gitlab-ci.yml`, then `build`, `test` and `deploy` +If no `stages` are defined in the `.gitlab-ci.yml` file, then `build`, `test` and `deploy` are the default pipeline stages. If a job does not specify a [`stage`](#stage), the job is assigned the `test` stage. @@ -339,10 +343,10 @@ include: > - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. > - Available for Starter, Premium, and Ultimate in GitLab 10.6 and later. -> - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Core in 11.4. +> - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Free in 11.4. Use the `include` keyword to include external YAML files in your CI/CD configuration. -You can break down one long `gitlab-ci.yml` into multiple files to increase readability, +You can break down one long `gitlab-ci.yml` file into multiple files to increase readability, or reduce duplication of the same configuration in multiple places. You can also store template files in a central repository and `include` them in projects. @@ -363,33 +367,27 @@ use the [`extends` keyword](#extends). | [`remote`](#includeremote) | Include a file from a remote URL. Must be publicly accessible. | | [`template`](#includetemplate) | Include templates that are provided by GitLab. | -`.gitlab-ci.yml` configuration included by all methods is evaluated at pipeline creation. +The `.gitlab-ci.yml` file configuration included by all methods is evaluated when the pipeline is created. The configuration is a snapshot in time and persisted in the database. Any changes to -referenced `.gitlab-ci.yml` configuration is not reflected in GitLab until the next pipeline is created. +the referenced `.gitlab-ci.yml` file configuration is not reflected in GitLab until the next pipeline is created. The files defined by `include` are: -- Deep merged with those in `.gitlab-ci.yml`. -- Always evaluated first and merged with the content of `.gitlab-ci.yml`, +- Deep merged with those in the `.gitlab-ci.yml` file. +- Always evaluated first and merged with the content of the `.gitlab-ci.yml` file, regardless of the position of the `include` keyword. NOTE: Use merging to customize and override included CI/CD configurations with local -definitions. Local definitions in `.gitlab-ci.yml` override included definitions. +configurations. Local configurations in the `.gitlab-ci.yml` file override included configurations. -#### Variables with `include` +#### Variables with `include` **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/284883) in GitLab 13.8. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-includepredefined-project-variables). **(CORE ONLY)** - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294294) in GitLab 13.9. You can [use some predefined variables in `include` sections](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) -in your `.gitlab-ci.yml`: +in your `.gitlab-ci.yml` file: ```yaml include: @@ -398,35 +396,15 @@ include: ``` For an example of how you can include these predefined variables, and their impact on CI jobs, -see the following [CI variable demo](https://youtu.be/4XR8gw3Pkos). - -##### Enable or disable include:predefined-project-variables **(CORE ONLY)** - -Use of predefined project variables in `include` section of `.gitlab-ci.yml` is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can enable it. - -To enable it: - -```ruby -Feature.enable(:variables_in_include_section_ci) -``` - -To disable it: - -```ruby -Feature.disable(:variables_in_include_section_ci) -``` +see the following [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos). #### `include:local` -`include:local` includes a file from the same repository as `.gitlab-ci.yml`. +`include:local` includes a file that is in the same repository as the `.gitlab-ci.yml` file. It's referenced with full paths relative to the root directory (`/`). -You can only use files that are tracked by Git on the same branch -your configuration file is on. If you `include:local`, make -sure that both `.gitlab-ci.yml` and the local file are on the same branch. +If you use `include:local`, make sure that both the `.gitlab-ci.yml` file and the local file +are on the same branch. You can't include local files through Git submodules paths. @@ -440,14 +418,14 @@ include: - local: '/templates/.gitlab-ci-template.yml' ``` -Local includes can be used as a replacement for symbolic links that are not followed. - This can be defined as a short local include: ```yaml include: '.gitlab-ci-production.yml' ``` +Use local includes instead of symbolic links. + #### `include:file` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53903) in GitLab 11.7. @@ -501,24 +479,23 @@ include: #### `include:remote` -`include:remote` can be used to include a file from a different location, -using HTTP/HTTPS, referenced by the full URL. The remote file must be -publicly accessible by a GET request, because authentication schemas -in the remote URL are not supported. For example: +Use `include:remote` with a full URL to include a file from a different location. +The remote file must be publicly accessible by an HTTP/HTTPS `GET` request, because +authentication in the remote URL is not supported. For example: ```yaml include: - - remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml' + - remote: 'https://gitlab.com/example-project/-/raw/master/.gitlab-ci.yml' ``` -All [nested includes](#nested-includes) are executed without context as a public user, +All [nested includes](#nested-includes) execute without context as a public user, so you can only `include` public projects or templates. #### `include:template` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53445) in GitLab 11.7. -`include:template` can be used to include `.gitlab-ci.yml` templates that are +Use `include:template` to include `.gitlab-ci.yml` templates that are [shipped with GitLab](https://gitlab.com/gitlab-org/gitlab/tree/master/lib/gitlab/ci/templates). For example: @@ -566,7 +543,7 @@ Used to specify [a Docker image](../docker/using_docker_images.md#what-is-an-ima For: - Usage examples, see [Define `image` and `services` from `.gitlab-ci.yml`](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml). -- Detailed usage information, refer to [Docker integration](../docker/README.md) documentation. +- Detailed usage information, refer to [Docker integration](../docker/index.md) documentation. #### `image:name` @@ -587,8 +564,8 @@ Used to specify a [service Docker image](../docker/using_docker_images.md#what-i For: - Usage examples, see [Define `image` and `services` from `.gitlab-ci.yml`](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml). -- Detailed usage information, refer to [Docker integration](../docker/README.md) documentation. -- For example services, see [GitLab CI/CD Services](../services/README.md). +- Detailed usage information, refer to [Docker integration](../docker/index.md) documentation. +- For example services, see [GitLab CI/CD Services](../services/index.md). ##### `services:name` @@ -677,7 +654,7 @@ job: #### `before_script` -`before_script` is used to define an array of commands that should run before each job, +Use `before_script` to define an array of commands that should run before each job, but after [artifacts](#artifacts) are restored. Scripts specified in `before_script` are concatenated with any scripts specified @@ -705,7 +682,7 @@ You can use [YAML anchors with `before_script`](#yaml-anchors-for-scripts). #### `after_script` -`after_script` is used to define an array of commands that run after each job, +Use `after_script` to define an array of commands that run after each job, including failed jobs. If a job times out or is cancelled, the `after_script` commands are not executed. @@ -756,7 +733,7 @@ You can use special syntax in [`script`](README.md#script) sections to: `stage` is defined per-job and relies on [`stages`](#stages), which is defined globally. Use `stage` to define which stage a job runs in, and jobs of the same -`stage` are executed in parallel (subject to [certain conditions](#using-your-own-runners)). For example: +`stage` are executed in parallel (subject to [certain conditions](#use-your-own-runners)). For example: ```yaml stages: @@ -789,7 +766,7 @@ job 5: script: make something useful at the end of pipeline ``` -#### Using your own runners +#### Use your own runners When you use your own runners, each runner runs only one job at a time by default. Jobs can run in parallel if they run on different runners. @@ -811,7 +788,7 @@ User-defined stages are executed after `.pre` and before `.post`. A pipeline is not created if all jobs are in `.pre` or `.post` stages. -The order of `.pre` and `.post` can't be changed, even if defined out of order in `.gitlab-ci.yml`. +The order of `.pre` and `.post` can't be changed, even if defined out of order in the `.gitlab-ci.yml` file. For example, the following are equivalent configuration: - Configured in order: @@ -846,11 +823,16 @@ For example, the following are equivalent configuration: > Introduced in GitLab 11.3. -`extends` defines entry names that a job that uses `extends` -inherits from. +Use `extends` to reuse configuration sections. It's an alternative to [YAML anchors](#anchors) +and is a little more flexible and readable. You can use `extends` to reuse configuration +from [included configuration files](#use-extends-and-include-together). + +In this example, the `rspec` job uses the configuration from the `.tests` template job. +GitLab: -It's an alternative to using [YAML anchors](#anchors) and is a little -more flexible and readable: +- Performs a reverse deep merge based on the keys. +- Merges the `.tests` content with the `rspec` job. +- Doesn't merge the values of the keys. ```yaml .tests: @@ -868,13 +850,7 @@ rspec: - $RSPEC ``` -In the example above, the `rspec` job inherits from the `.tests` template job. -GitLab performs a reverse deep merge based on the keys. GitLab: - -- Merges the `rspec` contents into `.tests` recursively. -- Doesn't merge the values of the keys. - -The result is this `rspec` job, where `script: rake test` is overwritten by `script: rake rspec`: +The result is this `rspec` job: ```yaml rspec: @@ -887,10 +863,8 @@ rspec: - $RSPEC ``` -If you do want to include the `rake test`, see [`before_script`](#before_script) or [`after_script`](#after_script). - `.tests` in this example is a [hidden job](#hide-jobs), but it's -possible to inherit from regular jobs as well. +possible to extend configuration from regular jobs as well. `extends` supports multi-level inheritance. You should avoid using more than three levels, but you can use as many as eleven. The following example has two levels of inheritance: @@ -979,51 +953,51 @@ rspec: Note that in the example above: -- `variables` sections have been merged but that `URL: "http://my-url.internal"` -has been overwritten by `URL: "http://docker-url.internal"`. -- `tags: ['production']` has been overwritten by `tags: ['docker']`. -- `script` has not been merged but rather `script: ['echo "Hello world!"']` has - been overwritten by `script: ['rake rspec']`. Arrays can be - merged using [YAML anchors](#anchors). +- The `variables` sections merge, but `URL: "http://docker-url.internal"` overwrites `URL: "http://my-url.internal"`. +- `tags: ['docker']` overwrites `tags: ['production']`. +- `script` does not merge, but `script: ['rake rspec']` overwrites + `script: ['echo "Hello world!"']`. You can use [YAML anchors](#anchors) to merge arrays. -#### Using `extends` and `include` together +#### Use `extends` and `include` together -`extends` works across configuration files combined with `include`. +To reuse configuration from different configuration files, +combine `extends` and [`include`](#include). -For example, if you have a local `included.yml` file: +In this example, a `script` is defined in the `included.yml` file. +Then, in the `.gitlab-ci.yml` file, you use `extends` to refer +to the contents of the `script`: -```yaml -.template: - script: - - echo Hello! -``` +- `included.yml`: -Then, in `.gitlab-ci.yml`: + ```yaml + .template: + script: + - echo Hello! + ``` -```yaml -include: included.yml +- `.gitlab-ci.yml`: -useTemplate: - image: alpine - extends: .template -``` + ```yaml + include: included.yml -This example runs a job called `useTemplate` that runs `echo Hello!` as defined in -the `.template` job, and uses the `alpine` Docker image as defined in the local job. + useTemplate: + image: alpine + extends: .template + ``` ### `rules` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3. -The `rules` keyword can be used to include or exclude jobs in pipelines. +Use the `rules` keyword to include or exclude jobs in pipelines. Rules are evaluated *in order* until the first match. When matched, the job is either included or excluded from the pipeline, depending on the configuration. If included, the job also has [certain attributes](#rules-attributes) added to it. -`rules` replaces [`only/except`](#onlyexcept-basic) and can't be used in conjunction with it. -If you attempt to use both keywords in the same job, the linter returns a +`rules` replaces [`only/except`](#onlyexcept-basic) and they can't be used together +in the same job. If you configure one job to use both keywords, the linter returns a `key may not be used with rules` error. #### Rules attributes @@ -1078,7 +1052,7 @@ The job is not added to the pipeline: `when: always`. - If a rule matches, and has `when: never` as the attribute. -For example, using `if` clauses to strictly limit when jobs run: +This example uses `if` to strictly limit when jobs run: ```yaml job: @@ -1090,8 +1064,6 @@ job: - if: '$CI_PIPELINE_SOURCE == "schedule"' ``` -In this example: - - If the pipeline is for a merge request, the first rule matches, and the job is added to the [merge request pipeline](../merge_request_pipelines/index.md) with attributes of: @@ -1165,7 +1137,7 @@ There are multiple ways to avoid duplicate pipelines: or push (branch) pipelines only. - Rewrite the rules to run the job only in very specific cases, - and avoid using a final `when:` rule: + and avoid a final `when:` rule: ```yaml job: @@ -1240,13 +1212,13 @@ or excluded from a pipeline. In plain English, `if` rules can be interpreted as `rules:if` differs slightly from `only:variables` by accepting only a single expression string per rule, rather than an array of them. Any set of expressions to be evaluated can be [conjoined into a single expression](../variables/README.md#conjunction--disjunction) -by using `&&` or `||`, and the [variable matching operators (`==`, `!=`, `=~` and `!~`)](../variables/README.md#syntax-of-environment-variable-expressions). +by using `&&` or `||`, and the [variable matching operators (`==`, `!=`, `=~` and `!~`)](../variables/README.md#syntax-of-cicd-variable-expressions). -Unlike variables in [`script`](../variables/README.md#syntax-of-environment-variables-in-job-scripts) +Unlike variables in [`script`](../variables/README.md#syntax-of-cicd-variables-in-job-scripts) sections, variables in rules expressions are always formatted as `$VARIABLE`. -`if:` clauses are evaluated based on the values of [predefined environment variables](../variables/predefined_variables.md) -or [custom environment variables](../variables/README.md#custom-environment-variables). +`if:` clauses are evaluated based on the values of [predefined CI/CD variables](../variables/predefined_variables.md) +or [custom CI/CD variables](../variables/README.md#custom-cicd-variables). For example: @@ -1281,8 +1253,8 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | Value | Description | |-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). | -| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/README.md) command. | -| `external` | When using CI services other than GitLab. | +| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. | +| `external` | When you use CI services other than GitLab. | | `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | | `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). | | `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | @@ -1331,7 +1303,7 @@ Other commonly used variables for `if` clauses: branch (usually `master`). Use when you want to have the same configuration in multiple projects with different default branches. - `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression. -- `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/README.md#custom-environment-variables) +- `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/README.md#custom-cicd-variables) `CUSTOM_VARIABLE` does **not** match a regular expression. - `if: '$CUSTOM_VARIABLE == "value1"'`: If the custom variable `CUSTOM_VARIABLE` is exactly `value1`. @@ -1387,7 +1359,7 @@ if there is no `if:` statement that limits the job to branch or merge request pi > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34272) in GitLab 13.6. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.7. -Environment variables can be used in `rules:changes` expressions to determine when +CI/CD variables can be used in `rules:changes` expressions to determine when to add jobs to a pipeline: ```yaml @@ -1400,7 +1372,7 @@ docker build: - $DOCKERFILES_DIR/* ``` -The `$` character can be used for both variables and paths. For example, if the +You can use The `$` character for both variables and paths. For example, if the `$DOCKERFILES_DIR` variable exists, its value is used. If it does not exist, the `$` is interpreted as being part of a path. @@ -1409,7 +1381,9 @@ The `$` character can be used for both variables and paths. For example, if the > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4. `exists` accepts an array of paths and matches if any of these paths exist -as files in the repository: +as files in the repository. + +In this example, `job` runs if a `Dockerfile` exists anywhere in the repository: ```yaml job: @@ -1419,7 +1393,9 @@ job: - Dockerfile ``` -You can also use glob patterns to match multiple files in any directory in the repository: +Paths are relative to the project directory (`$CI_PROJECT_DIR`) and can't directly link outside it. + +You can use glob patterns to match multiple files in any directory in the repository: ```yaml job: @@ -1429,15 +1405,17 @@ job: - spec/**.rb ``` -For performance reasons, using `exists` with patterns is limited to 10,000 -checks. After the 10,000th check, rules with patterned globs always match. +Glob patterns are interpreted with Ruby [`File.fnmatch`](https://docs.ruby-lang.org/en/2.7.0/File.html#method-c-fnmatch) +with the flags `File::FNM_PATHNAME | File::FNM_DOTMATCH | File::FNM_EXTGLOB`. + +For performance reasons, GitLab matches a maximum of 10,000 `exists` patterns. After the 10,000th check, rules with patterned globs always match. #### `rules:allow_failure` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30235) in GitLab 12.8. You can use [`allow_failure: true`](#allow_failure) in `rules:` to allow a job to fail, or a manual job to -wait for action, without stopping the pipeline itself. All jobs using `rules:` default to `allow_failure: false` +wait for action, without stopping the pipeline itself. All jobs that use `rules:` default to `allow_failure: false` if `allow_failure:` is not defined. The rule-level `rules:allow_failure` option overrides the job-level @@ -1462,7 +1440,7 @@ In this example, if the first rule matches, then the job has `when: manual` and > - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/289803) on GitLab 13.8. > - It's enabled on GitLab.com. > - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-rulesvariables). **(CORE ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-rulesvariables). **(FREE SELF)** WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -1487,7 +1465,7 @@ job: - echo "Run another script if $IS_A_FEATURE exists" ``` -##### Enable or disable rules:variables **(CORE ONLY)** +##### Enable or disable rules:variables **(FREE SELF)** rules:variables is under development but ready for production use. It is deployed behind a feature flag that is **enabled by default**. @@ -1553,12 +1531,10 @@ rules that use both `||` and `&&` may evaluate with an unexpected order of opera ### `only`/`except` (basic) NOTE: -The [`rules`](#rules) syntax is an improved, more powerful solution for defining -when jobs should run or not. Consider using `rules` instead of `only/except` to get -the most out of your pipelines. +`only` and `except` are not being actively developed. To define when +to add jobs to pipelines, use [`rules`](#rules). -`only` and `except` are two keywords that set a job policy to limit when -jobs are created: +`only` and `except` are two keywords that determine when to add jobs to pipelines: 1. `only` defines the names of branches and tags the job runs for. 1. `except` defines the names of branches and tags the job does @@ -1577,8 +1553,8 @@ In addition, `only` and `except` can use special keywords: |--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). | | `branches` | When the Git reference for a pipeline is a branch. | -| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/README.md) command. | -| `external` | When using CI services other than GitLab. | +| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. | +| `external` | When you use CI services other than GitLab. | | `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). | | `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). | | `pipelines` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#triggering-multi-project-pipelines-through-api), or the [`trigger`](#trigger) keyword. | @@ -1630,7 +1606,7 @@ job: - schedules ``` -The repository path can be used to have jobs executed only for the parent +Use the repository path to have jobs executed only for the parent repository and not forks: ```yaml @@ -1648,17 +1624,13 @@ except `master` and those with names prefixed with `release/`. If a job does not have an `only` rule, `only: ['branches', 'tags']` is set by default. If it does not have an `except` rule, it's empty. -For example, +For example, `job1` and `job2` are essentially the same: ```yaml -job: +job1: script: echo 'test' -``` - -is translated to: -```yaml -job: +job2: script: echo 'test' only: ['branches', 'tags'] ``` @@ -1728,7 +1700,7 @@ the pipeline if the following is true: In the example below, the `test` job is `only` created when **all** of the following are true: -- The pipeline has been [scheduled](../pipelines/schedules.md) **or** runs for `master`. +- The pipeline is [scheduled](../pipelines/schedules.md) **or** runs for `master`. - The `variables` keyword matches. - The `kubernetes` service is active on the project. @@ -1805,7 +1777,7 @@ The `variables` keyword defines variable expressions. These expressions determine whether or not a job should be created. -Examples of using variable expressions: +Examples of variable expressions: ```yaml deploy: @@ -1844,15 +1816,14 @@ job1: > `changes` policy [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/19232) in GitLab 11.4. -Using the `changes` keyword with `only` or `except` makes it possible to define if -a job should be created based on files modified by a Git push event. +Use the `changes` keyword with `only` to run a job, or with `except` to skip a job, +when a Git push event modifies a file. -Use the `only:changes` policy for pipelines triggered by the following -refs only: +Use `only:changes` with pipelines triggered by the following refs only: - `branches` - `external_pull_requests` -- `merge_requests` (see additional details about [using `only:changes` with pipelines for merge requests](#using-onlychanges-with-pipelines-for-merge-requests)) +- `merge_requests` (see additional details about [using `only:changes` with pipelines for merge requests](#use-onlychanges-with-pipelines-for-merge-requests)) WARNING: In pipelines with [sources other than the three above](../variables/predefined_variables.md) @@ -1860,7 +1831,13 @@ In pipelines with [sources other than the three above](../variables/predefined_v You can configure jobs to use `only: changes` with other `only: refs` keywords. However, those jobs ignore the changes and always run. -A basic example of using `only: changes`: +In this example, when you push commits to an existing branch, the `docker build` job +runs only if any of these files change: + +- The `Dockerfile` file. +- Files in the `docker/scripts/` directory. +- Files and subdirectories in the `dockerfiles` directory. +- Files with `rb`, `py`, `sh` extensions in the `more_scripts` directory. ```yaml docker build: @@ -1875,21 +1852,13 @@ docker build: - more_scripts/*.{rb,py,sh} ``` -When you push commits to an existing branch, -the `docker build` job is created, but only if changes were made to any of the following: - -- The `Dockerfile` file. -- Any of the files in the `docker/scripts/` directory. -- Any of the files and subdirectories in the `dockerfiles` directory. -- Any of the files with `rb`, `py`, `sh` extensions in the `more_scripts` directory. - WARNING: If you use `only:changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds), -you should [also use `only:merge_requests`](#using-onlychanges-with-pipelines-for-merge-requests). Otherwise it may not work as expected. +you should [also use `only:merge_requests`](#use-onlychanges-with-pipelines-for-merge-requests). Otherwise it may not work as expected. You can also use glob patterns to match multiple files in either the root directory of the repository, or in _any_ directory in the repository. However, they must be wrapped -in double quotes or GitLab can't parse them. For example: +in double quotes or GitLab can't parse them: ```yaml test: @@ -1918,10 +1887,10 @@ the `build` job is still skipped. The job does not run for any of the files. Read more about how to use this feature with: -- [New branches or tags *without* pipelines for merge requests](#using-onlychanges-without-pipelines-for-merge-requests). -- [Scheduled pipelines](#using-onlychanges-with-scheduled-pipelines). +- [New branches or tags *without* pipelines for merge requests](#use-onlychanges-without-pipelines-for-merge-requests). +- [Scheduled pipelines](#use-onlychanges-with-scheduled-pipelines). -##### Using `only:changes` with pipelines for merge requests +##### Use `only:changes` with pipelines for merge requests With [pipelines for merge requests](../merge_request_pipelines/index.md), it's possible to define a job to be created based on files modified @@ -1971,7 +1940,7 @@ it doesn't matter that an earlier pipeline failed because of a change that has n When you use this configuration, ensure that the most recent pipeline properly corrects any failures from previous pipelines. -##### Using `only:changes` without pipelines for merge requests +##### Use `only:changes` without pipelines for merge requests Without [pipelines for merge requests](../merge_request_pipelines/index.md), pipelines run on branches or tags that don't have an explicit association with a merge request. @@ -1979,14 +1948,13 @@ In this case, a previous SHA is used to calculate the diff, which is equivalent This can result in some unexpected behavior, including: - When pushing a new branch or a new tag to GitLab, the policy always evaluates to true. -- When pushing a new commit, the changed files are calculated using the previous commit +- When pushing a new commit, the changed files are calculated by using the previous commit as the base SHA. -##### Using `only:changes` with scheduled pipelines +##### Use `only:changes` with scheduled pipelines -`only:changes` always evaluates as "true" in [Scheduled pipelines](../pipelines/schedules.md). -All files are considered to have "changed" when a scheduled pipeline -runs. +`only:changes` always evaluates as true in [Scheduled pipelines](../pipelines/schedules.md). +All files are considered to have changed when a scheduled pipeline runs. ### `needs` @@ -2056,15 +2024,17 @@ This example creates four paths of execution: - For GitLab.com, the limit is 50. For more information, see our [infrastructure issue](https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/7541). - For self-managed instances, the limit is: 50. This limit [can be changed](#changing-the-needs-job-limit). -- If `needs:` refers to a job that is marked as `parallel:`. - the current job depends on all parallel jobs being created. +- If `needs:` refers to a job that uses the [`parallel`](#parallel) keyword, + it depends on all jobs created in parallel, not just one job. It also downloads + artifacts from all the parallel jobs by default. If the artifacts have the same + name, they overwrite each other and only the last one downloaded is saved. - `needs:` is similar to `dependencies:` in that it must use jobs from prior stages, meaning it's impossible to create circular dependencies. Depending on jobs in the current stage is not possible either, but support [is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/30632). - Related to the above, stages must be explicitly defined for all jobs that have the keyword `needs:` or are referred to by one. -##### Changing the `needs:` job limit **(CORE ONLY)** +##### Changing the `needs:` job limit **(FREE SELF)** The maximum number of jobs that can be defined in `needs:` defaults to 50. @@ -2081,14 +2051,11 @@ To disable directed acyclic graphs (DAG), set the limit to `0`. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14311) in GitLab v12.6. -When using `needs`, artifact downloads are controlled with `artifacts: true` (default) or `artifacts: false`. +Use `artifacts: true` (default) or `artifacts: false` to control when artifacts are +downloaded in jobs that use `needs`. -In GitLab 12.6 and later, you can't combine the [`dependencies`](#dependencies) keyword -with `needs` to control artifact downloads in jobs. `dependencies` is still valid -in jobs that do not use `needs`. - -In the example below, the `rspec` job downloads the `build_job` artifacts, while the -`rubocop` job doesn't: +In this example, the `rspec` job downloads the `build_job` artifacts, but the +`rubocop` job does not: ```yaml build_job: @@ -2110,9 +2077,11 @@ rubocop: artifacts: false ``` -Additionally, in the three syntax examples below, the `rspec` job downloads the artifacts -from all three `build_jobs`. `artifacts` is true for `build_job_1` and -**defaults** to true for both `build_job_2` and `build_job_3`. +In this example, the `rspec` job downloads the artifacts from all three `build_jobs`. +`artifacts` is: + +- Set to true for `build_job_1`. +- Defaults to true for both `build_job_2` and `build_job_3`. ```yaml rspec: @@ -2123,6 +2092,9 @@ rspec: - build_job_3 ``` +In GitLab 12.6 and later, you can't combine the [`dependencies`](#dependencies) keyword +with `needs`. + #### Cross project artifact downloads with `needs` **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14311) in GitLab v12.7. @@ -2146,7 +2118,7 @@ build_job: `build_job` downloads the artifacts from the latest successful `build-1` job on the `master` branch in the `group/project-name` project. If the project is in the -same group or namespace, you can omit them from the `project:` key. For example, +same group or namespace, you can omit them from the `project:` keyword. For example, `project: group/project-name` or `project: project-name`. The user running the pipeline must have at least `reporter` access to the group or project, or the group/project must have public visibility. @@ -2171,7 +2143,7 @@ build_job: artifacts: true ``` -Environment variables support for `project:`, `job:`, and `ref` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202093) +CI/CD variable support for `project:`, `job:`, and `ref` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202093) in GitLab 13.3. [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/235761) in GitLab 13.4. For example: @@ -2280,10 +2252,11 @@ osx job: ### `allow_failure` Use `allow_failure` when you want to let a job fail without impacting the rest of the CI -suite. -The default value is `false`, except for [manual](#whenmanual) jobs using the -`when: manual` syntax, unless using [`rules:`](#rules) syntax, where all jobs -default to false, *including* `when: manual` jobs. +suite. The default value is `false`, except for [manual](#whenmanual) jobs that use +the `when: manual` syntax. + +In jobs that use [`rules:`](#rules), all jobs default to `allow_failure: false`, +*including* `when: manual` jobs. When `allow_failure` is set to `true` and the job fails, the job shows an orange warning in the UI. However, the logical flow of the pipeline considers the job a @@ -2318,13 +2291,7 @@ job3: #### `allow_failure:exit_codes` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/273157) in GitLab 13.8. -> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-allow_failureexit_codes). **(CORE ONLY)** - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292024) in GitLab 13.9. Use `allow_failure:exit_codes` to dynamically control if a job should be allowed to fail. You can list which exit codes are not considered failures. The job fails @@ -2348,31 +2315,12 @@ test_job_2: - 255 ``` -##### Enable or disable `allow_failure:exit_codes` **(CORE ONLY)** - -`allow_failure:exit_codes` is under development but ready for production use. It is -deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can disable it. - -To disable it: - -```ruby -Feature.disable(:ci_allow_failure_with_exit_codes) -``` - -To enable it: - -```ruby -Feature.enable(:ci_allow_failure_with_exit_codes) -``` - ### `when` -`when` is used to implement jobs that are run in case of failure or despite the +Use `when` to implement jobs that run in case of failure or despite the failure. -`when` can be set to one of the following values: +The valid values of `when` are: 1. `on_success` (default) - Execute job only when all jobs in earlier stages succeed, or are considered successful because they have `allow_failure: true`. @@ -2510,8 +2458,8 @@ by authorized users. Use `when: delayed` to execute scripts after a waiting period, or if you want to avoid jobs immediately entering the `pending` state. -You can set the period with `start_in` key. The value of `start_in` key is an elapsed time in seconds, unless a unit is -provided. `start_in` key must be less than or equal to one week. Examples of valid values include: +You can set the period with `start_in` keyword. The value of `start_in` is an elapsed time in seconds, unless a unit is +provided. `start_in` must be less than or equal to one week. Examples of valid values include: - `'5'` - `5 seconds` @@ -2519,13 +2467,13 @@ provided. `start_in` key must be less than or equal to one week. Examples of val - `1 day` - `1 week` -When there is a delayed job in a stage, the pipeline doesn't progress until the delayed job has finished. -This keyword can also be used for inserting delays between different stages. +When a stage includes a delayed job, the pipeline doesn't progress until the delayed job finishes. +You can use this keyword to insert delays between different stages. -The timer of a delayed job starts immediately after the previous stage has completed. -Similar to other types of jobs, a delayed job's timer doesn't start unless the previous stage passed. +The timer of a delayed job starts immediately after the previous stage completes. +Similar to other types of jobs, a delayed job's timer doesn't start unless the previous stage passes. -The following example creates a job named `timed rollout 10%` that is executed 30 minutes after the previous stage has completed: +The following example creates a job named `timed rollout 10%` that is executed 30 minutes after the previous stage completes: ```yaml timed rollout 10%: @@ -2535,7 +2483,7 @@ timed rollout 10%: start_in: 30 minutes ``` -You can stop the active timer of a delayed job by clicking the **{time-out}** (**Unschedule**) button. +To stop the active timer of a delayed job, click the **{time-out}** (**Unschedule**) button. This job can no longer be scheduled to run automatically. You can, however, execute the job manually. To start a delayed job immediately, click the **Play** button. @@ -2561,8 +2509,8 @@ deployment to the `production` environment. #### `environment:name` -The `environment: name` keyword can use any of the defined CI variables, -including predefined, secure, or `.gitlab-ci.yml` [`variables`](#variables). +The `environment: name` keyword can use any of the defined CI/CD [variables](#variables), +including predefined, secure, or variables defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. @@ -2595,8 +2543,8 @@ deploy to production: #### `environment:url` -The `url` keyword can use any of the defined CI variables, -including predefined, secure, or `.gitlab-ci.yml` [`variables`](#variables). +The `environment:url` keyword can use any of the defined CI/CD [variables](#variables), +including predefined, secure, or variables defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. @@ -2632,7 +2580,7 @@ Read the `environment:action` section for an example. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/22191) in GitLab 8.13. -The `action` keyword can be used to specify jobs that prepare, start, or stop environments. +Use the `action` keyword to specify jobs that prepare, start, or stop environments. | **Value** | **Description** | |-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -2718,7 +2666,7 @@ For more information, see > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. -The `kubernetes` block is used to configure deployments to a +Use the `kubernetes` keyword to configure deployments to a [Kubernetes cluster](../../user/project/clusters/index.md) that is associated with your project. For example: @@ -2763,7 +2711,7 @@ deploy as review app: The `deploy as review app` job is marked as a deployment to dynamically create the `review/$CI_COMMIT_REF_NAME` environment. `$CI_COMMIT_REF_NAME` -is an [environment variable](../variables/README.md) set by the runner. The +is a [CI/CD variable](../variables/README.md) set by the runner. The `$CI_ENVIRONMENT_SLUG` variable is based on the environment name, but suitable for inclusion in URLs. If the `deploy as review app` job runs in a branch named `pow`, this environment would be accessible with a URL like `https://review-pow.example.com/`. @@ -2774,11 +2722,11 @@ as Review Apps. You can see an example that uses Review Apps at ### `cache` -`cache` is used to specify a list of files and directories that should be -cached between jobs. You can only use paths that are in the local working copy. +Use the `cache` keyword to specify a list of files and directories to +cache between jobs. You can only use paths that are in the local working copy. -If `cache` is defined outside the scope of jobs, it means it's set -globally and all jobs use that definition. +If `cache` is defined outside the scope of jobs, it's set +globally and all jobs use that configuration. Caching is shared between pipelines and jobs. Caches are restored before [artifacts](#artifacts). @@ -2789,7 +2737,7 @@ Read how caching works and find out some good practices in the Use the `paths` directive to choose which files or directories to cache. Paths are relative to the project directory (`$CI_PROJECT_DIR`) and can't directly link outside it. -Wildcards can be used that follow the [glob](https://en.wikipedia.org/wiki/Glob_(programming)) +You can use Wildcards that use [glob](https://en.wikipedia.org/wiki/Glob_(programming)) patterns and: - In [GitLab Runner 13.0](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2620) and later, @@ -2864,7 +2812,7 @@ URI-encoded `%2F`. A value made only of dots (`.`, `%2E`) is also forbidden. You can specify a [fallback cache key](#fallback-cache-key) to use if the specified `cache:key` is not found. -#### Fallback cache key +##### Fallback cache key > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4. @@ -2875,7 +2823,8 @@ to download cache that's tagged with `test`. If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to specify a cache to use when none exists. -For example: +In this example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined +by the `CACHE_FALLBACK_KEY` variable: ```yaml variables: @@ -2887,9 +2836,6 @@ cache: - binaries/ ``` -In this example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined -by the `CACHE_FALLBACK_KEY` variable. - ##### `cache:key:files` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18986) in GitLab v12.5. @@ -2900,7 +2846,7 @@ runs. When you include `cache:key:files`, you must also list the project files that are used to generate the key, up to a maximum of two files. The cache `key` is a SHA checksum computed from the most recent commits (up to two, if two files are listed) -that changed the given files. If neither file was changed in any commits, +that changed the given files. If neither file is changed in any commits, the fallback key is `default`. ```yaml @@ -2927,7 +2873,7 @@ use the new cache, instead of rebuilding the dependencies. When you want to combine a prefix with the SHA computed for `cache:key:files`, use the `prefix` keyword with `key:files`. For example, if you add a `prefix` of `test`, the resulting key is: `test-feef9576d21ee9b6a32e30c5c79d0a0ceb68d1e5`. -If neither file was changed in any commits, the prefix is added to `default`, so the +If neither file is changed in any commits, the prefix is added to `default`, so the key in the example would be `test-default`. Like `cache:key`, `prefix` can use any of the [predefined variables](../variables/README.md), @@ -3039,8 +2985,8 @@ rspec: - bundle exec rspec ... ``` -The `pull` policy speeds up job execution and reduces load on the cache server. It -can be used when you have many jobs that use caches executing in parallel. +Use the `pull` policy when you have many jobs executing in parallel that use caches. This +policy speeds up job execution and reduces load on the cache server. If you have a job that unconditionally recreates the cache without referring to its previous contents, you can skip the download step. @@ -3048,7 +2994,7 @@ To do so, add `policy: push` to the job. ### `artifacts` -`artifacts` is used to specify a list of files and directories that are +Use the `artifacts` keyword to specify a list of files and directories that are attached to the job when it [succeeds, fails, or always](#artifactswhen). The artifacts are sent to GitLab after the job finishes. They are @@ -3063,7 +3009,7 @@ artifacts are restored after [caches](#cache). #### `artifacts:paths` Paths are relative to the project directory (`$CI_PROJECT_DIR`) and can't directly -link outside it. Wildcards can be used that follow the [glob](https://en.wikipedia.org/wiki/Glob_(programming)) +link outside it. You can use Wildcards that use [glob](https://en.wikipedia.org/wiki/Glob_(programming)) patterns and: - In [GitLab Runner 13.0](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2620) and later, @@ -3122,6 +3068,32 @@ job: - path/*xyz/* ``` +#### `artifacts:public` + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49775) in GitLab 13.8 +> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. + +Use `artifacts:public` to determine whether the job artifacts should be +publicly available. + +The default for `artifacts:public` is `true` which means that the artifacts in +public pipelines are available for download by anonymous and guest users: + +```yaml +artifacts: + public: true +``` + +To deny read access for anonymous and guest users to artifacts in public +pipelines, set `artifacts:public` to `false`: + +```yaml +artifacts: + public: false +``` + #### `artifacts:exclude` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15122) in GitLab 13.1 @@ -3131,9 +3103,9 @@ job: archive. Similar to [`artifacts:paths`](#artifactspaths), `exclude` paths are relative -to the project directory. Wildcards can be used that follow the -[glob](https://en.wikipedia.org/wiki/Glob_(programming)) patterns and -[`filepath.Match`](https://golang.org/pkg/path/filepath/#Match). +to the project directory. You can use Wildcards that use +[glob](https://en.wikipedia.org/wiki/Glob_(programming)) or +[`filepath.Match`](https://golang.org/pkg/path/filepath/#Match) patterns. For example, to store all files in `binaries/`, but not `*.o` files located in subdirectories of `binaries/`: @@ -3153,7 +3125,7 @@ Files matched by [`artifacts:untracked`](#artifactsuntracked) can be excluded us > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15018) in GitLab 12.5. -The `expose_as` keyword can be used to expose [job artifacts](../pipelines/job_artifacts.md) +Use the `expose_as` keyword to expose [job artifacts](../pipelines/job_artifacts.md) in the [merge request](../../user/project/merge_requests/index.md) UI. For example, to match a single file: @@ -3270,8 +3242,8 @@ job: #### `artifacts:untracked` -`artifacts:untracked` is used to add all Git untracked files as artifacts (along -to the paths defined in `artifacts:paths`). `artifacts:untracked` ignores configuration +Use `artifacts:untracked` to add all Git untracked files as artifacts (along +with the paths defined in `artifacts:paths`). `artifacts:untracked` ignores configuration in the repository's `.gitignore` file. Send all Git untracked files: @@ -3301,7 +3273,7 @@ artifacts: #### `artifacts:when` -`artifacts:when` is used to upload artifacts on job failure or despite the +Use `artifacts:when` to upload artifacts on job failure or despite the failure. `artifacts:when` can be set to one of the following values: @@ -3361,12 +3333,14 @@ job: The latest artifacts for refs are locked against deletion, and kept regardless of the expiry time. [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/16267) GitLab 13.0 behind a disabled feature flag, and [made the default behavior](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) -in GitLab 13.4. In [GitLab 13.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/241026), you can [disable this behavior in the CI/CD settings](../pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). +in GitLab 13.4. + +In [GitLab 13.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/241026), you can [disable this behavior at the project level in the CI/CD settings](../pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). In [GitLab 13.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/276583), you can [disable this behavior instance-wide](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). #### `artifacts:reports` -The [`artifacts:reports` keyword](../pipelines/job_artifacts.md#artifactsreports) -is used for collecting test reports, code quality reports, and security reports from jobs. +Use [`artifacts:reports`](../pipelines/job_artifacts.md#artifactsreports) +to collect test reports, code quality reports, and security reports from jobs. It also exposes these reports in the GitLab UI (merge requests, pipeline views, and security dashboards). These are the available report types: @@ -3374,7 +3348,7 @@ These are the available report types: | Keyword | Description | |-----------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------| | [`artifacts:reports:cobertura`](../pipelines/job_artifacts.md#artifactsreportscobertura) | The `cobertura` report collects Cobertura coverage XML files. | -| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects CodeQuality issues. | +| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects Code Quality issues. | | [`artifacts:reports:container_scanning`](../pipelines/job_artifacts.md#artifactsreportscontainer_scanning) **(ULTIMATE)** | The `container_scanning` report collects Container Scanning vulnerabilities. | | [`artifacts:reports:dast`](../pipelines/job_artifacts.md#artifactsreportsdast) **(ULTIMATE)** | The `dast` report collects Dynamic Application Security Testing vulnerabilities. | | [`artifacts:reports:dependency_scanning`](../pipelines/job_artifacts.md#artifactsreportsdependency_scanning) **(ULTIMATE)** | The `dependency_scanning` report collects Dependency Scanning vulnerabilities. | @@ -3385,7 +3359,7 @@ These are the available report types: | [`artifacts:reports:load_performance`](../pipelines/job_artifacts.md#artifactsreportsload_performance) **(PREMIUM)** | The `load_performance` report collects load performance metrics. | | [`artifacts:reports:metrics`](../pipelines/job_artifacts.md#artifactsreportsmetrics) **(PREMIUM)** | The `metrics` report collects Metrics. | | [`artifacts:reports:performance`](../pipelines/job_artifacts.md#artifactsreportsperformance) **(PREMIUM)** | The `performance` report collects Browser Performance metrics. | -| [`artifacts:reports:sast`](../pipelines/job_artifacts.md#artifactsreportssast) **(ULTIMATE)** | The `sast` report collects Static Application Security Testing vulnerabilities. | +| [`artifacts:reports:sast`](../pipelines/job_artifacts.md#artifactsreportssast) | The `sast` report collects Static Application Security Testing vulnerabilities. | | [`artifacts:reports:terraform`](../pipelines/job_artifacts.md#artifactsreportsterraform) | The `terraform` report collects Terraform `tfplan.json` files. | #### `dependencies` @@ -3403,7 +3377,7 @@ An error occurs if you define jobs from the current or an upcoming stage. To prevent a job from downloading artifacts, define an empty array. When you use `dependencies`, the status of the previous job is not considered. -If a job fails or it's a manual job that was not run, no error occurs. +If a job fails or it's a manual job that isn't triggered, no error occurs. The following example defines two jobs with artifacts: `build:osx` and `build:linux`. When the `test:osx` is executed, the artifacts from `build:osx` @@ -3449,7 +3423,7 @@ deploy: > Introduced in GitLab 10.3. -If the artifacts of the job that is set as a dependency have been +If the artifacts of the job that is set as a dependency are [expired](#artifactsexpire_in) or [erased](../pipelines/job_artifacts.md#erasing-artifacts), then the dependent job fails. @@ -3478,7 +3452,7 @@ job1: The coverage is shown in the UI if at least one line in the job output matches the regular expression. If there is more than one matched line in the job output, the last line is used. -For the matched line, the first occurence of `\d+(\.\d+)?` is the code coverage. +For the matched line, the first occurrence of `\d+(\.\d+)?` is the code coverage. Leading zeros are removed. Coverage output from [child pipelines](../parent_child_pipelines.md) is not recorded @@ -3554,15 +3528,15 @@ Possible values for `when` are: - `script_failure`: Retry when the script failed. - `api_failure`: Retry on API failure. - `stuck_or_timeout_failure`: Retry when the job got stuck or timed out. -- `runner_system_failure`: Retry if there was a runner system failure (for example, job setup failed). -- `missing_dependency_failure`: Retry if a dependency was missing. -- `runner_unsupported`: Retry if the runner was unsupported. +- `runner_system_failure`: Retry if there is a runner system failure (for example, job setup failed). +- `missing_dependency_failure`: Retry if a dependency is missing. +- `runner_unsupported`: Retry if the runner is unsupported. - `stale_schedule`: Retry if a delayed job could not be executed. - `job_execution_timeout`: Retry if the script exceeded the maximum execution time set for the job. - `archived_failure`: Retry if the job is archived and can't be run. - `unmet_prerequisites`: Retry if the job failed to complete prerequisite tasks. - `scheduler_failure`: Retry if the scheduler failed to assign the job to a runner. -- `data_integrity_failure`: Retry if there was a structural integrity problem detected. +- `data_integrity_failure`: Retry if there is a structural integrity problem detected. You can specify the number of [retry attempts for certain stages of job execution](../runners/README.md#job-stages-attempts) using variables. @@ -3603,7 +3577,7 @@ test: ``` Every parallel job has a `CI_NODE_INDEX` and `CI_NODE_TOTAL` -[environment variable](../variables/README.md#predefined-environment-variables) set. +[predefined CI/CD variable](../variables/README.md#predefined-cicd-variables) set. Different languages and test suites have different methods to enable parallelization. For example, use [Semaphore Test Boosters](https://github.com/renderedtext/test-boosters) @@ -3640,9 +3614,9 @@ but with different variable values for each instance of the job. There can be from 2 to 50 jobs. Jobs can only run in parallel if there are multiple runners, or a single runner is -[configured to run multiple jobs concurrently](#using-your-own-runners). +[configured to run multiple jobs concurrently](#use-your-own-runners). -Every job gets the same `CI_NODE_TOTAL` [environment variable](../variables/README.md#predefined-environment-variables) value, and a unique `CI_NODE_INDEX` value. +Every job gets the same `CI_NODE_TOTAL` [CI/CD variable](../variables/README.md#predefined-cicd-variables) value, and a unique `CI_NODE_INDEX` value. ```yaml deploystacks: @@ -3699,10 +3673,10 @@ deploystacks: ### `trigger` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Core in 12.8. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. -Use `trigger` to define a downstream pipeline trigger. When GitLab starts a job created -with a `trigger` definition, a downstream pipeline is created. +Use `trigger` to define a downstream pipeline trigger. When GitLab starts a `trigger` job, +a downstream pipeline is created. Jobs with `trigger` can only use a [limited set of keywords](../multi_project_pipelines.md#limitations). For example, you can't run commands with [`script`](#script), [`before_script`](#before_script), @@ -3777,7 +3751,7 @@ upstream_bridge: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7. To create a [child pipeline](../parent_child_pipelines.md), specify the path to the -YAML file containing the CI config of the child pipeline: +YAML file that contains the configuration of the child pipeline: ```yaml trigger_job: @@ -3871,10 +3845,10 @@ The trigger token is different than the [`trigger`](#trigger) keyword. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32022) in GitLab 12.3. -`interruptible` is used to indicate that a running job should be canceled if made redundant by a newer pipeline run. +Use `interruptible` to indicate that a running job should be canceled if made redundant by a newer pipeline run. Defaults to `false` (uninterruptible). Jobs that have not started yet (pending) are considered interruptible and safe to be cancelled. -This value is used only if the [automatic cancellation of redundant pipelines feature](../pipelines/settings.md#auto-cancel-pending-pipelines) +This value is used only if the [automatic cancellation of redundant pipelines feature](../pipelines/settings.md#auto-cancel-redundant-pipelines) is enabled. When enabled, a pipeline is immediately canceled when a new pipeline starts on the same branch if either of the following is true: @@ -3924,11 +3898,11 @@ When an uninterruptible job is running, the pipeline cannot be canceled, regardl Sometimes running multiple jobs or pipelines at the same time in an environment can lead to errors during the deployment. -To avoid these errors, the `resource_group` attribute can be used to ensure that +To avoid these errors, use the `resource_group` attribute to make sure that the runner doesn't run certain jobs simultaneously. Resource groups behave similar to semaphores in other programming languages. -When the `resource_group` key is defined for a job in `.gitlab-ci.yml`, +When the `resource_group` keyword is defined for a job in the `.gitlab-ci.yml` file, job executions are mutually exclusive across different pipelines for the same project. If multiple jobs belonging to the same resource group are enqueued simultaneously, only one of the jobs is picked by the runner. The other jobs wait until the @@ -3954,13 +3928,67 @@ It can't start or end with `/`. For more information, see [Deployments Safety](../environments/deployment_safety.md). +#### Pipeline-level concurrency control with Cross-Project/Parent-Child pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39057) in GitLab 13.9. + +You can define `resource_group` for downstream pipelines that are sensitive to concurrent +executions. The [`trigger` keyword](#trigger) can trigger downstream pipelines. The +[`resource_group` keyword](#resource_group) can co-exist with it. This is useful to control the +concurrency for deployment pipelines, while running non-sensitive jobs concurrently. + +This example has two pipeline configurations in a project. When a pipeline starts running, +non-sensitive jobs are executed first and aren't affected by concurrent executions in other +pipelines. However, GitLab ensures that there are no other deployment pipelines running before +triggering a deployment (child) pipeline. If other deployment pipelines are running, GitLab waits +until those pipelines finish before running another one. + +```yaml +# .gitlab-ci.yml (parent pipeline) + +build: + stage: build + script: echo "Building..." + +test: + stage: test + script: echo "Testing..." + +deploy: + stage: deploy + trigger: + include: deploy.gitlab-ci.yml + strategy: depend + resource_group: AWS-production +``` + +```yaml +# deploy.gitlab-ci.yml (child pipeline) + +stages: + - provision + - deploy + +provision: + stage: provision + script: echo "Provisioning..." + +deployment: + stage: deploy + script: echo "Deploying..." +``` + +Note that you must define [`strategy: depend`](#linking-pipelines-with-triggerstrategy) +with the `trigger` keyword. This ensures that the lock isn't released until the downstream pipeline +finishes. + ### `release` > [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/19298) in GitLab 13.2. `release` indicates that the job creates a [Release](../../user/project/releases/index.md). -These methods are supported: +These keywords are supported: - [`tag_name`](#releasetag_name) - [`description`](#releasedescription) @@ -3983,7 +4011,7 @@ image: registry.gitlab.com/gitlab-org/release-cli:latest #### Script All jobs except [trigger](#trigger) jobs must have the `script` keyword. A `release` -job can use the output from script commands, but a placeholder script can be used if +job can use the output from script commands, but you can use a placeholder script if the script is not needed: ```yaml @@ -4055,7 +4083,7 @@ description. You can specify a file in `$CI_PROJECT_DIR` that contains the description. The file must be relative to the project directory (`$CI_PROJECT_DIR`), and if the file is a symbolic link it can't reside -outside of `$CI_PROJECT_DIR`. The `./path/to/file` and file name can't contain spaces. +outside of `$CI_PROJECT_DIR`. The `./path/to/file` and filename can't contain spaces. ```yaml job: @@ -4086,7 +4114,7 @@ released_at: '2021-03-15T08:00:00Z' Combining the individual examples given above for `release` results in the following code snippets. There are two options, depending on how you generate the -tags. These options cannot be used together, so choose one: +tags. You can't use these options together, so choose one: - To create a release when you push a Git tag, or when you add a Git tag in the UI by going to **Repository > Tags**: @@ -4179,10 +4207,10 @@ release-cli create --name "Release $CI_COMMIT_SHA" --description "Created using > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33014) in GitLab 13.4. -`secrets` indicates the [CI Secrets](../secrets/index.md) this job needs. It should be a hash, -and the keys should be the names of the environment variables that are made available to the job. +`secrets` indicates the [CI/CD Secrets](../secrets/index.md) this job needs. It should be a hash, +and the keys should be the names of the variables that are made available to the job. The value of each secret is saved in a temporary file. This file's path is stored in these -environment variables. +variables. #### `secrets:vault` **(PREMIUM)** @@ -4226,8 +4254,8 @@ job: ### `pages` -`pages` is a special job that is used to upload static content to GitLab that -can be used to serve your website. It has a special syntax, so the two +`pages` is a special job that uploads static content to GitLab that +is then published as a website. It has a special syntax, so the two requirements below must be met: - Any static content must be placed under a `public/` directory. @@ -4262,9 +4290,10 @@ They can be set globally and per-job. There are two types of variables. -- [Custom variables](../variables/README.md#custom-environment-variables): +- [Custom variables](../variables/README.md#custom-cicd-variables): You can define their values in the `.gitlab-ci.yml` file, in the GitLab UI, - or by using the API. + or by using the API. You can also input variables in the GitLab UI when + [running a pipeline manually](../pipelines/index.md#run-a-pipeline-manually). - [Predefined variables](../variables/predefined_variables.md): These values are set by the runner itself. One example is `CI_COMMIT_REF_NAME`, which is the branch or tag the project is built for. @@ -4297,13 +4326,27 @@ meaning it applies to all jobs. If you define a variable in a job, it's availabl to that job only. If a variable of the same name is defined globally and for a specific job, the -[job-specific variable is used](../variables/README.md#priority-of-environment-variables). +[job-specific variable overrides the global variable](../variables/README.md#priority-of-cicd-variables). All YAML-defined variables are also set to any linked [Docker service containers](../docker/using_docker_images.md#what-is-a-service). You can use [YAML anchors for variables](#yaml-anchors-for-variables). +### Prefill variables in manual pipelines + +> [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/30101) GitLab 13.7. + +You can use the `value` and `description` keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) +when [running a pipeline manually](../pipelines/index.md#run-a-pipeline-manually): + +```yaml +variables: + DEPLOY_ENVIRONMENT: + value: "staging" # Deploy to staging by default + description: "The deployment target. Change this variable to 'canary' or 'production' if needed." +``` + ### Configure runner behavior with variables You can use [CI/CD variables](../variables/README.md) to configure runner Git behavior: @@ -4315,6 +4358,9 @@ You can use [CI/CD variables](../variables/README.md) to configure runner Git be - [`GIT_FETCH_EXTRA_FLAGS`](../runners/README.md#git-fetch-extra-flags) - [`GIT_DEPTH`](../runners/README.md#shallow-cloning) (shallow cloning) - [`GIT_CLONE_PATH`](../runners/README.md#custom-build-directories) (custom build directories) +- [`TRANSFER_METER_FREQUENCY`](../runners/README.md#artifact-and-cache-settings) (artifact/cache meter update frequency) +- [`ARTIFACT_COMPRESSION_LEVEL`](../runners/README.md#artifact-and-cache-settings) (artifact archiver compression level) +- [`CACHE_COMPRESSION_LEVEL`](../runners/README.md#artifact-and-cache-settings) (cache archiver compression level) You can also use variables to configure how many times a runner [attempts certain stages of job execution](../runners/README.md#job-stages-attempts). @@ -4323,13 +4369,14 @@ You can also use variables to configure how many times a runner It's possible to use special YAML features like anchors (`&`), aliases (`*`) and map merging (`<<`). Use these features to reduce the complexity -of `.gitlab-ci.yml`. +of the code in the `.gitlab-ci.yml` file. Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/). In most cases, the [`extends` keyword](#extends) is more user friendly and should -be used over these special YAML features. YAML anchors may still -need to be used to merge arrays. +be used over these special YAML features. + +You can use YAML anchors to merge YAML arrays. ### Anchors @@ -4349,26 +4396,26 @@ The following example uses anchors and map merging. It creates two jobs, with their own custom `script` defined: ```yaml -.job_template: &job_definition # Hidden key that defines an anchor named 'job_definition' +.job_template: &job_configuration # Hidden yaml configuration that defines an anchor named 'job_configuration' image: ruby:2.6 services: - postgres - redis test1: - <<: *job_definition # Merge the contents of the 'job_definition' alias + <<: *job_configuration # Merge the contents of the 'job_configuration' alias script: - test1 project test2: - <<: *job_definition # Merge the contents of the 'job_definition' alias + <<: *job_configuration # Merge the contents of the 'job_configuration' alias script: - test2 project ``` -`&` sets up the name of the anchor (`job_definition`), `<<` means "merge the +`&` sets up the name of the anchor (`job_configuration`), `<<` means "merge the given hash into the current one", and `*` includes the named anchor -(`job_definition` again). The expanded version of the example above is: +(`job_configuration` again). The expanded version of the example above is: ```yaml .job_template: @@ -4399,31 +4446,31 @@ and `test:mysql` share the `script` defined in `.job_template`, but use differen `services`, defined in `.postgres_services` and `.mysql_services`: ```yaml -.job_template: &job_definition +.job_template: &job_configuration script: - test project tags: - dev .postgres_services: - services: &postgres_definition + services: &postgres_configuration - postgres - ruby .mysql_services: - services: &mysql_definition + services: &mysql_configuration - mysql - ruby test:postgres: - <<: *job_definition - services: *postgres_definition + <<: *job_configuration + services: *postgres_configuration tags: - postgres test:mysql: - <<: *job_definition - services: *mysql_definition + <<: *job_configuration + services: *mysql_configuration ``` The expanded version is: @@ -4465,7 +4512,7 @@ test:mysql: ``` You can see that the hidden jobs are conveniently used as templates, and -`tags: [dev]` has been overwritten by `tags: [postgres]`. +`tags: [postgres]` overwrites `tags: [dev]`. #### YAML anchors for scripts @@ -4475,28 +4522,37 @@ You can use [YAML anchors](#anchors) with [script](#script), [`before_script`](# and [`after_script`](#after_script) to use predefined commands in multiple jobs: ```yaml -.some-script: &some-script - - echo "Execute this script in `before_script` sections" - .some-script-before: &some-script-before - - echo "Execute this script in `script` sections" + - echo "Execute this script first" + +.some-script: &some-script + - echo "Execute this script second" + - echo "Execute this script too" .some-script-after: &some-script-after - - echo "Execute this script in `after_script` sections" + - echo "Execute this script last" -job_name: +job1: before_script: - *some-script-before script: - *some-script + - echo "Execute something, for this job only" after_script: - *some-script-after + +job2: + script: + - *some-script-before + - *some-script + - echo "Execute something else, for this job only" + - *some-script-after ``` #### YAML anchors for variables -[YAML anchors](#anchors) can be used with `variables`, to repeat assignment -of variables across multiple jobs. Use can also use YAML anchors when a job +Use [YAML anchors](#anchors) with `variables` to repeat assignment +of variables across multiple jobs. You can also use YAML anchors when a job requires a specific `variables` block that would otherwise override the global variables. In the example below, we override the `GIT_STRATEGY` variable without affecting @@ -4541,6 +4597,68 @@ Use this feature to ignore jobs, or use the [special YAML features](#special-yaml-features) and transform the hidden jobs into templates. +### `!reference` tags + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/266173) in GitLab 13.9. + +Use the `!reference` custom YAML tag to select keyword configuration from other job +sections and reuse it in the current section. Unlike [YAML anchors](#anchors), you can +use `!reference` tags to reuse configuration from [included](#include) configuration +files as well. + +In this example, a `script` and an `after_script` from two different locations are +reused in the `test` job: + +- `setup.yml`: + + ```yaml + .setup: + script: + - echo creating environment + ``` + +- `.gitlab-ci.yml`: + + ```yaml + include: + - local: setup.yml + + .teardown: + after_script: + - echo deleting environment + + test: + script: + - !reference [.setup, script] + - echo running my own command + after_script: + - !reference [.teardown, after_script] + ``` + +In this example, `test-vars-1` reuses the all the variables in `.vars`, while `test-vars-2` +selects a specific variable and reuses it as a new `MY_VAR` variable. + +```yaml +.vars: + variables: + URL: "http://my-url.internal" + IMPORTANT_VAR: "the details" + +test-vars-1: + variables: !reference [.vars, variables] + script: + - printenv + +test-vars-2: + variables: + MY_VAR: !reference [.vars, variables, IMPORTANT_VAR] + script: + - printenv +``` + +You can't reuse a section that already includes a `!reference` tag. Only one level +of nesting is supported. + ## Skip Pipeline To push a commit without triggering a pipeline, add `[ci skip]` or `[skip ci]`, using any diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md index e4ede9cf699..765b982dbeb 100644 --- a/doc/ci/yaml/gitlab_ci_yaml.md +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -5,7 +5,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- <!-- markdownlint-disable MD044 --> +<!-- vale gitlab.Spelling = NO --> # The .gitlab-ci.yml file +<!-- vale gitlab.Spelling = YES --> <!-- markdownlint-enable MD044 --> To use GitLab CI/CD, you need: diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index 2d26d9f8922..5750fe1ba61 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -121,7 +121,7 @@ job: - echo -e "\e[31mThis text is red,\e[0m but this text isn't\e[31m however this text is red again." ``` -You can define the color codes in Shell variables, or even [custom environment variables](../variables/README.md#custom-environment-variables), +You can define the color codes in Shell variables, or even [custom environment variables](../variables/README.md#custom-cicd-variables), which makes the commands easier to read and reusable. For example, using the same example as above and variables defined in a `before_script`: |
