diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-08-19 09:08:42 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-08-19 09:08:42 +0000 |
| commit | b76ae638462ab0f673e5915986070518dd3f9ad3 (patch) | |
| tree | bdab0533383b52873be0ec0eb4d3c66598ff8b91 /doc/ci | |
| parent | 434373eabe7b4be9593d18a585fb763f1e5f1a6f (diff) | |
| download | gitlab-ce-b76ae638462ab0f673e5915986070518dd3f9ad3.tar.gz | |
Add latest changes from gitlab-org/gitlab@14-2-stable-eev14.2.0-rc42
Diffstat (limited to 'doc/ci')
70 files changed, 1083 insertions, 725 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 5ab8653dc35..0e6c2f63f9e 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,5 +1,6 @@ --- redirect_to: 'index.md' +remove_date: '2021-09-28' --- This document was moved to [another location](index.md). diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 0ed1e978168..87c7af2030d 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -128,7 +128,7 @@ To disable it completely for a job, use an empty hash: ```yaml job: - cache: {} + cache: [] ``` ## Inherit global configuration, but override specific settings per job diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index 03a1005c9bc..a461147661c 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -49,7 +49,7 @@ If a job shouldn't be able to be triggered from chat, you can set the job to `ex Since ChatOps is built upon GitLab CI/CD, the job has all the same features and functions available. Consider these best practices when creating ChatOps jobs: -- GitLab strongly recommends you set `only: [chat]` so the job does not run as part +- GitLab strongly recommends you set [`rules`](../yaml/index.md#rules) so the job does not run as part of the standard CI pipeline. - If the job is set to `when: manual`, ChatOps creates the pipeline, but the job waits to be started. - ChatOps provides limited support for access control. If the user triggering the @@ -65,9 +65,13 @@ The output for jobs with a single command is sent to the channel as a reply. For example, the chat reply of the following job is `Hello World` in the channel: ```yaml +stages: +- chatops + hello-world: stage: chatops - only: [chat] + rules: + - if: '$CI_PIPELINE_SOURCE == "chat"' script: - echo "Hello World" ``` @@ -81,9 +85,13 @@ the `chat_reply` section. For example, the following job lists the files in the current directory: ```yaml +stages: +- chatops + ls: stage: chatops - only: [chat] + rules: + - if: '$CI_PIPELINE_SOURCE == "chat"' script: - echo "This command will not be shown." - echo -e "section_start:$( date +%s ):chat_reply\r\033[0K\n$( ls -la )\nsection_end:$( date +%s ):chat_reply\r\033[0K" diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index 7eaafcf21bb..e69daf6651a 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -14,15 +14,16 @@ GitLab CI/CD can be used with Bitbucket Cloud by: To use GitLab CI/CD with a Bitbucket Cloud repository: -1. <!-- vale gitlab.Spelling = NO --> In GitLab create a **CI/CD for external repository**, select - **Repo by URL** and create the project. - <!-- vale gitlab.Spelling = YES --> +1. In GitLab, create a project: + 1. On the top menu, select **Projects > Create new project**. + 1. Select **Run CI/CD for external repository**. + 1. Select **Repo by URL**.  GitLab imports the repository and enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository). -1. In GitLab create a +1. In GitLab, create a [Personal Access Token](../../user/profile/personal_access_tokens.md) with `api` scope. This is used to authenticate requests from the web hook that is created in Bitbucket to notify GitLab of new commits. @@ -120,7 +121,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: \"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }" ``` -1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push +1. In Bitbucket, create a `.gitlab-ci.yml` file to use the script to push pipeline success and failures to Bitbucket. ```yaml diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 7a79794f322..60a939496d6 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -27,27 +27,26 @@ repositories to GitLab, and the GitHub user must have the [owner role](https://d To perform a one-off authorization with GitHub to grant GitLab access your repositories: -1. Open <https://github.com/settings/tokens/new> to create a **Personal Access - Token**. This token is used to access your repository and push commit - statuses to GitHub. - - The `repo` and `admin:repo_hook` should be enable to allow GitLab access to - your project, update commit statuses, and create a web hook to notify - GitLab of new commits. - -1. In GitLab, go to the [new project page](../../user/project/working_with_projects.md#create-a-project), select the **CI/CD for external repository** tab, and then click - **GitHub**. - -1. Paste the token into the **Personal access token** field and click **List - Repositories**. Click **Connect** to select the repository. - +1. In GitHub, create a token: + 1. Open <https://github.com/settings/tokens/new>. + 1. Create a **Personal Access Token**. + 1. Enter a **Token description** and update the scope to allow + `repo` and `admin:repo_hook` so that GitLab can access your project, + update commit statuses, and create a web hook to notify GitLab of new commits. +1. In GitLab, create a project: + 1. On the top menu, select **Projects > Create new project**. + 1. Select **Run CI/CD for external repository**. + 1. Select **GitHub**. + 1. For **Personal access token**, paste the token. + 1. Select **List Repositories**. + 1. Select **Connect** to select the repository. 1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/index.md). GitLab: 1. Imports the project. -1. Enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository) -1. Enables [GitHub project integration](../../user/project/integrations/github.md) +1. Enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository). +1. Enables [GitHub project integration](../../user/project/integrations/github.md). 1. Creates a web hook on GitHub to notify GitLab of new commits. ## Connect manually @@ -56,30 +55,25 @@ To use **GitHub Enterprise** with **GitLab.com**, use this method. To manually enable GitLab CI/CD for your repository: -1. In GitHub open <https://github.com/settings/tokens/new> create a **Personal - Access Token.** GitLab uses this token to access your repository and - push commit statuses. - - Enter a **Token description** and update the scope to allow: - - `repo` so that GitLab can access your project and update commit statuses - -1. In GitLab create a **CI/CD project** using the Git URL option and the HTTPS - URL for your GitHub repository. If your project is private, use the personal - access token you just created for authentication. - - GitLab automatically configures polling-based pull mirroring. - -1. Still in GitLab, enable the [GitHub project integration](../../user/project/integrations/github.md) - from **Settings > Integrations.** - - Check the **Active** checkbox to enable the integration, paste your - personal access token and HTTPS repository URL into the form, and **Save.** - -1. Still in GitLab create a **Personal Access Token** with `API` scope to +1. In GitHub, create a token: + 1. Open <https://github.com/settings/tokens/new>. + 1. Create a **Personal Access Token**. + 1. Enter a **Token description** and update the scope to allow + `repo` so that GitLab can access your project and update commit statuses. +1. In GitLab, create a project: + 1. On the top menu, select **Projects > Create new project**. + 1. Select **Run CI/CD for external repository** and **Repo by URL**. + 1. In the **Git repository URL** field, enter the HTTPS URL for your GitHub repository. + If your project is private, use the personal access token you just created for authentication. + 1. Fill in all the other fields and select **Create project**. + GitLab automatically configures polling-based pull mirroring. +1. In GitLab, enable [GitHub project integration](../../user/project/integrations/github.md): + 1. On the left sidebar, select **Settings > Integrations**. + 1. Select the **Active** checkbox. + 1. Paste your personal access token and HTTPS repository URL into the form and select **Save**. +1. In GitLab, create a **Personal Access Token** with `API` scope to authenticate the GitHub web hook notifying GitLab of new commits. - -1. In GitHub from **Settings > Webhooks** create a web hook to notify GitLab of +1. In GitHub, from **Settings > Webhooks**, create a web hook to notify GitLab of new commits. The web hook URL should be set to the GitLab API to @@ -92,7 +86,7 @@ To manually enable GitLab CI/CD for your repository: Select the **Let me select individual events** option, then check the **Pull requests** and **Pushes** checkboxes. These settings are needed for [pipelines for external pull requests](index.md#pipelines-for-external-pull-requests). -1. In GitHub add a `.gitlab-ci.yml` to configure GitLab CI/CD. +1. In GitHub, add a `.gitlab-ci.yml` to configure GitLab CI/CD. <!-- ## Troubleshooting diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 8c961ea6128..27c808791e5 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -27,10 +27,10 @@ To connect to an external repository: <!-- vale gitlab.Spelling = NO --> -1. From your GitLab dashboard, click **New project**. -1. Switch to the **CI/CD for external repository** tab. -1. Choose **GitHub** or **Repo by URL**. -1. The next steps are similar to the [import flow](../../user/project/import/index.md). +1. On the top menu, select **Projects > Create new project**. +1. Select **Run CI/CD for external repository**. +1. Select **GitHub** or **Repo by URL**. +1. Complete the fields. <!-- vale gitlab.Spelling = YES --> diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index c14d94fcd61..a4b4fd9fd16 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Cloud deployment +# Cloud deployment **(FREE)** Interacting with a major cloud provider may have become a much needed task that's part of your delivery process. With GitLab you can diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 108d4e0dcad..6886899a54b 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -64,7 +64,7 @@ build: entrypoint: [""] script: - mkdir -p /kaniko/.docker - - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json + - echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n ${CI_REGISTRY_USER}:${CI_REGISTRY_PASSWORD} | base64)\"}}}" > /kaniko/.docker/config.json - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_TAG rules: - if: $CI_COMMIT_TAG @@ -91,7 +91,7 @@ build: - mkdir -p /kaniko/.docker - |- KANIKOPROXYBUILDARGS="" - KANIKOCFG="{ \"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}" + KANIKOCFG="{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n ${CI_REGISTRY_USER}:${CI_REGISTRY_PASSWORD} | base64)\"}}}" if [ "x${http_proxy}" != "x" -o "x${https_proxy}" != "x" ]; then KANIKOCFG="${KANIKOCFG}, \"proxies\": { \"default\": { \"httpProxy\": \"${http_proxy}\", \"httpsProxy\": \"${https_proxy}\", \"noProxy\": \"${no_proxy}\"}}" KANIKOPROXYBUILDARGS="--build-arg http_proxy=${http_proxy} --build-arg https_proxy=${https_proxy} --build-arg no_proxy=${no_proxy}" @@ -120,7 +120,7 @@ store: ```yaml before_script: - mkdir -p /kaniko/.docker - - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json + - echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n ${CI_REGISTRY_USER}:${CI_REGISTRY_PASSWORD} | base64)\"}}}" > /kaniko/.docker/config.json - | echo "-----BEGIN CERTIFICATE----- ... diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 341727f0f5d..421bca9e324 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -19,7 +19,7 @@ If you are using an external CI/CD server like Jenkins or Drone CI, it is advise to disable GitLab CI/CD in order to not have any conflicts with the commits status API. -GitLab CI/CD is exposed via the `/pipelines` and `/jobs` pages of a project. +GitLab CI/CD is exposed by using the `/pipelines` and `/jobs` pages of a project. Disabling GitLab CI/CD in a project does not delete any previous jobs. In fact, the `/pipelines` and `/jobs` pages can still be accessed, although it's hidden from the left sidebar menu. @@ -36,62 +36,68 @@ pipelines that are run from an [external integration](../user/project/integratio ## Per-project user setting -To enable or disable GitLab CI/CD Pipelines in your project: +To enable or disable GitLab CI/CD pipelines in your project: -1. Navigate to **Settings > General > Visibility, project features, permissions**. -1. Expand the **Repository** section -1. Enable or disable the **Pipelines** toggle as required. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Visibility, project features, permissions**. +1. In the **Repository** section, turn on or off **CI/CD** as required. **Project visibility** also affects pipeline visibility. If set to: - **Private**: Only project members can access pipelines. - **Internal** or **Public**: Pipelines can be set to either **Only Project Members** - or **Everyone With Access** via the dropdown box. + or **Everyone With Access** by using the dropdown box. Press **Save changes** for the settings to take effect. -## Site-wide admin setting +## Make GitLab CI/CD disabled by default in new projects -You can disable GitLab CI/CD site-wide, by modifying the settings in `gitlab.yml` -for source installations, and `gitlab.rb` for Omnibus installations. +You can set GitLab CI/CD to be disabled by default in all new projects by modifying the settings in: -Two things to note: +- `gitlab.yml` for source installations. +- `gitlab.rb` for Omnibus GitLab installations. -- Disabling GitLab CI/CD affects only newly-created projects. Projects that - had it enabled prior to this modification work as before. -- Even if you disable GitLab CI/CD, users can still enable it in the - project's settings. +Existing projects that already had CI/CD enabled are unchanged. Also, this setting only changes +the project default, so project owners can still enable CI/CD in the project settings. -For installations from source, open `gitlab.yml` with your editor and set -`builds` to `false`: +For installations from source: -```yaml -## Default project features settings -default_projects_features: - issues: true - merge_requests: true - wiki: true - snippets: false - builds: false -``` +1. Open `gitlab.yml` with your editor and set `builds` to `false`: -Save the file and restart GitLab: + ```yaml + ## Default project features settings + default_projects_features: + issues: true + merge_requests: true + wiki: true + snippets: false + builds: false + ``` -```shell -sudo service gitlab restart -``` +1. Save the `gitlab.yml` file. -For Omnibus installations, edit `/etc/gitlab/gitlab.rb` and add the line: +1. Restart GitLab: -```ruby -gitlab_rails['gitlab_default_projects_features_builds'] = false -``` + ```shell + sudo service gitlab restart + ``` -Save the file and reconfigure GitLab: +For Omnibus GitLab installations: -```shell -sudo gitlab-ctl reconfigure -``` +1. Edit `/etc/gitlab/gitlab.rb` and add this line: + + ```ruby + gitlab_rails['gitlab_default_projects_features_builds'] = false + ``` + +1. Save the `/etc/gitlab/gitlab.rb` file. + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` <!-- ## Troubleshooting diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 033e499d3d0..4e34cc7ce38 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -4,7 +4,7 @@ group: Release info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Deployment safety +# Deployment safety **(FREE)** Deployment jobs can be more sensitive than other jobs in a pipeline, and might need to be treated with extra care. GitLab has several features diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index 39e3dd1ca75..e473d52f957 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Incremental Rollouts with GitLab CI/CD +# Incremental rollouts with GitLab CI/CD **(FREE)** When rolling out changes to your application, it is possible to release production changes to only a portion of your Kubernetes pods as a risk mitigation strategy. By releasing @@ -22,9 +22,9 @@ Manual and Timed rollouts are included automatically in projects controlled by [Auto DevOps](../../topics/autodevops/index.md), but they are also configurable through GitLab CI/CD in the `.gitlab-ci.yml` configuration file. -Manually triggered rollouts can be implemented with your [Continuously Delivery](../introduction/index.md#continuous-delivery) +Manually triggered rollouts can be implemented with your [Continuous Delivery](../introduction/index.md#continuous-delivery) methodology, while timed rollouts do not require intervention and can be part of your -[Continuously Deployment](../introduction/index.md#continuous-deployment) strategy. +[Continuous Deployment](../introduction/index.md#continuous-deployment) strategy. You can also combine both of them in a way that the app is deployed automatically unless you eventually intervene manually if necessary. @@ -37,8 +37,6 @@ use as examples to build your own: ## Manual Rollouts -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5415) in GitLab 10.8. - It is possible to configure GitLab to do incremental rollouts manually through `.gitlab-ci.yml`. Manual configuration allows more control over the this feature. The steps in an incremental rollout depend on the number of pods that are defined for the deployment, which are configured when the Kubernetes @@ -77,8 +75,6 @@ available, demonstrating manually triggered incremental rollouts. ## Timed Rollouts -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7545) in GitLab 11.4. - Timed rollouts behave in the same way as manual rollouts, except that each job is defined with a delay in minutes before it deploys. Clicking the job reveals the countdown. diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index e647a704a87..1b4d8890c6e 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -31,7 +31,8 @@ Prerequisites: To view a list of environments and deployments: -1. Go to the project's **Deployments > Environments** page. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. The environments are displayed.  @@ -57,7 +58,8 @@ You can create an environment and deployment in the UI or in your `.gitlab-ci.ym In the UI: -1. Go to the project's **Deployments > Environments** page. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. 1. Select **New environment**. 1. Enter a name and external URL. 1. Select **Save**. @@ -326,7 +328,8 @@ If there is a problem with a deployment, you can retry it or roll it back. To retry or rollback a deployment: -1. Go to the project's **Deployments > Environments**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment. 1. To the right of the deployment name: - To retry a deployment, select **Re-deploy to environment**. @@ -353,12 +356,12 @@ For example:  -#### Going from source files to public pages +#### Go from source files to public pages With GitLab [Route Maps](../review_apps/index.md#route-maps), you can go directly from source files to public pages in the environment set for Review Apps. -### Stopping an environment +### Stop an environment When you stop an environment: @@ -465,7 +468,8 @@ GitLab automatically triggers the `stop_review_app` job to stop the environment. You can view a deployment's expiration date in the GitLab UI. -1. Go to the project's **Deployments > Environments** page. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. 1. Select the name of the deployment. In the top left, next to the environment name, the expiration date is displayed. @@ -474,7 +478,8 @@ In the top left, next to the environment name, the expiration date is displayed. You can manually override a deployment's expiration date. -1. Go to the project's **Deployments > Environments** page. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. 1. Select the deployment name. 1. On the top right, select the thumbtack (**{thumbtack}**). @@ -486,12 +491,13 @@ The `auto_stop_in` setting is overwritten and the environment remains active unt > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20620) in GitLab 12.10. -You can delete [stopped environments](#stopping-an-environment) in the GitLab UI or by using +You can delete [stopped environments](#stop-an-environment) in the GitLab UI or by using [the API](../../api/environments.md#delete-an-environment). To delete a stopped environment in the GitLab UI: -1. Go to the project's **Deployments > Environments** page. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. 1. Select the **Stopped** tab. 1. Next to the environment you want to delete, select **Delete environment**. 1. On the confirmation dialog box, select **Delete environment**. @@ -519,7 +525,7 @@ build: url: https://staging.example.com ``` -This gives you access to [environment-scoped variables](#scoping-environments-with-specs), +This gives you access to [environment-scoped variables](#scope-environments-with-specs), and can be used to [protect builds from unauthorized access](protected_environments.md). ### Group similar environments @@ -596,11 +602,13 @@ Limitations of GitLab Auto Rollback: GitLab Auto Rollback is turned off by default. To turn it on: -1. Go to **Project > Settings > CI/CD > Automatic deployment rollbacks**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Automatic deployment rollbacks**. 1. Select the checkbox for **Enable automatic rollbacks**. 1. Select **Save changes**. -### Monitoring environments +### Monitor environments To monitor the behavior of your app as it runs in each environment, enable [Prometheus for monitoring system and response metrics](../../user/project/integrations/prometheus.md). @@ -619,7 +627,7 @@ It may take a minute or two for data to appear after initial deployment.  -#### Embedding metrics in GitLab Flavored Markdown +#### Embed metrics in GitLab Flavored Markdown Metric charts can be embedded in GitLab Flavored Markdown. See [Embedding Metrics in GitLab Flavored Markdown](../../operations/metrics/embed.md) for more details. @@ -672,7 +680,7 @@ fetch line: fetch = +refs/environments/*:refs/remotes/origin/environments/* ``` -### Scoping environments with specs +### Scope environments with specs > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/2112) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.4. > - [Environment scoping for CI/CD variables was moved to all tiers](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30779) in GitLab 12.2. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 9bc3c97837c..3cd4ebdbdf1 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -30,18 +30,19 @@ To protect, update, or unprotect an environment, you need to have at least the To protect an environment: -1. Navigate to your project's **Settings > CI/CD**. -1. Expand the **Protected environments** section. -1. From the **Environment** dropdown menu, select the environment you want to protect. -1. In the **Allowed to Deploy** dropdown menu, select the role, users, or groups you +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Protected environments**. +1. From the **Environment** list, select the environment you want to protect. +1. In the **Allowed to deploy** list, select the role, users, or groups you want to give deploy access to. Keep in mind that: - There are two roles to choose from: - - **Maintainers**: Allows access to all maintainers in the project. - - **Developers**: Allows access to all maintainers and all developers in the project. - - You can only select groups that are already associated with the project. - - Only users that have at least the Developer role appear in - the **Allowed to Deploy** dropdown menu. -1. Click the **Protect** button. + - **Maintainers**: Allows access to all of the project's users with the Maintainer role. + - **Developers**: Allows access to all of the project's users with the Maintainer and Developer role. + - You can select groups that are already associated with the project only. + - Users must have at least the Developer role to appear in + the **Allowed to deploy** list. +1. Select **Protect**. The protected environment now appears in the list of protected environments. @@ -121,7 +122,7 @@ access to a protected environment through any of these methods: If the user also has push or merge access to the branch deployed on production, they have the following privileges: -- [Stopping an environment](index.md#stopping-an-environment). +- [Stop an environment](index.md#stop-an-environment). - [Delete a stopped environment](index.md#delete-a-stopped-environment). - [Create an environment terminal](index.md#web-terminals). @@ -134,10 +135,10 @@ appears in the dropdown menu for deployment-only access. To add deployment-only access: -1. Add a group with Reporter permissions. -1. Add user(s) to the group. +1. Add a group with the Reporter role. +1. Add users to the group. 1. Invite the group to be a project member. -1. Follow the steps outlined in [Protecting Environments](#protecting-environments). +1. Follow the steps in [Protecting Environments](#protecting-environments). Note that deployment-only access is the only possible access level for groups with [Reporter permissions](../../user/permissions.md). @@ -163,7 +164,7 @@ For more information, see [Deployment safety](deployment_safety.md). > - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-group-level-protected-environments). **(FREE SELF)** This in-development feature might not be available for your use. There can be -[risks when enabling features still in development](../../user/feature_flags.md#risks-when-enabling-features-still-in-development). +[risks when enabling features still in development](../../administration/feature_flags.md#risks-when-enabling-features-still-in-development). Refer to this feature's version history for more details. Typically, large enterprise organizations have an explicit permission boundary @@ -214,7 +215,7 @@ configured: - Operators should be assigned the [maintainer role](../../user/permissions.md) (or above) to the top-level group. They can maintain CI/CD configurations for the higher environments (such as production) in the group-level settings page, - wnich includes group-level protected environments, + which includes group-level protected environments, [group-level runners](../runners/runners_scope.md#group-runners), [group-level clusters](../../user/group/clusters/index.md), etc. Those configurations are inherited to the child projects as read-only entries. @@ -231,9 +232,9 @@ configured: cannot override it. - [Project-level protected environments](#protecting-environments) can be combined with the group-level setting. If both group-level and project-level - environment configurations exist, the user must be allowed in **both** - rulesets in order to run a deployment job. - - Within a project or a sub-group of the top-level group, developers can be + environment configurations exist, to run a deployment job, the user must be allowed in **both** + rulesets. + - In a project or a subgroup of the top-level group, developers can be safely assigned the Maintainer role to tune their lower environments (such as `testing`). diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 5ab8653dc35..0e6c2f63f9e 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -1,5 +1,6 @@ --- redirect_to: 'index.md' +remove_date: '2021-09-28' --- This document was moved to [another location](index.md). diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index 65fe9fb44d4..238c76b2c3c 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Authenticating and Reading Secrets With HashiCorp Vault +# Authenticating and reading secrets with HashiCorp Vault **(PREMIUM)** This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD. @@ -193,7 +193,7 @@ Combined with [protected branches](../../../user/project/protected_branches.md), For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). WARNING: -Always restrict your roles to project or namespace by using one of the provided claims (e.g. `project_id` or `namespace_id`). Otherwise any JWT generated by this instance may be allowed to authenticate using this role. +Always restrict your roles to project or namespace by using one of the provided claims (for example, `project_id` or `namespace_id`). Otherwise any JWT generated by this instance may be allowed to authenticate using this role. Now, configure the JWT Authentication method: diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 5ab8653dc35..0e6c2f63f9e 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -1,5 +1,6 @@ --- redirect_to: 'index.md' +remove_date: '2021-09-28' --- This document was moved to [another location](index.md). diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 62607320410..6817c7cac8e 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Running Composer and npm scripts with deployment via SCP in GitLab CI/CD +# Running Composer and npm scripts with deployment via SCP in GitLab CI/CD **(FREE)** This guide covers the building of dependencies of a PHP project while compiling assets via an npm script using [GitLab CI/CD](../../README.md). @@ -123,7 +123,7 @@ Therefore, for a production environment we use additional steps to ensure that a Since this was a WordPress project, I gave real life code snippets. Some further ideas you can pursue: - Having a slightly different script for the default branch allows you to deploy to a production server from that branch and to a stage server from any other branches. -- Instead of pushing it live, you can push it to WordPress official repository (with creating a SVN commit, etc.). +- Instead of pushing it live, you can push it to WordPress official repository. - You could generate i18n text domains on the fly. --- diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index 00c613865a3..14fb77dc49f 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Using Dpl as deployment tool +# Using Dpl as a deployment tool **(FREE)** [Dpl](https://github.com/travis-ci/dpl) (pronounced like the letters D-P-L) is a deploy tool made for continuous deployment that's developed and used by Travis CI, but can also be @@ -105,27 +105,31 @@ production: We created two deploy jobs that are executed on different events: -1. `staging` is executed for all commits that were pushed to `master` branch, -1. `production` is executed for all pushed tags. +- `staging`: Executed for all commits pushed to the `master` branch +- `production`: Executed for all pushed tags We also use two secure variables: -1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, -1. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. +- `HEROKU_STAGING_API_KEY`: Heroku API key used to deploy staging app +- `HEROKU_PRODUCTION_API_KEY`: Heroku API key used to deploy production app ## Storing API keys -To add secure variables, navigate to your project's -**Settings > CI/CD > Variables**. The variables that are defined -in the project settings are sent along with the build script to the runner. +To store API keys as secure variables: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Variables**. + +The variables defined in the project settings are sent along with the build script to the runner. The secure variables are stored out of the repository. Never store secrets in -your project's `.gitlab-ci.yml`. It is also important that the secret's value +your project's `.gitlab-ci.yml` file. It is also important that the secret's value is hidden in the job log. You access added variable by prefixing it's name with `$` (on non-Windows runners) or `%` (for Windows Batch runners): -1. `$VARIABLE` - use it for non-Windows runners -1. `%VARIABLE%` - use it for Windows Batch runners +- `$VARIABLE`: Use for non-Windows runners +- `%VARIABLE%`: Use for Windows Batch runners -Read more about the [CI/CD variables](../../variables/index.md). +Read more about [CI/CD variables](../../variables/index.md). diff --git a/doc/ci/examples/index.md b/doc/ci/examples/index.md index 9d6a4f53123..0569bb281b9 100644 --- a/doc/ci/examples/index.md +++ b/doc/ci/examples/index.md @@ -73,7 +73,7 @@ choose one of these templates: - [Django (`Django.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Django.gitlab-ci.yml) - [Docker (`Docker.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Docker.gitlab-ci.yml) - [dotNET (`dotNET.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET.gitlab-ci.yml) -- [dotNET Core (`dotNET-Core.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET-Core.yml) +- [dotNET Core (`dotNET-Core.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET-Core.gitlab-ci.yml) - [Elixir (`Elixir.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Elixir.gitlab-ci.yml) - [Flutter (`Flutter.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Flutter.gitlab-ci.yml) - [goLang (`Go.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Go.gitlab-ci.yml) diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index e7b9b9b51ad..eb758218f17 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -12,7 +12,7 @@ You can also view or fork the complete [example source](https://gitlab.com/gitla ## Initialize the module -1. Open a terminal and navigate to the project's repository +1. Open a terminal and navigate to the project's repository. 1. Run `npm init`. Name the module according to [the Package Registry's naming conventions](../../user/packages/npm_registry/index.md#package-naming-convention). For example, if the project's path is `gitlab-examples/semantic-release-npm`, name the module `@gitlab-examples/semantic-release-npm`. 1. Install the following npm packages: @@ -88,12 +88,22 @@ The default `before_script` generates a temporary `.npmrc` that is used to authe As part of publishing a package, semantic-release increases the version number in `package.json`. For semantic-release to commit this change and push it back to GitLab, the pipeline requires a custom CI/CD variable named `GITLAB_TOKEN`. To create this variable: -1. Navigate to **Project > Settings > Access Tokens**. -1. Give the token a name, and select the `api` scope. -1. Click **Create project access token** and copy its value. -1. Navigate to **Project > Settings > CI/CD > Variables**. -1. Click **Add Variable**. -1. In the **Key** field, enter `GITLAB_TOKEN`. In the **Value** field, paste the token created above. Check the **Mask variable** option and click **Add variable**. +<!-- markdownlint-disable MD044 --> +1. On the top bar, on the top right, select your avatar. +1. On the left sidebar, select **Access Tokens**. +1. In the **Token name** box, enter a token name. +1. Under **select scopes**, select the **api** checkbox. +1. Select **Create project access token**. +1. Copy the value. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Variables**. +1. Select **Add variable**. +1. In the **Key** box, enter `GITLAB_TOKEN`. +1. In the **Value** box, paste the token. +1. Select the **Mask variable** checkbox. +1. Select **Add variable**. +<!-- markdownlint-enable MD044 --> ## Configure semantic-release diff --git a/doc/ci/img/add_file_template_11_10.png b/doc/ci/img/add_file_template_11_10.png Binary files differdeleted file mode 100644 index ca04d72615b..00000000000 --- a/doc/ci/img/add_file_template_11_10.png +++ /dev/null diff --git a/doc/ci/index.md b/doc/ci/index.md index 593f50d4ffd..9175c20f580 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -61,7 +61,7 @@ GitLab CI/CD uses a number of concepts to describe and run your build and deploy | [Cache dependencies](caching/index.md) | Cache your dependencies for a faster execution. | | [GitLab Runner](https://docs.gitlab.com/runner/) | Configure your own runners to execute your scripts. | | [Pipeline efficiency](pipelines/pipeline_efficiency.md) | Configure your pipelines to run quickly and efficiently. | -| [Test cases](test_cases/index.md) | Configure your pipelines to run quickly and efficiently. <!--- this seems to be a duplicate description ---> | +| [Test cases](test_cases/index.md) | Create testing scenarios. | ## Configuration @@ -79,7 +79,7 @@ GitLab CI/CD supports numerous configuration options: | [Optimize GitLab and GitLab Runner for large repositories](large_repositories/index.md) | Recommended strategies for handling large repositories. | | [`.gitlab-ci.yml` full reference](yaml/index.md) | All the attributes you can use with GitLab CI/CD. | -Note that certain operations can only be performed according to the +Certain operations can only be performed according to the [user](../user/permissions.md#gitlab-cicd-permissions) and [job](../user/permissions.md#job-permissions) permissions. ## Feature set diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index 46abcf4a21d..7e72bd44503 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -33,8 +33,8 @@ Two things need to be configured for the interactive web terminal to work: NOTE: Interactive web terminals are not yet supported by -[`gitlab-runner` Helm chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-runner/index.html), -but support [is planned](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/79). +[`gitlab-runner` Helm chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-runner/index.html). +Support is tracked [in this issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/79). ## Debugging a running job diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 3fe30c78d6a..aa12e1181cb 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -168,7 +168,7 @@ for a single run of the manual job. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21767) in GitLab 11.4. -When you do not want to run a job immediately, you can use the [`when:delayed`](../yaml/index.md#whendelayed) keyword to +When you do not want to run a job immediately, you can use the [`when:delayed`](../jobs/job_control.md#run-a-job-after-a-delay) keyword to delay a job's execution for a certain period. This is especially useful for timed incremental rollout where new code is rolled out gradually. @@ -259,3 +259,27 @@ job1: - echo 'this line should be hidden automatically after loading the job log' - echo -e "\e[0Ksection_end:`date +%s`:my_first_section\r\e[0K" ``` + +## Deployment jobs + +Deployment jobs are a specific kind of CI job in that they deploy code to +[environments](../environments/index.md). A deployment job is any job that +uses the `environment` keyword and the [`start` environment `action`](../yaml/index.md#environmentaction). +Deployment jobs do not need to be in the `deploy` stage. The following `deploy me` +job is an example of a deployment job. `action: start` is the default behavior and +is defined for the sake of the example, but you can omit it: + +```yaml +deploy me: + script: + - deploy-to-cats.sh + environment: + name: production + url: https://cats.example.com + action: start +``` + +The behavior of deployment jobs can be controlled with +[deployment safety](../environments/deployment_safety.md) settings like +[skipping outdated deployment jobs](../environments/deployment_safety.md#prevent-deployments-during-deploy-freeze-windows) +and [ensuring only one deployment job runs at a time](../environments/deployment_safety.md#ensure-only-one-deployment-job-runs-at-a-time). diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 634214aedc3..b8b05426297 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -497,6 +497,120 @@ test: - "README.md" ``` +## Create a job that must be run manually + +You can require that a job doesn't run unless a user starts it. This is called a **manual job**. +You might want to use a manual job for something like deploying to production. + +To specify a job as manual, add [`when: manual`](../yaml/index.md#when) to the job +in the `.gitlab-ci.yml` file. + +By default, manual jobs display as skipped when the pipeline starts. + +You can use [protected branches](../../user/project/protected_branches.md) to more strictly +[protect manual deployments](#protect-manual-jobs) from being run by unauthorized users. + +### Types of manual jobs + +Manual jobs can be either optional or blocking: + +- **Optional**: The default setting for manual jobs. + - They have [`allow_failure: true`](../yaml/index.md#allow_failure) by default. + - The status does not contribute to the overall pipeline status. A pipeline can + succeed even if all of its manual jobs fail. +- **Blocking**: An optional setting for manual jobs. + - Add `allow_failure: false` to the job configuration. + - The pipeline stops at the stage where the job is defined. To let the pipeline + continue running, [run the manual job](#run-a-manual-job). + - Merge requests in projects with [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) + enabled can't be merged with a blocked pipeline. Blocked pipelines show a status + of **blocked**. + +### Run a manual job + +To run a manual job, you must have permission to merge to the assigned branch. + +To run a manual job: + +1. Go to the pipeline, job, [environment](../environments/index.md#configure-manual-deployments), + or deployment view. +1. Next to the manual job, select **Play** (**{play}**). + +### Protect manual jobs **(PREMIUM)** + +Use [protected environments](../environments/protected_environments.md) +to define a list of users authorized to run a manual job. You can authorize only +the users associated with a protected environment to trigger manual jobs, which can: + +- More precisely limit who can deploy to an environment. +- Block a pipeline until an approved user "approves" it. + +To protect a manual job: + +1. Add an `environment` to the job. For example: + + ```yaml + deploy_prod: + stage: deploy + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + ``` + +1. In the [protected environments settings](../environments/protected_environments.md#protecting-environments), + select the environment (`production` in this example) and add the users, roles or groups + that are authorized to trigger the manual job to the **Allowed to Deploy** list. Only those in + this list can trigger this manual job, as well as GitLab administrators + who are always able to use protected environments. + +You can use protected environments with blocking manual jobs to have a list of users +allowed to approve later pipeline stages. Add `allow_failure: false` to the protected +manual job and the pipeline's next stages only run after the manual job is triggered +by authorized users. + +## Run a job after a delay + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51352) in GitLab 11.4. + +Use [`when: delayed`](../yaml/index.md#when) to execute scripts after a waiting period, or if you want to avoid +jobs immediately entering the `pending` state. + +You can set the period with `start_in` keyword. The value of `start_in` is an elapsed time in seconds, unless a unit is +provided. `start_in` must be less than or equal to one week. Examples of valid values include: + +- `'5'` (a value with no unit must be surrounded by single quotes) +- `5 seconds` +- `30 minutes` +- `1 day` +- `1 week` + +When a stage includes a delayed job, the pipeline doesn't progress until the delayed job finishes. +You can use this keyword to insert delays between different stages. + +The timer of a delayed job starts immediately after the previous stage completes. +Similar to other types of jobs, a delayed job's timer doesn't start unless the previous stage passes. + +The following example creates a job named `timed rollout 10%` that is executed 30 minutes after the previous stage completes: + +```yaml +timed rollout 10%: + stage: deploy + script: echo 'Rolling out 10% ...' + when: delayed + start_in: 30 minutes +``` + +To stop the active timer of a delayed job, click the **{time-out}** (**Unschedule**) button. +This job can no longer be scheduled to run automatically. You can, however, execute the job manually. + +To start a delayed job immediately, select **Play** (**{play}**). +Soon GitLab Runner starts the job. + ## Use predefined CI/CD variables to run jobs only in specific pipeline types You can use [predefined CI/CD variables](../variables/predefined_variables.md) to choose diff --git a/doc/ci/lint.md b/doc/ci/lint.md index 746638442a7..4e3ac8b9e93 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -39,8 +39,8 @@ more complicated issues. To validate the configuration by running a pipeline simulation: 1. Paste the GitLab CI configuration to verify into the text box. -1. Click the **Simulate pipeline creation for the default branch** checkbox. -1. Click **Validate**. +1. Select the **Simulate pipeline creation for the default branch** checkbox. +1. Select **Validate**.  diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index c852800d0a9..13f4ca428c6 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -1,5 +1,6 @@ --- redirect_to: '../pipelines/merge_request_pipelines.md' +remove_date: '2021-09-29' --- This document was moved to [another location](../pipelines/merge_request_pipelines.md). diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index 76a79ba1356..5b68c4ca931 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -1,5 +1,6 @@ --- redirect_to: '../../pipelines/pipelines_for_merged_results.md' +remove_date: '2021-09-29' --- This document was moved to [another location](../../pipelines/pipelines_for_merged_results.md). diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index 8b2316fd340..c8e8dffbdcd 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -1,5 +1,6 @@ --- redirect_to: '../../../pipelines/merge_trains.md' +remove_date: '2021-09-29' --- This document was moved to [another location](../../../pipelines/merge_trains.md). diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 0b0226f428a..9a220121f54 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -9,7 +9,7 @@ type: reference > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9788) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.10. Requires GitLab Runner 11.10 and above. -GitLab provides a lot of great reporting tools for [merge requests](../user/project/merge_requests/index.md) - [Unit test reports](unit_test_reports.md), [code quality](../user/project/merge_requests/code_quality.md), performance tests, etc. While JUnit is a great open framework for tests that "pass" or "fail", it is also important to see other types of metrics from a given change. +GitLab provides a lot of great reporting tools for things like [merge requests](../user/project/merge_requests/index.md) - [Unit test reports](unit_test_reports.md), [code quality](../user/project/merge_requests/code_quality.md), and performance tests. While JUnit is a great open framework for tests that "pass" or "fail", it is also important to see other types of metrics from a given change. You can configure your job to use custom Metrics Reports, and GitLab displays a report on the merge request so that it's easier and faster to identify changes without having to check the entire log. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index 968adf2e161..106f5b3d819 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -188,7 +188,7 @@ release-branch-workflow: - testing ``` -Example of the same workflow using [`when: manual`](../yaml/index.md#whenmanual) in GitLab CI/CD: +Example of the same workflow using [`when: manual`](../jobs/job_control.md#create-a-job-that-must-be-run-manually) in GitLab CI/CD: ```yaml deploy_prod: diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index 9f2115fa4a0..1a987a0ce47 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -16,7 +16,7 @@ The following list of recommended steps was created after observing organization that were able to quickly complete this migration: 1. Start by reading the GitLab CI/CD [Quick Start Guide](../quick_start/index.md) and [important product differences](#important-product-differences). -1. Learn the importance of [managing the organizational transition](#managing-the-organizational-transition). +1. Learn the importance of [managing the organizational transition](#manage-organizational-transition). 1. [Add runners](../runners/index.md) to your GitLab instance. 1. Educate and enable your developers to independently perform the following steps in their projects: 1. Review the [Quick Start Guide](../quick_start/index.md) and [Pipeline Configuration Reference](../yaml/index.md). @@ -39,7 +39,7 @@ to GitLab! If you have questions that are not answered here, the [GitLab community forum](https://forum.gitlab.com/) can be a great resource. -## Managing the organizational transition +## Manage organizational transition An important part of transitioning from Jenkins to GitLab is the cultural and organizational changes that comes with the move, and successfully managing them. There are a few @@ -89,13 +89,13 @@ There are some high level differences between the products worth mentioning: permissions), and then any project can use them. This central project could also contain scripts or other reusable code. - You can also use the [`extends` keyword](../yaml/index.md#extends) to reuse configuration - within a single pipeline configuration. -- All jobs within a single stage always run in parallel, and all stages run in sequence. We are planning + in a single pipeline configuration. +- All jobs in a single stage always run in parallel, and all stages run in sequence. We are planning to allow certain jobs to break this sequencing as needed with our [directed acyclic graph](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47063) feature. - The [`parallel`](../yaml/index.md#parallel) keyword can automatically parallelize tasks, like tests that support parallelization. -- Normally all jobs within a single stage run in parallel, and all stages run in sequence. +- Normally all jobs in a single stage run in parallel, and all stages run in sequence. There are different [pipeline architectures](../pipelines/pipeline_architectures.md) that allow you to change this behavior. - The new [`rules` syntax](../yaml/index.md#rules) is the recommended method of @@ -104,12 +104,12 @@ There are some high level differences between the products worth mentioning: fresh environment in each job. Passing artifacts between jobs is controlled using the [`artifacts`](../yaml/index.md#artifacts) and [`dependencies`](../yaml/index.md#dependencies) keywords. When finished, use the planned [Workspaces](https://gitlab.com/gitlab-org/gitlab/-/issues/29265) - feature to more easily persist a common workspace between serial jobs. + feature to persist a common workspace between serial jobs. - The `.gitlab-ci.yml` file is checked in to the root of your repository, much like a Jenkinsfile, but is in the YAML format (see [complete reference](../yaml/index.md)) instead of a Groovy DSL. It's most analogous to the declarative Jenkinsfile format. -- Manual approvals or gates can be set up as [`when:manual` jobs](../yaml/index.md#whenmanual). These can - also leverage [`protected environments`](../yaml/index.md#protecting-manual-jobs) +- Manual approvals or gates can be set up as [`when:manual` jobs](../jobs/job_control.md#create-a-job-that-must-be-run-manually). These can + also leverage [`protected environments`](../jobs/job_control.md#run-a-job-after-a-delay) to control who is able to approve them. - GitLab comes with a [container registry](../../user/packages/container_registry/index.md), and we recommend using container images to set up your build environment. For example, set up one pipeline that builds your build environment @@ -122,7 +122,7 @@ There are some high level differences between the products worth mentioning: ## Agents vs. runners Both Jenkins agents and GitLab runners are the hosts that run jobs. To convert the -Jenkins agent, simply uninstall it and then [install and register the runner](../runners/index.md). +Jenkins agent, uninstall it and then [install and register the runner](../runners/index.md). Runners do not require much overhead, so you can size them similarly to the Jenkins agents you were using. @@ -215,7 +215,7 @@ be used by all projects in the group. An instance administrator can set a group the source for [instance project templates](../../user/group/custom_project_templates.md), which can be used by projects in that instance. -## Converting a declarative Jenkinsfile +## Convert a declarative Jenkinsfile A declarative Jenkinsfile contains "Sections" and "Directives" which are used to control the behavior of your pipelines. There are equivalents for all of these in GitLab, which we've documented below. diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 06cbf63e512..93e04bf8a0e 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -1,5 +1,6 @@ --- redirect_to: 'pipelines/multi_project_pipelines.md' +remove_date: '2021-09-29' --- This document was moved to [another location](pipelines/multi_project_pipelines.md). diff --git a/doc/ci/parent_child_pipelines.md b/doc/ci/parent_child_pipelines.md index 0b8df2e9f4c..f2edc263397 100644 --- a/doc/ci/parent_child_pipelines.md +++ b/doc/ci/parent_child_pipelines.md @@ -1,5 +1,6 @@ --- redirect_to: 'pipelines/parent_child_pipelines.md' +remove_date: '2021-09-29' --- This document was moved to [another location](pipelines/parent_child_pipelines.md). diff --git a/doc/ci/pipelines/img/coverage_check_approval_rule_14_1.png b/doc/ci/pipelines/img/coverage_check_approval_rule_14_1.png Binary files differindex 00eb5c84ca9..8c1d005e074 100644 --- a/doc/ci/pipelines/img/coverage_check_approval_rule_14_1.png +++ b/doc/ci/pipelines/img/coverage_check_approval_rule_14_1.png diff --git a/doc/ci/pipelines/img/pipelines_graph_dependency_view_hover_v13_12.png b/doc/ci/pipelines/img/pipelines_graph_dependency_view_hover_v13_12.png Binary files differindex ff6b3af0a28..8c4b9d73cdc 100644 --- a/doc/ci/pipelines/img/pipelines_graph_dependency_view_hover_v13_12.png +++ b/doc/ci/pipelines/img/pipelines_graph_dependency_view_hover_v13_12.png diff --git a/doc/ci/pipelines/img/pipelines_graph_dependency_view_links_v13_12.png b/doc/ci/pipelines/img/pipelines_graph_dependency_view_links_v13_12.png Binary files differindex b0923ab96d9..4ad68849e2a 100644 --- a/doc/ci/pipelines/img/pipelines_graph_dependency_view_links_v13_12.png +++ b/doc/ci/pipelines/img/pipelines_graph_dependency_view_links_v13_12.png diff --git a/doc/ci/pipelines/img/pipelines_graph_dependency_view_v13_12.png b/doc/ci/pipelines/img/pipelines_graph_dependency_view_v13_12.png Binary files differindex ae7cdc5b43e..2bb399f61f9 100644 --- a/doc/ci/pipelines/img/pipelines_graph_dependency_view_v13_12.png +++ b/doc/ci/pipelines/img/pipelines_graph_dependency_view_v13_12.png diff --git a/doc/ci/pipelines/img/pipelines_graph_stage_view_v13_12.png b/doc/ci/pipelines/img/pipelines_graph_stage_view_v13_12.png Binary files differindex b3b98313350..f2f38979e18 100644 --- a/doc/ci/pipelines/img/pipelines_graph_stage_view_v13_12.png +++ b/doc/ci/pipelines/img/pipelines_graph_stage_view_v13_12.png diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 4f818d658b7..22f18235f99 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -53,7 +53,7 @@ Pipelines can be configured in many different ways: - [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. - [Parent-Child pipelines](parent_child_pipelines.md) break down complex pipelines into one parent pipeline that can trigger multiple child sub-pipelines, which all - run in the same project and with the same SHA. + run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. - [Pipelines for Merge Requests](../pipelines/merge_request_pipelines.md) run for merge requests only (rather than for every commit). - [Pipelines for Merged Results](../pipelines/pipelines_for_merged_results.md) @@ -94,7 +94,7 @@ This table lists the refspecs injected for each pipeline type: The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a running pipeline job. This ref can be created even after the associated branch or tag has been -deleted. It's therefore useful in some features such as [automatically stopping an environment](../environments/index.md#stopping-an-environment), +deleted. It's therefore useful in some features such as [automatically stopping an environment](../environments/index.md#stop-an-environment), and [merge trains](../pipelines/merge_trains.md) that might run pipelines after branch deletion. @@ -123,6 +123,9 @@ you can filter the pipeline list by: - Status ([GitLab 13.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/217617)) - Tag ([GitLab 13.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/217617)) +[Starting in GitLab 14.2](https://gitlab.com/gitlab-org/gitlab/-/issues/26621), you can change the +pipeline column to display the pipeline ID or the pipeline IID. + ### Run a pipeline manually Pipelines can be manually executed, with predefined or manually-specified [variables](../variables/index.md). @@ -132,13 +135,13 @@ operation of the pipeline. To execute a pipeline manually: -1. Navigate to your project's **CI/CD > Pipelines**. -1. Select the **Run pipeline** button. -1. On the **Run pipeline** page: - 1. Select the branch or tag to run the pipeline for in the **Run for branch name or tag** field. - 1. Enter any [environment variables](../variables/index.md) required for the pipeline run. - You can set specific variables to have their [values prefilled in the form](#prefill-variables-in-manual-pipelines). - 1. Click the **Run pipeline** button. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Pipelines**. +1. Select **Run pipeline**. +1. In the **Run for branch name or tag** field, select the branch or tag to run the pipeline for. +1. Enter any [environment variables](../variables/index.md) required for the pipeline to run. + You can set specific variables to have their [values prefilled in the form](#prefill-variables-in-manual-pipelines). +1. Select **Run pipeline**. The pipeline now executes the jobs as configured. @@ -167,6 +170,9 @@ variables: You cannot set job-level variables to be pre-filled when you run a pipeline manually. +Pre-filled variables do not show up when the CI/CD configuration is [external to the project](settings.md#specify-a-custom-cicd-configuration-file). +See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336184) for more details. + ### Run a pipeline by using a URL query string > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24146) in GitLab 12.5. @@ -200,7 +206,7 @@ For each `var` or `file_var`, a key and value are required. ### Add manual interaction to your pipeline -Manual actions, configured using the [`when:manual`](../yaml/index.md#whenmanual) keyword, +[Manual jobs](../jobs/job_control.md#create-a-job-that-must-be-run-manually), allow you to require manual interaction before moving forward in the pipeline. You can do this straight from the pipeline graph. Just click the play button @@ -357,12 +363,10 @@ you visualize the entire pipeline, including all cross-project inter-dependencie > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/298973) in GitLab 13.12. > - [Deployed behind a feature flag](../../user/feature_flags.md), disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/328538) in GitLab 13.12. -> - Enabled on GitLab.com. -> - Recommended for production use. -> - To disable in GitLab self-managed instances, ask a GitLab administrator to [disable it](#enable-or-disable-job-dependency-view). **(FREE SELF)** +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328538) in GitLab 14.2. This in-development feature might not be available for your use. There can be -[risks when enabling features still in development](../../user/feature_flags.md#risks-when-enabling-features-still-in-development). +[risks when enabling features still in development](../../administration/feature_flags.md#risks-when-enabling-features-still-in-development). Refer to this feature's version history for more details. You can arrange jobs in the pipeline graph based on their [`needs`](../yaml/index.md#needs) @@ -385,29 +389,11 @@ To see the full `needs` dependency tree for a job, hover over it:  -#### Enable or disable job dependency view **(FREE SELF)** - -The job dependency view is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can disable it. - -To enable it: - -```ruby -Feature.enable(:pipeline_graph_layers_view) -``` - -To disable it: - -```ruby -Feature.disable(:pipeline_graph_layers_view) -``` - ### Pipeline mini graphs Pipeline mini graphs take less space and can tell you at a quick glance if all jobs passed or something failed. The pipeline mini graph can -be found when you navigate to: +be found when you go to: - The pipelines index page. - A single commit page. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index b9a42c76293..c6b6f61ef11 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -100,8 +100,8 @@ artifacts and log. You must be: To delete a job: 1. Go to a job's detail page. -1. At the top right of the job's log, select the trash icon. -1. Confirm the deletion. +1. On the top right of the job's log, select **Erase job log** (**{remove}**). +1. On the confirmation dialog, select **OK**. ## Retrieve job artifacts for other projects @@ -186,7 +186,9 @@ Keeping the latest artifacts can use a large amount of storage space in projects with a lot of jobs or large artifacts. If the latest artifacts are not needed in a project, you can disable this behavior to save space: -1. Go to the project's **Settings > CI/CD > Artifacts**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Artifacts**. 1. Clear the **Keep artifacts from most recent successful jobs** checkbox. You can disable this behavior for all projects on a self-managed instance in the diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 29c12551f12..b3dfe8753c7 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -16,8 +16,8 @@ changes are pushed to a branch. If you want the pipeline to run jobs **only** on commits associated with a merge request, you can use *pipelines for merge requests*. -In the UI, these pipelines are labeled as `detached`. Otherwise, these pipelines are the same -as other pipelines. +These pipelines are labeled as `detached` in the UI, and they do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable). +Otherwise, these pipelines are the same as other pipelines. Pipelines for merge requests can run when you: diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 3e6ad071d7e..06c1a6fef44 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -6,12 +6,12 @@ type: reference last_update: 2019-07-03 --- -# Merge Trains **(PREMIUM)** +# Merge trains **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9186) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.0. > - [Squash and merge](../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13001) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.6. -For more information about why you might want to use Merge Trains, read [How merge trains keep your master green](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). +For more information about why you might want to use merge trains, read [How merge trains keep your master green](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). When [pipelines for merged results](pipelines_for_merged_results.md) are enabled, the pipeline jobs run as if the changes from your source branch have already @@ -63,7 +63,7 @@ Read more about [how merge trains keep your master green](https://about.gitlab.c <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch this video for a demonstration on [how parallel execution -of Merge Trains can prevent commits from breaking the default +of merge trains can prevent commits from breaking the default branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ). ## Prerequisites @@ -83,11 +83,13 @@ To enable merge trains for your project: 1. If you are on a self-managed GitLab instance, ensure the [feature flag](#merge-trains-feature-flag) is set correctly. 1. [Configure your CI/CD configuration file](merge_request_pipelines.md#configure-pipelines-for-merge-requests) so that the pipeline or individual jobs run for merge requests. -1. Visit your project's **Settings > General** and expand **Merge requests**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Merge requests**. 1. In the **Merge method** section, verify that **Merge commit** is selected. You cannot use **Merge commit with semi-linear history** or **Fast-forward merge** with merge trains. -1. In the **Merge options** section, select **Enable merged results pipelines.** (if not already selected) and **Enable merge trains.** -1. Click **Save changes** +1. In the **Merge options** section, select **Enable merged results pipelines** (if not already selected) and **Enable merge trains**. +1. Select **Save changes**. In GitLab 13.5 and earlier, there is only one checkbox, named **Enable merge trains and pipelines for merged results**. @@ -102,7 +104,7 @@ unresolved state or your pipelines may be dropped. To start a merge train: 1. Visit a merge request. -1. Click the **Start merge train** button. +1. Select **Start merge train**.  @@ -113,7 +115,7 @@ Other merge requests can now be added to the train. To add a merge request to a merge train: 1. Visit a merge request. -1. Click the **Add to merge train** button. +1. Select **Add to merge train**. If pipelines are already running for the merge request, you cannot add the merge request to the train. Instead, you can schedule to add the merge request to a merge train **when the latest @@ -124,7 +126,7 @@ pipeline succeeds**. ## Remove a merge request from a merge train 1. Visit a merge request. -1. Click the **Remove from merge train** button. +1. Select **Remove from merge train**.  @@ -153,31 +155,32 @@ is recreated and all pipelines restart. ### Merge request dropped from the merge train immediately -If a merge request is not mergeable (for example, it's a draft merge request, there is a merge -conflict, etc.), your merge request is dropped from the merge train automatically. +If a merge request is not mergeable (for example, it's a draft merge request or it has a merge +conflict), the merge train drops your merge request automatically. In these cases, the reason for dropping the merge request is in the **system notes**. To check the reason: 1. Open the merge request that was dropped from the merge train. -1. Open the **Discussion** tab. +1. Select the **Discussion** tab. 1. Find a system note that includes either: - - The text **... removed this merge request from the merge train because ...** + - **... removed this merge request from the merge train because ...** - **... aborted this merge request from the merge train because ...** - The reason is given in the text after the **because ...** phrase. - +The reason is given in the text after the **because ...** phrase. + + ### Merge When Pipeline Succeeds cannot be chosen [Merge When Pipeline Succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) -is currently unavailable when Merge Trains are enabled. +is currently unavailable when merge trains are enabled. See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/12267) for more information. -### Merge Train Pipeline cannot be retried +### Merge train pipeline cannot be retried When a pipeline for merge trains fails the merge request is dropped from the train and the pipeline can't be retried. Pipelines for merge trains run on the merged result of the changes in the merge request and @@ -189,7 +192,7 @@ again, which triggers a new pipeline. ### Unable to add to merge train with message "The pipeline for this merge request failed." -Sometimes the **Start/Add to Merge Train** button is not available and the merge request says, +Sometimes the **Start/Add to merge train** button is not available and the merge request says, "The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure." This issue occurs when [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) @@ -197,19 +200,19 @@ is enabled in **Settings > General > Merge requests**. This option requires that run a new successful pipeline before you can re-add a merge request to a merge train. Merge trains ensure that each pipeline has succeeded before a merge happens, so -you can clear the **Pipelines must succeed** check box and keep -**Enable merge trains and pipelines for merged results** (merge trains) enabled. +you can clear the **Pipelines must succeed** checkbox and keep +**Enable merge trains and pipelines for merged results** (merge trains) selected. -If you want to keep the **Pipelines must succeed** option enabled along with Merge -Trains, create a new pipeline for merged results when this error occurs: +If you want to keep the **Pipelines must succeed** option selected along with merge +trains, create a new pipeline for merged results when this error occurs: -1. Go to the **Pipelines** tab and click **Run pipeline**. -1. Click **Start/Add to merge train when pipeline succeeds**. +1. On the **Pipelines** tab, select **Run pipeline**. +1. Select **Start/Add to merge train when pipeline succeeds**. See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/35135) for more information. -### Merge Trains feature flag **(PREMIUM SELF)** +### Merge trains feature flag **(PREMIUM SELF)** In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/244831), you can [enable or disable merge trains in the project settings](#enable-merge-trains). @@ -227,7 +230,7 @@ Feature.enable(:disable_merge_trains) ``` After you enable this feature flag, all existing merge trains are cancelled and -the **Start/Add to Merge Train** button no longer appears in merge requests. +the **Start/Add to merge train** button no longer appears in merge requests. To disable the feature flag, and enable merge trains again: diff --git a/doc/ci/pipelines/multi_project_pipelines.md b/doc/ci/pipelines/multi_project_pipelines.md index e3fe0fd20f5..3007d91d1b4 100644 --- a/doc/ci/pipelines/multi_project_pipelines.md +++ b/doc/ci/pipelines/multi_project_pipelines.md @@ -304,8 +304,8 @@ Prerequisites: To trigger the pipeline when the upstream project is rebuilt: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD** page. -1. Expand the **Pipeline subscriptions** section. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Pipeline subscriptions**. 1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. 1. Select **Subscribe**. diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index 2e29f4fe812..b266a721d11 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -157,9 +157,13 @@ build a matrix of targets and architectures. For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM). <!-- vale gitlab.Spelling = NO --> + We also have an example project using [Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) -which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. You could use a similar process for other templating languages like [Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/). +which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. +You could use a similar process for other templating languages like +[Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/). + <!-- vale gitlab.Spelling = NO --> The artifact path is parsed by GitLab, not the runner, so the path must match the diff --git a/doc/ci/pipelines/pipelines_for_merged_results.md b/doc/ci/pipelines/pipelines_for_merged_results.md index efa6a373ef3..08d7d119787 100644 --- a/doc/ci/pipelines/pipelines_for_merged_results.md +++ b/doc/ci/pipelines/pipelines_for_merged_results.md @@ -64,7 +64,7 @@ To enable pipelines for merged results for your project: 1. Click **Save changes**. WARNING: -If you select the check box but don't configure your CI/CD to use +If you select the checkbox but don't configure your CI/CD to use pipelines for merge requests, your merge requests may become stuck in an unresolved state or your pipelines may be dropped. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index db6fa7f4d23..b48c62dccc5 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -122,8 +122,13 @@ If the CI/CD configuration file is on an external site, the URL must end with `. - `http://example.com/generate/ci/config.yml` -If the CI/CD configuration file is in a different project, the path must be relative -to the root directory in the other project. Include the group and project name at the end: +If the CI/CD configuration file is in a different project: + +- The file must exist on its default branch. +- The path must be relative to the root directory in the other project. +- The path must include the group and project name at the end. + +For example: - `.gitlab-ci.yml@mygroup/another-project` - `my/path/.my-custom-file.yml@mygroup/another-project` @@ -228,6 +233,7 @@ Use this regex for commonly used test tools. - `go test -cover` (Go). Example: `coverage: \d+.\d+% of statements`. - .NET (OpenCover). Example: `(Visited Points).*\((.*)\)`. - .NET (`dotnet test` line coverage). Example: `Total\s*\|\s*(\d+(?:\.\d+)?)`. +- tarpaulin (Rust). Example: `^\d+.\d+% coverage`. <!-- vale gitlab.Spelling = YES --> @@ -260,6 +266,7 @@ when merging a merge request would cause the project's test coverage to decline. Follow these steps to enable the `Coverage-Check` MR approval rule: +1. Set up a [`coverage:`](../yaml/index.md#coverage) regular expression for all jobs you want to include in the overall coverage value. 1. Go to your project and select **Settings > General**. 1. Expand **Merge request approvals**. 1. Select **Enable** next to the `Coverage-Check` approval rule. diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index 8d800d49be3..257e170b0a5 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -165,8 +165,3 @@ To view your pipeline:  If the job status is `stuck`, check to ensure a runner is properly configured for the project. - -> To learn more about GitLab CI/CD, check out these video walkthroughs: -> -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [First time GitLab & CI/CD](https://www.youtube.com/watch?v=kTNfi5z6Uvk&t=150s). -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [Intro to GitLab CI](https://www.youtube.com/watch?v=l5705U8s_nQ&t=358s). diff --git a/doc/ci/review_apps/img/review_button.png b/doc/ci/review_apps/img/review_button.png Binary files differdeleted file mode 100644 index 4f1cf4d7cfd..00000000000 --- a/doc/ci/review_apps/img/review_button.png +++ /dev/null diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 1ed736de1e8..8af04388f92 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -5,19 +5,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Review Apps +# Review Apps **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21971) in GitLab 8.12. Further additions were made in GitLab 8.13 and 8.14. -> - Inspired by [Heroku's Review Apps](https://devcenter.heroku.com/articles/github-integration-review-apps), which itself was inspired by [Fourchette](https://github.com/rainforestapp/fourchette). - -Review Apps is a collaboration tool that takes the hard work out of providing an environment to showcase product changes. +Review Apps is a collaboration tool that assists with providing an environment to showcase product changes. NOTE: If you have a Kubernetes cluster, you can automate this feature in your applications by using [Auto DevOps](../../topics/autodevops/index.md). -## Introduction - Review Apps: - Provide an automatic live preview of changes made in a feature branch by spinning up a dynamic environment for your merge requests. @@ -27,7 +22,7 @@ Review Apps:  -In the above example: +In the previous example: - A Review App is built every time a commit is pushed to `topic branch`. - The reviewer fails two reviews before passing the third review. @@ -46,13 +41,13 @@ The following is an example of a merge request with an environment set dynamical In this example, a branch was: - Successfully built. -- Deployed under a dynamic environment that can be reached by clicking on the **View app** button. +- Deployed under a dynamic environment that can be reached by selecting **View app**. After adding Review Apps to your workflow, you follow the branched Git flow. That is: 1. Push a branch and let the runner deploy the Review App based on the `script` definition of the dynamic environment job. 1. Wait for the runner to build and deploy your web application. -1. Click on the link provided in the merge request related to the branch to see the changes live. +1. To view the changes live, select the link in the merge request related to the branch. ## Configuring Review Apps @@ -65,27 +60,32 @@ The process of configuring Review Apps is as follows: 1. Set up a job in `.gitlab-ci.yml` that uses the [predefined CI/CD variable](../variables/index.md) `${CI_COMMIT_REF_NAME}` to create dynamic environments and restrict it to run only on branches. Alternatively, you can get a YML template for this job by [enabling review apps](#enable-review-apps-button) for your project. -1. Optionally, set a job that [manually stops](../environments/index.md#stopping-an-environment) the Review Apps. +1. Optionally, set a job that [manually stops](../environments/index.md#stop-an-environment) the Review Apps. ### Enable Review Apps button > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118844) in GitLab 12.8. -When configuring Review Apps for a project, you need to add a new job to `.gitlab-ci.yml`, -as mentioned above. To facilitate this and if you are using Kubernetes, you can click -the **Enable Review Apps** button and GitLab prompts you with a template code block that -you can copy and paste into `.gitlab-ci.yml` as a starting point. To do so: +When configuring Review Apps for a project, you add a new job to the `.gitlab-ci.yml` file, +as mentioned above. To facilitate this, and if you are using Kubernetes, you can select +**Enable Review Apps** and GitLab prompts you with a template code block that +you can copy and paste into `.gitlab-ci.yml` as a starting point. + +Prerequisite: + +- You need at least the Developer [role](../../user/permissions.md) for the project. -1. Go to the project your want to create a Review App job for. -1. From the left nav, go to **Deployments > Environments**. -1. Click on the **Enable Review Apps** button. It is available to you - if you have Developer or higher [permissions](../../user/permissions.md) to that project. +To use the Review Apps template: + +1. On the top bar, select **Menu > Projects** and find the project you want to create a Review App job for. +1. On the left sidebar, select **Deployments > Environments**. +1. Select **Enable Review Apps**. 1. Copy the provided code snippet and paste it into your `.gitlab-ci.yml` file:  -1. Feel free to tune this template to your own needs. +You can edit this template as needed. ## Review Apps auto-stop @@ -107,8 +107,6 @@ Other examples of Review Apps: ## Route Maps -> Introduced in GitLab 8.17. In GitLab 11.5, the file links are available in the merge request widget. - Route Maps allows you to go directly from source files to public pages on the [environment](../environments/index.md) defined for Review Apps. @@ -175,9 +173,9 @@ and results in a public path of `index.html`, instead of After you have the route mapping set up, it takes effect in the following locations: -- In the merge request widget. The: - - **View app** button takes you to the environment URL set in `.gitlab-ci.yml`. - - Dropdown lists the first 5 matched items from the route map, but you can filter them if more +- In the merge request widget: + - The **View app** button takes you to the environment URL set in the `.gitlab-ci.yml` file. + - The list shows the first 5 matched items from the route map, but you can filter them if more than 5 are available.  @@ -209,14 +207,14 @@ Review App, the Visual Reviews feedback form is overlaid on the right side of ev To use the feedback form to make a comment in the merge request: -1. Click the **Review** tab on the right side of a page. +1. On the right side of a page, select the **Review** tab. 1. Make a comment on the visual review. You can make use of all the [Markdown annotations](../../user/markdown.md) that are also available in merge request comments. 1. Enter your personal information: - If [`data-require-auth`](#authentication-for-visual-reviews) is `true`, you must enter your [personal access token](../../user/profile/personal_access_tokens.md). - Otherwise, enter your name, and optionally your email. -1. Click **Send feedback**. +1. Select **Send feedback**. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> To see Visual reviews in action, see the [Visual Reviews Walk through](https://youtu.be/1_tvWTlPfM4). @@ -224,14 +222,7 @@ To see Visual reviews in action, see the [Visual Reviews Walk through](https://y ### Configure Review Apps for Visual Reviews The feedback form is served through a script you add to pages in your Review App. -If you have the [Developer role](../../user/permissions.md) in the project, -you can access it by clicking the **Review** button in the **Pipeline** section -of the merge request. The form modal also shows a dropdown for changed pages -if [route maps](#route-maps) are configured in the project. - - - -The provided script should be added to the `<head>` of your application and +It should be added to the `<head>` of your application and consists of some project and merge request specific values. Here's how it looks for a project with code hosted in a project on GitLab.com: diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 5ab8653dc35..0e6c2f63f9e 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,5 +1,6 @@ --- redirect_to: 'index.md' +remove_date: '2021-09-28' --- This document was moved to [another location](index.md). diff --git a/doc/ci/runners/build_cloud/linux_build_cloud.md b/doc/ci/runners/build_cloud/linux_build_cloud.md index 710054921ef..1125d8dbcb4 100644 --- a/doc/ci/runners/build_cloud/linux_build_cloud.md +++ b/doc/ci/runners/build_cloud/linux_build_cloud.md @@ -4,11 +4,11 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Linux shared runners +# Build Cloud runners for Linux -Linux shared runners on GitLab.com run in autoscale mode and are powered by Google Cloud Platform. +GitLab Build Cloud runners for Linux run in autoscale mode and are powered by Google Cloud Platform. -Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available for users and customers on GitLab.com. +Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available on GitLab.com. GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our [Open Source](https://about.gitlab.com/solutions/open-source/join/), [Education](https://about.gitlab.com/solutions/education/), and [Startups](https://about.gitlab.com/solutions/startups/) programs. For private projects, GitLab offers various [plans](https://about.gitlab.com/pricing/), starting with a Free tier. @@ -23,7 +23,7 @@ Jobs handled by the shared runners on GitLab.com (`shared-runners-manager-X.gitl **time out after 3 hours**, regardless of the timeout configured in a project. Check the issues [4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. -Below are the shared runners settings. +Below are the runners' settings. | Setting | GitLab.com | Default | | ----------- | ----------------- | ---------- | @@ -33,7 +33,7 @@ Below are the shared runners settings. ## Pre-clone script -Linux shared runners on GitLab.com provide a way to run commands in a CI +Build Cloud runners for Linux provide a way to run commands in a CI job before the runner attempts to run `git init` and `git fetch` to download a GitLab repository. The [`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) diff --git a/doc/ci/runners/build_cloud/macos/environment.md b/doc/ci/runners/build_cloud/macos/environment.md new file mode 100644 index 00000000000..e84b7d0a207 --- /dev/null +++ b/doc/ci/runners/build_cloud/macos/environment.md @@ -0,0 +1,43 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# VM instances and images for Build Cloud for macOS + +When you use the Build Cloud for macOS: + +- Each of your jobs runs in a newly provisioned VM, which is dedicated to the specific job. +- The VM is active only for the duration of the job and immediately deleted. + +## VM types + +The virtual machine where your job runs has `sudo` access with no password. +For the Beta, there is only one available machine type, `gbc-macos-large`. + +| Instance type | vCPUS | Memory (GB) | +| --------- | --- | ------- | +| `gbc-macos-large` | 4 | 10 | + +## VM images + +You can execute your build on one of the following images. +You specify this image in your `.gitlab-ci.yml` file. + +Each image is running a specific version of macOS and Xcode. + +| VM image | Included software | +|---------------------------|--------------------| +| macos-10.13-xcode-7 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | +| macos-10.13-xcode-8 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | +| macos-10.13-xcode-9 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | +| macos-10.14-xcode-10 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/mojave.yml> | +| macos-10.15-xcode-11 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/catalina.yml> | +| macos-11-xcode-12 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/big-sur.yml> | + +### Image update policy + +- Support for new macOS versions is planned. +- Additional details on the support policy and image update release process are documented + [in this project](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/55bf59c8fa88712960afff2bf6ecc5daa879a8f5/docs/overview.md#os-images). diff --git a/doc/ci/runners/build_cloud/macos_build_cloud.md b/doc/ci/runners/build_cloud/macos_build_cloud.md new file mode 100644 index 00000000000..1400c7e08db --- /dev/null +++ b/doc/ci/runners/build_cloud/macos_build_cloud.md @@ -0,0 +1,62 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Build Cloud runners for macOS (Beta) + +The GitLab Build Cloud for macOS Beta provides on-demand runners integrated with GitLab SaaS [CI/CD](../../../ci/index.md). +Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS, iOS, tvOS). You can take advantage +of all the capabilities of the GitLab single DevOps platform and not have to manage or operate a +build environment. + +Build Cloud runners for macOS are in [Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +and shouldn't be relied upon for mission-critical production jobs. + +## Quickstart + +To start using Build Cloud for macOS Beta, you must submit an access request issue. After your +access has been granted and your build environment configured, you must configure your +`.gitlab-ci.yml` pipeline file: + +1. Add a `.gitlab-ci.yml` file to your project repository. +1. Specify the [image](macos/environment.md#vm-images) you want to use. +1. Commit a change to your repository. + +The runners automatically run your build. + +## Example `.gitlab-ci.yml` file + +The following sample `.gitlab-ci.yml` file shows how to start using the runners for macOS: + +```yaml +.macos_buildcloud_runners: + tags: + - shared-macos-amd64 + image: macos-11-xcode-12 + +stages: + - build + - test + +before_script: + - echo "started by ${GITLAB_USER_NAME}" + +build: + extends: + - .macos_buildcloud_runners + stage: build + script: + - echo "running scripts in the build job" + +test: + extends: + - .macos_buildcloud_runners + stage: test + script: + - echo "running scripts in the test job" +``` + +NOTE: +During the Beta period, the architecture of this solution will change. Rather than the jobs running on a specific VM instance, they will run on an ephemeral VM instance that is created by an autoscaling instance, known as the Runner Manager. We will notify all Beta participants of any downtime required to do this work. diff --git a/doc/ci/runners/build_cloud/windows_build_cloud.md b/doc/ci/runners/build_cloud/windows_build_cloud.md index 5a85f28e4b9..0004041a3e8 100644 --- a/doc/ci/runners/build_cloud/windows_build_cloud.md +++ b/doc/ci/runners/build_cloud/windows_build_cloud.md @@ -4,24 +4,24 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Windows shared runners (beta) +# Build Cloud runners for Windows (beta) -The Windows shared runners are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +GitLab Build Cloud runners for Windows are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) and shouldn't be used for production workloads. During this beta period, the [shared runner pipeline quota](../../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota) applies for groups and projects in the same manner as Linux runners. This may change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). -Windows shared runners on GitLab.com autoscale by launching virtual machines on +Windows runners on GitLab.com autoscale by launching virtual machines on the Google Cloud Platform. This solution uses an [autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/tree/master/docs/readme.md) developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). -Windows shared runners execute your CI/CD jobs on `n1-standard-2` instances with +Windows runners execute your CI/CD jobs on `n1-standard-2` instances with 2 vCPUs and 7.5 GB RAM. You can find a full list of available Windows packages in the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md). -We want to keep iterating to get Windows shared runners in a stable state and +We want to keep iterating to get Windows runners in a stable state and [generally available](https://about.gitlab.com/handbook/product/gitlab-the-product/#generally-available-ga). You can follow our work towards this goal in the [related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). @@ -89,10 +89,9 @@ VMTag = "windows" Directory = "C:\\GitLab-Runner\\autoscaler\\machines" ``` -## Example +## Example `.gitlab-ci.yml` file -Below is a simple `.gitlab-ci.yml` file to show how to start using the -Windows shared runners: +Below is a sample `.gitlab-ci.yml` file that shows how to start using the runners for Windows: ```yaml .shared_windows_runners: @@ -131,14 +130,14 @@ test: definition](https://about.gitlab.com/handbook/product/#beta). - The average provisioning time for a new Windows VM is 5 minutes. This means that you may notice slower build start times - on the Windows shared runner fleet during the beta. In a future + on the Windows runner fleet during the beta. In a future release we intend to update the autoscaler to enable the pre-provisioning of virtual machines. This is intended to significantly reduce the time it takes to provision a VM on the Windows fleet. You can follow along in the [related issue](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/issues/32). -- The Windows shared runner fleet may be unavailable occasionally +- The Windows runner fleet may be unavailable occasionally for maintenance or updates. -- The Windows shared runner virtual machine instances do not use the +- The Windows runner virtual machine instances do not use the GitLab Docker executor. This means that you can't specify [`image`](../../../ci/yaml/index.md#image) or [`services`](../../../ci/yaml/index.md#services) in your pipeline configuration. @@ -150,7 +149,7 @@ test: installation of additional software packages needs to be repeated for each job in your pipeline. - The job may stay in a pending state for longer than the - Linux shared runners. + Linux runners. - There is the possibility that we introduce breaking changes which will - require updates to pipelines that are using the Windows shared runner + require updates to pipelines that are using the Windows runner fleet. diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index da7289a0ebd..6642913a9d8 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -5,14 +5,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab SaaS runners +# GitLab Build Cloud runners If you are using self-managed GitLab or you want to use your own runners on GitLab.com, you can [install and configure your own runners](https://docs.gitlab.com/runner/install/). -If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on shared runners. No configuration is required. -Your jobs can run on [Linux](build_cloud/linux_build_cloud.md) or [Windows](build_cloud/windows_build_cloud.md). +If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on runners in the GitLab Build Cloud. +No configuration is required. Your jobs can run on: -The number of minutes you can use on these shared runners depends on your +- [Linux runners](build_cloud/linux_build_cloud.md). +- [Windows runners](build_cloud/windows_build_cloud.md) (beta). +- [macOS runners](build_cloud/macos_build_cloud.md) (beta). + +The number of minutes you can use on these runners depends on your [quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota), which depends on your [subscription plan](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes). diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index fa56be3a151..2af373384d2 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -33,13 +33,13 @@ If you are using a self-managed instance of GitLab: If you are using GitLab.com: -- You can select from a list of [shared runners that GitLab maintains](../../user/gitlab_com/index.md#shared-runners). +- You can select from a list of [shared runners that GitLab maintains](index.md). - The shared runners consume the [pipelines minutes](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes) included with your account. ### Enable shared runners -On GitLab.com, [shared runners](#shared-runners) are enabled in all projects by +On GitLab.com, [shared runners](index.md) are enabled in all projects by default. On self-managed instances of GitLab, an administrator must [install](https://docs.gitlab.com/runner/install/index.html) diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index 7fbc3448d4e..d3c34a6922e 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Using external secrets in CI +# Using external secrets in CI **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218746) in GitLab 13.4 and GitLab Runner 13.4. > - `file` setting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250695) in GitLab 14.1 and GitLab Runner 14.1. @@ -91,8 +91,7 @@ To configure your Vault server: - `VAULT_AUTH_PATH` - (Optional) The path where the authentication method is mounted, default is `jwt`. NOTE: - Support for [providing these values in the user interface](https://gitlab.com/gitlab-org/gitlab/-/issues/218677) - is planned but not yet implemented. + Support for providing these values in the user interface [is tracked in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/218677). ## Use Vault secrets in a CI job **(PREMIUM)** @@ -181,3 +180,32 @@ You can also specify some attributes for the resulting Vault tokens, such as tim IP address range, and number of uses. The full list of options is available in [Vault's documentation on creating roles](https://www.vaultproject.io/api/auth/jwt#create-role) for the JSON web token method. + +## Using a self-signed Vault server + +When the Vault server is using a self-signed certificate, you will see the following error in the job logs: + +```plaintext +ERROR: Job failed (system failure): resolving secrets: initializing Vault service: preparing authenticated client: checking Vault server health: Get https://vault.example.com:8000/v1/sys/health?drsecondarycode=299&performancestandbycode=299&sealedcode=299&standbycode=299&uninitcode=299: x509: certificate signed by unknown authority +``` + +You have two options to solve this error: + +- Add the self-signed certificate to the GitLab Runner server's CA store. + If you deployed GitLab Runner using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html), you will have to create your own GitLab Runner image. +- Use the `VAULT_CACERT` environment variable to configure GitLab Runner to trust the certificate: + - If you are using systemd to manage GitLab Runner, see [how to add an environment variable for GitLab Runner](https://docs.gitlab.com/runner/configuration/init.html#setting-custom-environment-variables). + - If you deployed GitLab Runner using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html): + 1. [Provide a custom certificate for accessing GitLab](https://docs.gitlab.com/runner/install/kubernetes.html#providing-a-custom-certificate-for-accessing-gitlab), and make sure to add the certificate for the Vault server instead of the certificate for GitLab. If your GitLab instance is also using a self-signed certificate, you should be able to add both in the same `Secret`. + 1. Add the following lines in your `values.yaml` file: + + ```yaml + ## Replace both the <SECRET_NAME> and the <VAULT_CERTIFICATE> + ## with the actual values you used to create the secret + + certsSecretName: <SECRET_NAME> + + envVars: + - name: VAULT_CACERT + value: "/home/gitlab-runner/.gitlab-runner/certs/<VAULT_CERTIFICATE>" + ``` diff --git a/doc/ci/ssh_keys/index.md b/doc/ci/ssh_keys/index.md index 6136f3be3f6..1e761643a10 100644 --- a/doc/ci/ssh_keys/index.md +++ b/doc/ci/ssh_keys/index.md @@ -13,7 +13,7 @@ environment (where the GitLab Runner runs). The SSH keys can be useful when: 1. You want to checkout internal submodules -1. You want to download private packages using your package manager (e.g., Bundler) +1. You want to download private packages using your package manager (for example, Bundler) 1. You want to deploy your application to your own server, or, for example, Heroku 1. You want to execute SSH commands from the build environment to a remote server 1. You want to rsync files from the build environment to a remote server @@ -21,9 +21,9 @@ The SSH keys can be useful when: If anything of the above rings a bell, then you most likely need an SSH key. The most widely supported method is to inject an SSH key into your build -environment by extending your `.gitlab-ci.yml`, and it's a solution which works +environment by extending your `.gitlab-ci.yml`, and it's a solution that works with any type of [executor](https://docs.gitlab.com/runner/executors/) -(Docker, shell, etc.). +(like Docker or shell, for example). ## How it works diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 5ab8653dc35..0e6c2f63f9e 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -1,5 +1,6 @@ --- redirect_to: 'index.md' +remove_date: '2021-09-28' --- This document was moved to [another location](index.md). diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index 383887bd389..880473d402d 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -205,7 +205,7 @@ file. The parameter is of the form: variables[key]=value ``` -This information is also exposed in the UI. Please note that _values_ are only viewable by Owners and Maintainers. +This information is also exposed in the UI. _Values_ are only viewable by users with the Owner and Maintainer role.  diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index f845c79ef45..7677908e93a 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -114,16 +114,15 @@ ruby: ### Go example -Use the following job in `.gitlab-ci.yml`, and ensure you use `-set-exit-code`, -otherwise the pipeline is marked successful, even if the tests fail: +Use the following job in `.gitlab-ci.yml`: ```yaml -## Use https://github.com/jstemmer/go-junit-report to generate a JUnit report format XML file with go +## Use https://github.com/gotestyourself/gotestsum to generate a JUnit report format XML file with go golang: stage: test script: - - go get -u github.com/jstemmer/go-junit-report - - go test -v 2>&1 | go-junit-report -set-exit-code > report.xml + - go get gotest.tools/gotestsum + - gotestsum --junitfile report.xml --format testname artifacts: when: always reports: diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 5ab8653dc35..0e6c2f63f9e 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -1,5 +1,6 @@ --- redirect_to: 'index.md' +remove_date: '2021-09-28' --- This document was moved to [another location](index.md). diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index e6768968d83..a00b8b678ec 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -112,20 +112,33 @@ job1: - echo This job does not need any variables ``` -You can use variables to help define other variables. Use `$$` to ignore a variable -name inside another variable: +Use the [`value` and `description`](../yaml/index.md#prefill-variables-in-manual-pipelines) +keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) +for [manually-triggered pipelines](../pipelines/index.md#run-a-pipeline-manually). + +### Use variables or `$` in other variables + +You can use variables inside other variables: ```yaml -variables: - FLAGS: '-al' - LS_CMD: 'ls "$FLAGS" $$TMP_DIR' -script: - - 'eval "$LS_CMD"' # Executes 'ls -al $TMP_DIR' +job: + variables: + FLAGS: '-al' + LS_CMD: 'ls "$FLAGS"' + script: + - 'eval "$LS_CMD"' # Executes 'ls -al' ``` -Use the [`value` and `description`](../yaml/index.md#prefill-variables-in-manual-pipelines) -keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) -for [manually-triggered pipelines](../pipelines/index.md#run-a-pipeline-manually). +If you do not want the `$` interpreted as the start of a variable, use `$$` instead: + +```yaml +job: + variables: + FLAGS: '-al' + LS_CMD: 'ls "$FLAGS" $$TMP_DIR' + script: + - 'eval "$LS_CMD"' # Executes 'ls -al $TMP_DIR' +``` ### Add a CI/CD variable to a project @@ -294,7 +307,7 @@ To mask a variable: 1. In the project, group, or Admin Area, go to **Settings > CI/CD**. 1. Expand the **Variables** section. 1. Next to the variable you want to protect, select **Edit**. -1. Select the **Mask variable** check box. +1. Select the **Mask variable** checkbox. 1. Select **Update variable**. The value of the variable must: @@ -311,18 +324,27 @@ NOTE: Masking a CI/CD variable is not a guaranteed way to prevent malicious users from accessing variable values. To make variables more secure, you can [use external secrets](../secrets/index.md). +WARNING: +Due to a technical limitation, masked variables that are more than 4 KiB in length are not recommended. Printing such +a large value to the trace log has the potential to be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). +When using GitLab Runner 14.2, only the tail of the variable, characters beyond 4KiB in length, have the potential to +be revealed. + ### Protect a CI/CD variable You can protect a project, group or instance CI/CD variable so it is only passed to pipelines running on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). +[Pipelines for merge requests](../pipelines/merge_request_pipelines.md) do not have access to protected variables. +An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/28002) regarding this limitation. + To protect a variable: 1. Go to **Settings > CI/CD** in the project, group or instance admin area. 1. Expand the **Variables** section. 1. Next to the variable you want to protect, select **Edit**. -1. Select the **Protect variable** check box. +1. Select the **Protect variable** checkbox. 1. Select **Update variable**. The variable is available for all subsequent pipelines. @@ -398,7 +420,7 @@ job_name: ``` In [some cases](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4115#note_157692820) -environment variables might need to be surrounded by quotes to expand properly: +environment variables must be surrounded by quotes to expand properly: ```yaml job_name: @@ -626,7 +648,7 @@ the environment scope of a variable. GitLab does this by [defining which environments and corresponding jobs](../environments/index.md) the variable can be available for. -To learn more about scoping environments, see [Scoping environments with specs](../environments/index.md#scoping-environments-with-specs). +To learn more about scoping environments, see [Scoping environments with specs](../environments/index.md#scope-environments-with-specs). To learn more about ensuring CI/CD variables are only exposed in pipelines running from protected branches or tags, see [Protect a CI/CD Variable](#protect-a-cicd-variable). @@ -770,6 +792,8 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_PROJECT_VISIBILITY=public ++ export CI_PROJECT_REPOSITORY_LANGUAGES= ++ CI_PROJECT_REPOSITORY_LANGUAGES= +++ export CI_PROJECT_CLASSIFICATION_LABEL= +++ CI_PROJECT_CLASSIFICATION_LABEL= ++ export CI_DEFAULT_BRANCH=main ++ CI_DEFAULT_BRANCH=main ++ export CI_REGISTRY=registry.gitlab.com diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index b6dd9446644..b5999a555c9 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -38,7 +38,7 @@ There are also [Kubernetes-specific deployment variables](../../user/project/clu | `CI_COMMIT_TITLE` | 10.8 | all | The title of the commit. The full first line of the message. | | `CI_CONCURRENT_ID` | all | 11.10 | The unique ID of build execution in a single executor. | | `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | The unique ID of build execution in a single executor and project. | -| `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to the CI/CD configuration file. Defaults to `.gitlab-ci.yml`. | +| `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to the CI/CD configuration file. Defaults to `.gitlab-ci.yml`. Read-only inside a running pipeline. | | `CI_DEBUG_TRACE` | all | 1.7 | `true` if [debug logging (tracing)](index.md#debug-logging) is enabled. | | `CI_DEFAULT_BRANCH` | 12.4 | all | The name of the project's default branch. | | `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | 13.7 | all | The image prefix for pulling images through the Dependency Proxy. | @@ -89,8 +89,9 @@ There are also [Kubernetes-specific deployment variables](../../user/project/clu | `CI_PROJECT_TITLE` | 12.4 | all | The human-readable project name as displayed in the GitLab web interface. | | `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address of the project. | | `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility. Can be `internal`, `private`, or `public`. | +| `CI_PROJECT_CLASSIFICATION_LABEL` | 14.2 | all | The project [external authorization classification label](../../user/admin_area/settings/external_authorization.md). | | `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | The address of the project's Container Registry. Only available if the Container Registry is enabled for the project. | -| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. | +| `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. This password value is the same as the `CI_JOB_TOKEN` and is valid only as long as the job is running. Use the `CI_DEPLOY_PASSWORD` for long-lived access to the registry | | `CI_REGISTRY_USER` | 9.0 | all | The username to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. | | `CI_REGISTRY` | 8.10 | 0.5 | The address of the GitLab Container Registry. Only available if the Container Registry is enabled for the project. This variable includes a `:port` value if one is specified in the registry configuration. | | `CI_REPOSITORY_URL` | 9.0 | all | The URL to clone the Git repository. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index beb19c8beea..8009687dbca 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -113,7 +113,7 @@ Feature.disable(:variable_inside_variable, Project.find(<project id>)) - Supported: project/group variables, `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers, pipeline schedules, and manual pipelines. -- Not supported: variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`). +- Not supported: variables defined inside of scripts (for example, `export MY_VARIABLE="test"`). The runner uses Go's `os.Expand()` method for variable expansion. It means that it handles only variables defined as `$variable` and `${variable}`. What's also important, is that @@ -132,7 +132,7 @@ use a different variables syntax. Supported: -- The `script` may use all available variables that are default for the shell (e.g., `$PATH` which +- The `script` may use all available variables that are default for the shell (for example, `$PATH` which should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (project/group variables, `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules). - The `script` may also use all variables defined in the lines before. So, for example, if you define @@ -174,7 +174,7 @@ They are: - Script execution shell. - Not supported: - For definitions where the ["Expansion place"](#gitlab-ciyml-file) is GitLab. - - In the `only` and `except` [variables expressions](../jobs/job_control.md#cicd-variable-expressions). + - In the `only`, `except`, and `rules` [variables expressions](../jobs/job_control.md#cicd-variable-expressions). Some of the persisted variables contain tokens and cannot be used by some definitions due to security reasons. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 5ab8653dc35..0e6c2f63f9e 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1,5 +1,6 @@ --- redirect_to: 'index.md' +remove_date: '2021-09-28' --- This document was moved to [another location](index.md). diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md index 2723cb19c1f..6cd900123e0 100644 --- a/doc/ci/yaml/gitlab_ci_yaml.md +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -4,11 +4,8 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference --- -<!-- markdownlint-disable MD044 --> -<!-- vale gitlab.Spelling = NO --> -# The .gitlab-ci.yml file **(FREE)** -<!-- vale gitlab.Spelling = YES --> -<!-- markdownlint-enable MD044 --> + +# The `.gitlab-ci.yml` file **(FREE)** To use GitLab CI/CD, you need: diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index c2b8ef50e6a..63f626e524e 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -126,6 +126,15 @@ Use `stages` to define stages that contain groups of jobs. `stages` is defined g for the pipeline. Use [`stage`](#stage) in a job to define which stage the job is part of. +If `stages` is not defined in the `.gitlab-ci.yml` file, then the default +pipeline stages are: + +- [`.pre`](#stage-pre) +- `build` +- `test` +- `deploy` +- [`.post`](#stage-post) + The order of the `stages` items defines the execution order for jobs: - Jobs in the same stage run in parallel. @@ -148,9 +157,6 @@ stages: If any job fails, the pipeline is marked as `failed` and jobs in later stages do not start. Jobs in the current stage are not stopped and continue to run. -If no `stages` are defined in the `.gitlab-ci.yml` file, then `build`, `test` and `deploy` -are the default pipeline stages. - If a job does not specify a [`stage`](#stage), the job is assigned the `test` stage. If a stage is defined, but no jobs use it, the stage is not visible in the pipeline. This is @@ -415,13 +421,20 @@ NOTE: Use merging to customize and override included CI/CD configurations with local configurations. Local configurations in the `.gitlab-ci.yml` file override included configurations. -#### Variables with `include` **(FREE SELF)** +#### Variables with `include` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/284883) in GitLab 13.8. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294294) in GitLab 13.9. +> - [Support for project, group, and instance variables added](https://gitlab.com/gitlab-org/gitlab/-/issues/219065) in GitLab 14.2. + +In `include` sections in your `.gitlab-ci.yml` file, you can use: -You can [use some predefined variables in `include` sections](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) -in your `.gitlab-ci.yml` file: +- `$CI_COMMIT_REF_NAME` [predefined variable](../variables/predefined_variables.md) in GitLab 14.2 + and later. +- [Project variables](../variables/index.md#add-a-cicd-variable-to-a-project) +- [Group variables](../variables/index.md#add-a-cicd-variable-to-a-group) +- [Instance variables](../variables/index.md#add-a-cicd-variable-to-an-instance) +- Project [predefined variables](../variables/predefined_variables.md). ```yaml include: @@ -432,6 +445,32 @@ include: For an example of how you can include these predefined variables, and the variables' impact on CI/CD jobs, see this [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos). +There is a [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337633) +that proposes expanding this feature to support more variables. + +#### `rules` with `include` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276515) in GitLab 14.2. + +NOTE: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the `ci_include_rules` flag](../../administration/feature_flags.md). +On GitLab.com, this feature is not available. The feature is not ready for production use. + +You can use [`rules`](#rules) with `include` to conditionally include other configuration files. +You can only use `rules:if` in `include` with [certain variables](#variables-with-include). + +```yaml +include: + - local: builds.yml + rules: + - if: '$INCLUDE_BUILDS == "true"' + +test: + stage: test + script: exit 0 +``` + #### `include:local` Use `include:local` to include a file that is in the same repository as the `.gitlab-ci.yml` file. @@ -463,15 +502,7 @@ Use local includes instead of symbolic links. ##### `include:local` with wildcard file paths > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25921) in GitLab 13.11. -> - [Deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/327315) in GitLab 13.12. -> - Enabled on GitLab.com. -> - Recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to disable it. **(CORE ONLY)** - -There can be -[risks when disabling released features](../../user/feature_flags.md#risks-when-disabling-released-features). -Refer to this feature's version history for more details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/327315) in GitLab 14.2. You can use wildcard paths (`*` and `**`) with `include:local`. @@ -495,23 +526,6 @@ When the pipeline runs, GitLab: include: 'configs/**/*.yml' ``` -The wildcard file paths feature is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:ci_wildcard_file_paths) -``` - -To disable it: - -```ruby -Feature.disable(:ci_wildcard_file_paths) -``` - #### `include:file` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53903) in GitLab 11.7. @@ -680,160 +694,148 @@ For more information, see [Available settings for `services`](../services/index. ### `script` -Use `script` to specify a shell script for the runner to execute. +Use `script` to specify commands for the runner to execute. All jobs except [trigger jobs](#trigger) require a `script` keyword. -For example: +**Keyword type**: Job keyword. You can use it only as part of a job. -```yaml -job: - script: "bundle exec rspec" -``` +**Possible inputs**: An array including: -You can use [YAML anchors with `script`](#yaml-anchors-for-scripts). +- Single line commands. +- Long commands [split over multiple lines](script.md#split-long-commands). +- [YAML anchors](#yaml-anchors-for-scripts). -The `script` keyword can also contain several commands in an array: +**Example of `script`:** ```yaml -job: +job1: + script: "bundle exec rspec" + +job2: script: - uname -a - bundle exec rspec ``` -Sometimes, `script` commands must be wrapped in single or double quotes. -For example, commands that contain a colon (`:`) must be wrapped in single quotes (`'`). -The YAML parser needs to interpret the text as a string rather than -a "key: value" pair. +**Additional details**: -For example, this script uses a colon: +You might need to use single quotes (`'`) or double quotes (`"`) when using +[special characters in `script`](script.md#use-special-characters-with-script). -```yaml -job: - script: - - curl --request POST --header 'Content-Type: application/json' "https://gitlab/api/v4/projects" -``` +**Related topics**: + +- You can [ignore non-zero exit codes](script.md#ignore-non-zero-exit-codes). +- [Use color codes with `script`](script.md#add-color-codes-to-script-output) + to make job logs easier to review. +- [Create custom collapsible sections](../jobs/index.md#custom-collapsible-sections) + to simplify job log output. -To be considered valid YAML, you must wrap the entire command in single quotes. If -the command already uses single quotes, you should change them to double quotes (`"`) -if possible: +#### `before_script` -```yaml -job: - script: - - 'curl --request POST --header "Content-Type: application/json" "https://gitlab/api/v4/projects"' -``` +Use `before_script` to define an array of commands that should run before each job's +`script` commands, but after [artifacts](#artifacts) are restored. -You can verify the syntax is valid with the [CI Lint](../lint.md) tool. +**Keyword type**: Job keyword. You can use it only as part of a job or in the +[`default:` section](#custom-default-keyword-values). -Be careful when using these characters as well: +**Possible inputs**: An array including: -- `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. +- Single line commands. +- Long commands [split over multiple lines](script.md#split-long-commands). +- [YAML anchors](#yaml-anchors-for-scripts). -If any of the script commands return an exit code other than zero, the job -fails and further commands are not executed. Store the exit code in a variable to -avoid this behavior: +**Example of `before_script`:** ```yaml job: + before_script: + - echo "Execute this command before any `script:` commands." script: - - false || exit_code=$? - - if [ $exit_code -ne 0 ]; then echo "Previous command failed"; fi; + - echo "This command executes after the job's `before_script` commands." ``` -#### `before_script` - -Use `before_script` to define an array of commands that should run before each job, -but after [artifacts](#artifacts) are restored. +**Additional details**: Scripts you specify in `before_script` are concatenated with any scripts you specify -in the main [`script`](#script). The combine scripts execute together in a single shell. +in the main [`script`](#script). The combined scripts execute together in a single shell. -You can overwrite a globally-defined `before_script` if you define it in a job: +**Related topics**: -```yaml -default: - before_script: - - echo "Execute this script in all jobs that don't already have a before_script section." +- [Use `before_script` with `default`](script.md#set-a-default-before_script-or-after_script-for-all-jobs) + to define a default array of commands that should run before the `script` commands in all jobs. +- You can [ignore non-zero exit codes](script.md#ignore-non-zero-exit-codes). +- [Use color codes with `before_script`](script.md#add-color-codes-to-script-output) + to make job logs easier to review. +- [Create custom collapsible sections](../jobs/index.md#custom-collapsible-sections) + to simplify job log output. -job1: - script: - - echo "This script executes after the global before_script." +#### `after_script` -job: - before_script: - - echo "Execute this script instead of the global before_script." - script: - - echo "This script executes after the job's `before_script`" -``` +Use `after_script` to define an array of commands that run after each job, including failed jobs. -You can use [YAML anchors with `before_script`](#yaml-anchors-for-scripts). +**Keyword type**: Job keyword. You can use it only as part of a job or in the +[`default:` section](#custom-default-keyword-values). -#### `after_script` +**Possible inputs**: An array including: -Use `after_script` to define an array of commands that run after each job, -including failed jobs. +- Single line commands. +- Long commands [split over multiple lines](script.md#split-long-commands). +- [YAML anchors](#yaml-anchors-for-scripts). -If a job times out or is cancelled, the `after_script` commands do not execute. -An [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/15603) exists to support -executing `after_script` commands for timed-out or cancelled jobs. +**Example of `after_script`:** + +```yaml +job: + script: + - echo "An example script section." + after_script: + - echo "Execute this command after the `script` section completes." +``` + +**Additional details**: Scripts you specify in `after_script` execute in a new shell, separate from any -`before_script` or `script` scripts. As a result, they: +`before_script` or `script` commands. As a result, they: - Have a current working directory set back to the default. -- Have no access to changes done by scripts defined in `before_script` or `script`, including: +- Don't have access to changes done by commands defined in the `before_script` or `script`, + including: - Command aliases and variables exported in `script` scripts. - Changes outside of the working tree (depending on the runner executor), like software installed by a `before_script` or `script` script. -- Have a separate timeout, which is hard coded to 5 minutes. See the - [related issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2716) for details. +- Have a separate timeout, which is [hard-coded to 5 minutes](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2716). - Don't affect the job's exit code. If the `script` section succeeds and the `after_script` times out or fails, the job exits with code `0` (`Job Succeeded`). -```yaml -default: - after_script: - - echo "Execute this script in all jobs that don't already have an after_script section." - -job1: - script: - - echo "This script executes first. When it completes, the global after_script executes." - -job: - script: - - echo "This script executes first. When it completes, the job's `after_script` executes." - after_script: - - echo "Execute this script instead of the global after_script." -``` - -You can use [YAML anchors with `after_script`](#yaml-anchors-for-scripts). - -#### Script syntax +If a job times out or is cancelled, the `after_script` commands do not execute. +[An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/15603) to add support for executing `after_script` commands for timed-out or cancelled jobs. -You can use syntax in [`script`](#script) sections to: +**Related topics**: -- [Split long commands](script.md#split-long-commands) into multiline commands. -- [Use color codes](script.md#add-color-codes-to-script-output) to make job logs easier to review. +- [Use `after_script` with `default`](script.md#set-a-default-before_script-or-after_script-for-all-jobs) + to define a default array of commands that should run after all jobs. +- You can [ignore non-zero exit codes](script.md#ignore-non-zero-exit-codes). +- [Use color codes with `after_script`](script.md#add-color-codes-to-script-output) + to make job logs easier to review. - [Create custom collapsible sections](../jobs/index.md#custom-collapsible-sections) to simplify job log output. ### `stage` -Use `stage` to define which stage a job runs in. Jobs in the same -`stage` can execute in parallel (subject to [certain conditions](#use-your-own-runners)). +Use `stage` to define which [stage](#stages) a job runs in. Jobs in the same +`stage` can execute in parallel (see **Additional details**). -Jobs without a `stage` entry use the `test` stage by default. If you do not define -[`stages`](#stages) in the pipeline, you can use the 5 default stages, which execute in -this order: +If `stage` is not defined, the job uses the `test` stage by default. -- [`.pre`](#pre-and-post) -- `build` -- `test` -- `deploy` -- [`.post`](#pre-and-post) -For example: +**Keyword type**: Job keyword. You can use it only as part of a job. + +**Possible inputs**: An array including any number of stage names. Stage names can be: + +- The [default stages](#stages). +- User-defined stages. + +**Example of `stage`**: ```yaml stages: @@ -841,76 +843,101 @@ stages: - test - deploy -job 0: - stage: .pre - script: make something useful before build stage - -job 1: - stage: build - script: make build dependencies - -job 2: +job1: stage: build - script: make build artifacts + script: + - echo "This job compiles code." -job 3: +job2: stage: test - script: make test + script: + - echo "This job tests the compiled code. It runs when the build stage completes." -job 4: - stage: deploy - script: make deploy +job3: + script: + - echo "This job also runs in the test stage". -job 5: - stage: .post - script: make something useful at the end of pipeline +job4: + stage: deploy + script: + - echo "This job deploys the code. It runs when the test stage completes." ``` -#### Use your own runners - -When you use your own runners, each runner runs only one job at a time by default. -Jobs can run in parallel if they run on different runners. +**Additional details**: -If you have only one runner, jobs can run in parallel if the runner's -[`concurrent` setting](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) -is greater than `1`. +- Jobs can run in parallel if they run on different runners. +- If you have only one runner, jobs can run in parallel if the runner's + [`concurrent` setting](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) + is greater than `1`. -#### `.pre` and `.post` +#### `stage: .pre` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31441) in GitLab 12.4. -Use `pre` and `post` for jobs that need to run first or last in a pipeline. - -- `.pre` is guaranteed to always be the first stage in a pipeline. -- `.post` is guaranteed to always be the last stage in a pipeline. - -User-defined stages are executed after `.pre` and before `.post`. +Use the `.pre` stage to make a job run at the start of a pipeline. `.pre` is +always the first stage in a pipeline. User-defined stages execute after `.pre`. +You do not need to define `.pre` in [`stages`](#stages). You must have a job in at least one stage other than `.pre` or `.post`. -You can't change the order of `.pre` and `.post`, even if you define them out of order in the `.gitlab-ci.yml` file. -For example, the following configurations are equivalent: +**Keyword type**: You can only use it with a job's `stage` keyword. -```yaml -stages: - - .pre - - a - - b - - .post -``` +**Example of `stage: .pre`**: ```yaml stages: - - a - - .pre - - b - - .post + - build + - test + +job1: + stage: build + script: + - echo "This job runs in the build stage." + +first-job: + stage: .pre + script: + - echo "This job runs in the .pre stage, before all other stages." + +job2: + stage: test + script: + - echo "This job runs in the test stage." ``` +#### `stage: .post` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31441) in GitLab 12.4. + +Use the `.post` stage to make a job run at the end of a pipeline. `.post` +is always the last stage in a pipeline. User-defined stages execute before `.post`. +You do not need to define `.post` in [`stages`](#stages). + +You must have a job in at least one stage other than `.pre` or `.post`. + +**Keyword type**: You can only use it with a job's `stage` keyword. + +**Example of `stage: .post`**: + ```yaml stages: - - a - - b + - build + - test + +job1: + stage: build + script: + - echo "This job runs in the build stage." + +last-job: + stage: .post + script: + - echo "This job runs in the .post stage, after all other stages." + +job2: + stage: test + script: + - echo "This job runs in the test stage." ``` ### `extends` @@ -1504,6 +1531,7 @@ in the project. > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47063) in GitLab 12.2. > - In GitLab 12.3, maximum number of jobs in `needs` array raised from five to 50. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30631) in GitLab 12.8, `needs: []` lets jobs start immediately. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30632) in GitLab 14.2, you can refer to jobs in the same stage as the job you are configuring. Use `needs:` to execute jobs out-of-order. Relationships between jobs that use `needs` can be visualized as a [directed acyclic graph](../directed_acyclic_graph/index.md). @@ -1563,13 +1591,11 @@ production: #### Requirements and limitations -- In [GitLab 14.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/30632) - you can refer to jobs in the same stage as the job you are configuring. This feature - is [Deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -- Disabled on GitLab.com. -- Not recommended for production use. -- For GitLab self-managed instances, GitLab adminsitrators - can choose to [disable it](#enable-or-disable-needs-for-jobs-in-the-same-stage) +- In [GitLab 14.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/30632) you + can refer to jobs in the same stage as the job you are configuring. This feature is + enabled on GitLab.com and ready for production use. On self-managed [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/30632) + this feature is available by default. To hide the feature, ask an administrator to + [disable the `ci_same_stage_job_needs` flag](../../administration/feature_flags.md). - In GitLab 14.0 and older, you can only refer to jobs in earlier stages. - In GitLab 13.9 and older, if `needs:` refers to a job that might not be added to a pipeline because of `only`, `except`, or `rules`, the pipeline might fail to create. @@ -1583,26 +1609,10 @@ production: name, they overwrite each other and only the last one downloaded is saved. - `needs:` is similar to `dependencies:` in that it must use jobs from prior stages, meaning it's impossible to create circular dependencies. Depending on jobs in the - current stage is not possible either, but support [is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/30632). + current stage is not possible either, but [an issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/30632). - Stages must be explicitly defined for all jobs that have the keyword `needs:` or are referred to by one. -##### Enable or disable `needs` for jobs in the same stage **(FREE SELF)** - -`needs` for jobs in the same stage is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails -console](../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -`Feature.enable(:ci_same_stage_job_needs)` - -To disable it: - -`Feature.disable(:ci_same_stage_job_needs)` - ##### Changing the `needs:` job limit **(FREE SELF)** The maximum number of jobs that can be defined in `needs:` defaults to 50. @@ -1860,11 +1870,26 @@ osx job: - echo "Hello, $USER!" ``` +In [GitLab 14.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/35742), you can +use [CI/CD variables](../variables/index.md) with `tags` for dynamic runner selection: + +```yaml +variables: + KUBERNETES_RUNNER: kubernetes + + job: + tags: + - docker + - $KUBERNETES_RUNNER + script: + - echo "Hello runner selector feature" +``` + ### `allow_failure` Use `allow_failure` when you want to let a job fail without impacting the rest of the CI -suite. The default value is `false`, except for [manual](#whenmanual) jobs that use -the `when: manual` syntax. +suite. The default value is `false`, except for [manual](../jobs/job_control.md#create-a-job-that-must-be-run-manually) jobs that use +the [`when: manual`](#when) syntax. In jobs that use [`rules:`](#rules), all jobs default to `allow_failure: false`, *including* `when: manual` jobs. @@ -1928,28 +1953,23 @@ test_job_2: ### `when` -Use `when` to implement jobs that run in case of failure or despite the -failure. +Use `when` to configure the conditions for when jobs run. If not defined in a job, +the default value is `when: on_success`. -The valid values of `when` are: +**Keyword type**: Job keyword. You can use it only as part of a job. -1. `on_success` (default) - Execute job only when all jobs in earlier stages succeed, - or are considered successful because they have `allow_failure: true`. -1. `on_failure` - Execute job only when at least one job in an earlier stage fails. -1. `always` - Execute job regardless of the status of jobs in earlier stages. -1. `manual` - Execute job [manually](#whenmanual). -1. `delayed` - [Delay the execution of a job](#whendelayed) for a specified duration. - Added in GitLab 11.14. -1. `never`: - - With job [`rules`](#rules), don't execute job. - - With [`workflow:rules`](#workflow), don't run pipeline. +**Possible inputs**: -In the following example, the script: +- `on_success` (default): Run the job only when all jobs in earlier stages succeed + or have `allow_failure: true`. +- `manual`: Run the job only when [triggered manually](../jobs/job_control.md#create-a-job-that-must-be-run-manually). +- `always`: Run the job regardless of the status of jobs in earlier stages. +- `on_failure`: Run the job only when at least one job in an earlier stage fails. +- `delayed`: [Delay the execution of a job](../jobs/job_control.md#run-a-job-after-a-delay) + for a specified duration. +- `never`: Don't run the job. -1. Executes `cleanup_build_job` only when `build_job` fails. -1. Always executes `cleanup_job` as the last step in pipeline regardless of - success or failure. -1. Executes `deploy_job` when you run it manually in the GitLab UI. +**Example of `when`**: ```yaml stages: @@ -1988,116 +2008,26 @@ cleanup_job: when: always ``` -#### `when:manual` - -A manual job is a type of job that is not executed automatically and must be explicitly -started by a user. You might want to use manual jobs for things like deploying to production. - -To make a job manual, add `when: manual` to its configuration. - -When the pipeline starts, manual jobs display as skipped and do not run automatically. -They can be started from the pipeline, job, [environment](../environments/index.md#configure-manual-deployments), -and deployment views. - -Manual jobs can be either optional or blocking: - -- **Optional**: Manual jobs have [`allow_failure: true](#allow_failure) set by default - and are considered optional. The status of an optional manual job does not contribute - to the overall pipeline status. A pipeline can succeed even if all its manual jobs fail. - -- **Blocking**: To make a blocking manual job, add `allow_failure: false` to its configuration. - Blocking manual jobs stop further execution of the pipeline at the stage where the - job is defined. To let the pipeline continue running, click **{play}** (play) on - the blocking manual job. - - Merge requests in projects with [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) - enabled can't be merged with a blocked pipeline. Blocked pipelines show a status - of **blocked**. - -When you use [`rules:`](#rules), `allow_failure` defaults to `false`, including for manual jobs. - -To trigger a manual job, a user must have permission to merge to the assigned branch. -You can use [protected branches](../../user/project/protected_branches.md) to more strictly -[protect manual deployments](#protecting-manual-jobs) from being run by unauthorized users. - -In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201938) and later, you -can use `when:manual` in the same job as [`trigger`](#trigger). In GitLab 13.4 and -earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. - -##### Protecting manual jobs **(PREMIUM)** - -Use [protected environments](../environments/protected_environments.md) -to define a list of users authorized to run a manual job. You can authorize only -the users associated with a protected environment to trigger manual jobs, which can: - -- More precisely limit who can deploy to an environment. -- Block a pipeline until an approved user "approves" it. - -To protect a manual job: - -1. Add an `environment` to the job. For example: - - ```yaml - deploy_prod: - stage: deploy - script: - - echo "Deploy to production server" - environment: - name: production - url: https://example.com - when: manual - rules: - - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH - ``` +In this example, the script: -1. In the [protected environments settings](../environments/protected_environments.md#protecting-environments), - select the environment (`production` in this example) and add the users, roles or groups - that are authorized to trigger the manual job to the **Allowed to Deploy** list. Only those in - this list can trigger this manual job, as well as GitLab administrators - who are always able to use protected environments. - -You can use protected environments with blocking manual jobs to have a list of users -allowed to approve later pipeline stages. Add `allow_failure: false` to the protected -manual job and the pipeline's next stages only run after the manual job is triggered -by authorized users. - -#### `when:delayed` - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51352) in GitLab 11.4. - -Use `when: delayed` to execute scripts after a waiting period, or if you want to avoid -jobs immediately entering the `pending` state. - -You can set the period with `start_in` keyword. The value of `start_in` is an elapsed time in seconds, unless a unit is -provided. `start_in` must be less than or equal to one week. Examples of valid values include: - -- `'5'` -- `5 seconds` -- `30 minutes` -- `1 day` -- `1 week` - -When a stage includes a delayed job, the pipeline doesn't progress until the delayed job finishes. -You can use this keyword to insert delays between different stages. - -The timer of a delayed job starts immediately after the previous stage completes. -Similar to other types of jobs, a delayed job's timer doesn't start unless the previous stage passes. +1. Executes `cleanup_build_job` only when `build_job` fails. +1. Always executes `cleanup_job` as the last step in pipeline regardless of + success or failure. +1. Executes `deploy_job` when you run it manually in the GitLab UI. -The following example creates a job named `timed rollout 10%` that is executed 30 minutes after the previous stage completes: +**Additional details**: -```yaml -timed rollout 10%: - stage: deploy - script: echo 'Rolling out 10% ...' - when: delayed - start_in: 30 minutes -``` +- In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201938) and later, you + can use `when:manual` in the same job as [`trigger`](#trigger). In GitLab 13.4 and + earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. +- The default behavior of `allow_failure` changes to `true` with `when: manual`. + However, if you use `when: manual` with [`rules`](#rules), `allow_failure` defaults + to `false`. -To stop the active timer of a delayed job, click the **{time-out}** (**Unschedule**) button. -This job can no longer be scheduled to run automatically. You can, however, execute the job manually. +**Related topics**: -To start a delayed job immediately, click the **Play** button. -Soon GitLab Runner picks up and starts the job. +- `when` can be used with [`rules`](#rules) for more dynamic job control. +- `when` can be used with [`workflow`](#workflow) to control when a pipeline can start. ### `environment` @@ -2225,11 +2155,11 @@ In the above example, the `review_app` job deploys to the `review` environment. A new `stop_review_app` job is listed under `on_stop`. After the `review_app` job is finished, it triggers the `stop_review_app` job based on what is defined under `when`. In this case, -it is set to `manual`, so it needs a [manual action](#whenmanual) from +it is set to `manual`, so it needs a [manual action](../jobs/job_control.md#create-a-job-that-must-be-run-manually) from the GitLab UI to run. Also in the example, `GIT_STRATEGY` is set to `none`. If the -`stop_review_app` job is [automatically triggered](../environments/index.md#stopping-an-environment), +`stop_review_app` job is [automatically triggered](../environments/index.md#stop-an-environment), the runner won't try to check out the code after the branch is deleted. The example also overwrites global variables. If your `stop` `environment` job depends @@ -3262,7 +3192,7 @@ dashboards. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207528) in GitLab 13.0. > - Requires [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 and above. -The `terraform` report obtains a Terraform `tfplan.json` file. [JQ processing required to remove credentials](../../user/infrastructure/mr_integration.md#setup). The collected Terraform +The `terraform` report obtains a Terraform `tfplan.json` file. [JQ processing required to remove credentials](../../user/infrastructure/mr_integration.md#configure-terraform-report-artifacts). The collected Terraform plan report uploads to GitLab as an artifact and displays in merge requests. For more information, see [Output `terraform plan` information into a merge request](../../user/infrastructure/mr_integration.md). @@ -3346,6 +3276,50 @@ Coverage output from [child pipelines](../pipelines/parent_child_pipelines.md) i or displayed. Check [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/280818) for more details. +### `dast_configuration` **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5981) in GitLab 14.1. + +Use the `dast_configuration` keyword to specify a site profile and scanner profile to be used in a +CI/CD configuration. Both profiles must first have been created in the project. The job's stage must +be `dast`. + +**Keyword type**: Job keyword. You can use only as part of a job. + +**Possible inputs**: One each of `site_profile` and `scanner_profile`. + +- Use `site_profile` to specify the site profile to be used in the job. +- Use `scanner_profile` to specify the scanner profile to be used in the job. + +**Example of `dast_configuration`**: + +```yaml +stages: + - build + - dast + +include: + - template: DAST.gitlab-ci.yml + +dast: + dast_configuration: + site_profile: "Example Co" + scanner_profile: "Quick Passive Test" +``` + +In this example, the `dast` job extends the `dast` configuration added with the `include:` keyword +to select a specific site profile and scanner profile. + +**Additional details**: + +- Settings contained in either a site profile or scanner profile take precedence over those + contained in the DAST template. + +**Related topics**: + +- [Site profile](../../user/application_security/dast/index.md#site-profile). +- [Scanner profile](../../user/application_security/dast/index.md#scanner-profile). + ### `retry` > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3515) in GitLab 11.5, you can control which failures to retry on. @@ -3500,7 +3474,7 @@ but with different variable values for each instance of the job. There can be from 2 to 50 jobs. Jobs can only run in parallel if there are multiple runners, or a single runner is -[configured to run multiple jobs concurrently](#use-your-own-runners). +configured to run multiple jobs concurrently. Every job gets the same `CI_NODE_TOTAL` [CI/CD variable](../variables/index.md#predefined-cicd-variables) value, and a unique `CI_NODE_INDEX` value. @@ -3590,6 +3564,23 @@ deploystacks: [gcp, data] deploystacks: [vultr, data] ``` +In [GitLab 14.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/239737), you can +use the variables defined in `parallel: matrix` with the [`tags`](#tags) keyword for +dynamic runner selection. + +```yaml +deploystacks: + stage: deploy + parallel: + matrix: + - PROVIDER: aws + STACK: [monitoring, app1] + - PROVIDER: gcp + STACK: [data] + tags: + - ${PROVIDER}-${STACK} +``` + ### `trigger` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. @@ -3612,7 +3603,7 @@ view which job triggered a downstream pipeline. In the [pipeline graph](../pipel hover over the downstream pipeline job. In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201938) and later, you -can use [`when:manual`](#whenmanual) in the same job as `trigger`. In GitLab 13.4 and +can use [`when:manual`](#when) in the same job as `trigger`. In GitLab 13.4 and earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. You [cannot start `manual` trigger jobs with the API](https://gitlab.com/gitlab-org/gitlab/-/issues/284086). @@ -3931,19 +3922,28 @@ image: registry.gitlab.com/gitlab-org/release-cli:latest #### `release-cli` for shell executors -> [Introduced](https://gitlab.com/gitlab-org/release-cli/-/issues/21) in GitLab 13.8. +> - [Introduced](https://gitlab.com/gitlab-org/release-cli/-/issues/21) in GitLab 13.8. +> - [Changed](https://gitlab.com/gitlab-org/release-cli/-/merge_requests/108): the `release-cli` binaries are also +[available in the Package Registry](https://gitlab.com/jaime/release-cli/-/packages) +starting from GitLab 14.2. For GitLab Runner shell executors, you can download and install the `release-cli` manually for your [supported OS and architecture](https://release-cli-downloads.s3.amazonaws.com/latest/index.html). Once installed, the `release` keyword should be available to you. **Install on Unix/Linux** -1. Download the binary for your system, in the following example for amd64 systems: +1. Download the binary for your system from S3, in the following example for amd64 systems: ```shell curl --location --output /usr/local/bin/release-cli "https://release-cli-downloads.s3.amazonaws.com/latest/release-cli-linux-amd64" ``` +Or from the GitLab package registry: + + ```shell + curl --location --output /usr/local/bin/release-cli "https://gitlab.com/api/v4/projects/gitlab-org%2Frelease-cli/packages/generic/release-cli/latest/release-cli-darwin-amd64" + ``` + 1. Give it permissions to execute: ```shell @@ -4527,50 +4527,6 @@ You can use [CI/CD variables](../variables/index.md) to configure how the runner You can also use variables to configure how many times a runner [attempts certain stages of job execution](../runners/configure_runners.md#job-stages-attempts). -## `dast_configuration` **(ULTIMATE)** - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5981) in GitLab 14.1. - -Use the `dast_configuration` keyword to specify a site profile and scanner profile to be used in a -CI/CD configuration. Both profiles must first have been created in the project. The job's stage must -be `dast`. - -**Keyword type**: Job keyword. You can use only as part of a job. - -**Possible inputs**: One each of `site_profile` and `scanner_profile`. - -- Use `site_profile` to specify the site profile to be used in the job. -- Use `scanner_profile` to specify the scanner profile to be used in the job. - -**Example of `dast_configuration`**: - -```yaml -stages: - - build - - dast - -include: - - template: DAST.gitlab-ci.yml - -dast: - dast_configuration: - site_profile: "Example Co" - scanner_profile: "Quick Passive Test" -``` - -In this example, the `dast` job extends the `dast` configuration added with the `include:` keyword -to select a specific site profile and scanner profile. - -**Additional details**: - -- Settings contained in either a site profile or scanner profile take precedence over those - contained in the DAST template. - -**Related topics**: - -- [Site profile](../../user/application_security/dast/index.md#site-profile). -- [Scanner profile](../../user/application_security/dast/index.md#scanner-profile). - ## YAML-specific features In your `.gitlab-ci.yml` file, you can use YAML-specific features like anchors (`&`), aliases (`*`), diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index 9e118895d7c..93c1a6afe69 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -14,6 +14,85 @@ You can use special syntax in [`script`](index.md#script) sections to: - [Create custom collapsible sections](../jobs/index.md#custom-collapsible-sections) to simplify job log output. +## Use special characters with `script:` + +Sometimes, `script` commands must be wrapped in single or double quotes. +For example, commands that contain a colon (`:`) must be wrapped in single quotes (`'`). +The YAML parser needs to interpret the text as a string rather than +a "key: value" pair. + +For example, this script uses a colon: + +```yaml +job: + script: + - curl --request POST --header 'Content-Type: application/json' "https://gitlab/api/v4/projects" +``` + +To be considered valid YAML, you must wrap the entire command in single quotes. If +the command already uses single quotes, you should change them to double quotes (`"`) +if possible: + +```yaml +job: + script: + - 'curl --request POST --header "Content-Type: application/json" "https://gitlab/api/v4/projects"' +``` + +You can verify the syntax is valid with the [CI Lint](../lint.md) tool. + +Be careful when using these characters as well: + +- `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. + +## Ignore non-zero exit codes + +When script commands return an exit code other than zero, the job fails and further +commands do not execute. + +Store the exit code in a variable to avoid this behavior: + +```yaml +job: + script: + - false || exit_code=$? + - if [ $exit_code -ne 0 ]; then echo "Previous command failed"; fi; +``` + +## Set a default `before_script` or `after_script` for all jobs + +You can use [`before_script`](index.md#before_script) and [`after_script`](index.md#after_script) +with [`default`](index.md#custom-default-keyword-values): + +- Use `before_script` with `default` to define a default array of commands that + should run before the `script` commands in all jobs. +- Use `after_script` with default to define a default array of commands + that should run after the job completes. + +You can overwrite a default by defining a different one in a job. To ignore the default +use `before_script: []` or `after_script: []`: + +```yaml +default: + before_script: + - echo "Execute this `before_script` in all jobs by default." + after_script: + - echo "Execute this `after_script` in all jobs by default." + +job1: + script: + - echo "These script commands execute after the default `before_script`," + - echo "and before the default `after_script`." + +job2: + before_script: + - echo "Execute this script instead of the default `before_script`." + script: + - echo "This script executes after the job's `before_script`," + - echo "but the job does not use the default `after_script`." + after_script: [] +``` + ## Split long commands You can split long commands into multiline commands to improve readability with |
