diff options
Diffstat (limited to 'doc/ci')
42 files changed, 480 insertions, 208 deletions
diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 777bbf6053f..88d59b1f223 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -2,12 +2,11 @@ stage: Verify group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: index, concepts, howto --- # Caching in GitLab CI/CD **(FREE)** -A cache is one or more files that a job downloads and saves. Subsequent jobs that use +A cache is one or more files a job downloads and saves. Subsequent jobs that use the same cache don't have to download the files again, so they execute more quickly. To learn how to define the cache in your `.gitlab-ci.yml` file, @@ -31,7 +30,7 @@ can't link to files outside it. - Subsequent pipelines can use the cache. - Subsequent jobs in the same pipeline can use the cache, if the dependencies are identical. - Different projects cannot share the cache. -- Protected and non-protected branches do not share the cache. +- By default, protected and non-protected branches [do not share the cache](#cache-key-names). However, you can [change this behavior](#use-the-same-cache-for-all-branches). ### Artifacts @@ -447,13 +446,11 @@ is stored on the machine where GitLab Runner is installed. The location also dep If you use cache and artifacts to store the same path in your jobs, the cache might be overwritten because caches are restored before artifacts. -### Segregation of caches between protected and non-protected branches +#### Cache key names > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330047) in GitLab 15.0. A suffix is added to the cache key, with the exception of the [fallback cache key](#use-a-fallback-cache-key). -This is done in order to prevent cache poisoning that might occur through manipulation of the cache in a non-protected -branch. Any subsequent protected-branch jobs would then potentially use a poisoned cache from the preceding job. As an example, assuming that `cache.key` is set to `$CI_COMMIT_REF_SLUG`, and that we have two branches `main` and `feature`, then the following table represents the resulting cache keys: @@ -463,6 +460,24 @@ and `feature`, then the following table represents the resulting cache keys: | `main` | `main-protected` | | `feature` | `feature-non_protected` | +##### Use the same cache for all branches + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361643) in GitLab 15.0. + +If you do not want to use [cache key names](#cache-key-names), +you can have all branches (protected and unprotected) use the same cache. + +The cache separation with [cache key names](#cache-key-names) is a security feature +and should only be disabled in an environment where all users with Developer role are highly trusted. + +To use the same cache for all branches: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **General pipelines**. +1. Clear the **Use separate caches for protected branches** checkbox. +1. Select **Save changes**. + ### How archiving and extracting works This example shows two jobs in two consecutive stages: @@ -552,12 +567,10 @@ The next time the pipeline runs, the cache is stored in a different location. ### Clear the cache manually -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41249) in GitLab 10.4. - You can clear the cache in the GitLab UI: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Pipelines** page. +1. On the left sidebar, select **CI/CD > Pipelines**. 1. In the top right, select **Clear runner caches**. On the next commit, your CI/CD jobs use a new cache. @@ -576,7 +589,7 @@ If you have a cache mismatch, follow these steps to troubleshoot. | You use multiple standalone runners (not in autoscale mode) attached to one project without a shared cache. | Use only one runner for your project or use multiple runners with distributed cache enabled. | | You use runners in autoscale mode without a distributed cache enabled. | Configure the autoscale runner to use a distributed cache. | | The machine the runner is installed on is low on disk space or, if you've set up distributed cache, the S3 bucket where the cache is stored doesn't have enough space. | Make sure you clear some space to allow new caches to be stored. There's no automatic way to do this. | -| You use the same `key` for jobs where they cache different paths. | Use different cache keys to that the cache archive is stored to a different location and doesn't overwrite wrong caches. | +| You use the same `key` for jobs where they cache different paths. | Use different cache keys so that the cache archive is stored to a different location and doesn't overwrite wrong caches. | #### Cache mismatch example 1 diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index c785e2f29ea..21b970536bc 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -73,7 +73,7 @@ stages: hello-world: stage: chatops rules: - - if: '$CI_PIPELINE_SOURCE == "chat"' + - if: $CI_PIPELINE_SOURCE == "chat" script: - echo "Hello World" ``` @@ -93,7 +93,7 @@ stages: ls: stage: chatops rules: - - if: '$CI_PIPELINE_SOURCE == "chat"' + - if: $CI_PIPELINE_SOURCE == "chat" script: - echo "This command will not be shown." - echo -e "section_start:$( date +%s ):chat_reply\r\033[0K\n$( ls -la )\nsection_end:$( date +%s ):chat_reply\r\033[0K" diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 80e4d8fbe27..c89ce2675e4 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -59,7 +59,7 @@ Some credentials are required to be able to run `aws` commands: | `AWS_DEFAULT_REGION` | Your region code | NOTE: - When you create a variable it's set to be [protected by default](../variables/index.md#protect-a-cicd-variable). If you want to use the `aws` commands on branches or tags that are not protected, make sure to uncheck the **Protect variable** checkbox. + When you create a variable it's set to be [protected by default](../variables/index.md#protected-cicd-variables). If you want to use the `aws` commands on branches or tags that are not protected, make sure to uncheck the **Protect variable** checkbox. 1. You can now use `aws` commands in the `.gitlab-ci.yml` file of this project: diff --git a/doc/ci/cloud_services/aws/index.md b/doc/ci/cloud_services/aws/index.md index aa38562c866..dc8dfd95899 100644 --- a/doc/ci/cloud_services/aws/index.md +++ b/doc/ci/cloud_services/aws/index.md @@ -90,3 +90,4 @@ This error can occur for multiple reasons: - The cloud administrator has not configured the project to use OIDC with GitLab. - The role is restricted from being run on the branch or tag. See [configure a conditional role](../index.md). +- `StringEquals` is used instead of `StringLike` when using a wildcard condition. See [related issue](https://gitlab.com/guided-explorations/aws/configure-openid-connect-in-aws/-/issues/2#note_852901934). diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 65c907b8e7b..2e514fd0f0a 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -31,7 +31,7 @@ If you disable GitLab CI/CD in a project: - Existing jobs and pipelines are not deleted. Re-enable CI/CD to access them again. The project or instance settings do not enable or disable pipelines run in an -[external integration](../user/project/integrations/overview.md#integrations-listing). +[external integration](../user/project/integrations/index.md#available-integrations). ## Enable CI/CD in a project diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index 2a5927a96c2..23fa5d45196 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -34,7 +34,7 @@ To configure deployment approvals for a project: ### Create a deployment job -Create a deployment job in the `.gitlab-ci.yaml` file of the desired project. The job does **not** need to be manual (`when: manual`). +Create a deployment job in the `.gitlab-ci.yml` file of the desired project. The job does **not** need to be manual (`when: manual`). Example: @@ -92,7 +92,8 @@ Maintainer role. #### Multiple approval rules -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 14.10 with a flag named `deployment_approval_rules`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 14.10 with a flag named `deployment_approval_rules`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 15.0. [Feature flag `deployment_approval_rules`](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) removed. 1. Using the [REST API](../../api/group_protected_environments.md#protect-an-environment). 1. `deploy_access_levels` represents which entity can execute the deployment job. @@ -138,7 +139,9 @@ To approve or reject a deployment to a protected environment using the UI: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. +1. Select the environment's name. 1. In the deployment's row, select **Approval options** (**{thumb-up}**). +1. Optional. Add a comment which describes your reason for approving or rejecting the deployment. 1. Select **Approve** or **Reject**. NOTE: diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 0e73dc4f7cd..5b2e2045bdc 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -6,7 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Deployment safety **(FREE)** -Deployment jobs can be more sensitive than other jobs in a pipeline, +[Deployment jobs](../jobs/#deployment-jobs) are a specific kind of CI/CD +job. They can be more sensitive than other jobs in a pipeline, and might need to be treated with extra care. GitLab has several features that help maintain deployment security and stability. @@ -64,14 +65,14 @@ For more information, see [Resource Group documentation](../resource_groups/inde ## Skip outdated deployment jobs -The execution order of pipeline jobs can vary from run to run, which could cause -undesired behavior. For example, a deployment job in a newer pipeline could -finish before a deployment job in an older pipeline. -This creates a race condition where the older deployment finished later, +The effective execution order of pipeline jobs can vary from run to run, which +could cause undesired behavior. For example, a [deployment job](../jobs/#deployment-jobs) +in a newer pipeline could finish before a deployment job in an older pipeline. +This creates a race condition where the older deployment finishes later, overwriting the "newer" deployment. You can ensure that older deployment jobs are cancelled automatically when a newer deployment -runs by enabling the [Skip outdated deployment jobs](../pipelines/settings.md#skip-outdated-deployment-jobs) feature. +job is started by enabling the [Skip outdated deployment jobs](../pipelines/settings.md#skip-outdated-deployment-jobs) feature. Example of a problematic pipeline flow **before** enabling Skip outdated deployment jobs: @@ -85,7 +86,7 @@ The improved pipeline flow **after** enabling Skip outdated deployment jobs: 1. Pipeline-A is created on the default branch. 1. Later, Pipeline-B is created on the default branch (with a newer SHA). 1. The `deploy` job in Pipeline-B finishes first, and deploys the newer code. -1. The `deploy` job in Pipeline-A is automatically cancelled, so that it doesn't overwrite the deployment from the newer pipeline. +1. The `deploy` job in Pipeline-A was automatically cancelled, so that it doesn't overwrite the deployment from the newer pipeline. ## Prevent deployments during deploy freeze windows @@ -111,7 +112,7 @@ for an explanation of these roles and the permissions of each. Production secrets are needed to deploy successfully. For example, when deploying to the cloud, cloud providers require these secrets to connect to their services. In the project settings, you can -define and protect CI/CD variables for these secrets. [Protected variables](../variables/index.md#protect-a-cicd-variable) +define and protect CI/CD variables for these secrets. [Protected variables](../variables/index.md#protected-cicd-variables) are only passed to pipelines running on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). The other pipelines don't get the protected variable. You can also @@ -132,7 +133,7 @@ permission model that isolates the CD permissions from the original project and original users with the Maintainer role for the project from accessing the production secret and CD configuration. You can connect the CD project to your development projects by using [multi-project pipelines](../pipelines/multi_project_pipelines.md). -## Protect `gitlab-ci.yml` from change +## Protect `.gitlab-ci.yml` from change A `.gitlab-ci.yml` may contain rules to deploy an application to the production server. This deployment usually runs automatically after pushing a merge request. To prevent developers from diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index bc52fd1f16c..d359ed078ff 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -114,7 +114,7 @@ available, [demonstrating configuration of timed rollouts](https://gitlab.com/gl ## Blue-Green Deployment NOTE: -As of GitLab 13.7, teams can leverage an Ingress annotation and [set traffic weight](../../user/project/canary_deployments.md#how-to-change-the-traffic-weight-on-a-canary-ingress) +As of GitLab 13.7, teams can leverage an Ingress annotation and [set traffic weight](../../user/project/canary_deployments.md#how-to-change-the-traffic-weight-on-a-canary-ingress-deprecated) as an alternative approach to the blue-green deployment strategy documented here. Also sometimes known as A/B deployment or red-black deployment, this technique is used to reduce diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 3822b32181f..7e1ef5efaa5 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -560,11 +560,8 @@ GitLab automatically triggers the `stop_review_app` job to stop the environment. #### Multiple stop actions for an environment -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22456) in GitLab 14.10 [with a flag](../../administration/feature_flags.md) named `environment_multiple_stop_actions`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `environment_multiple_stop_actions`. -On GitLab.com, this feature is not available. We are enabling in phases and the status can be tracked in [issue 358911](https://gitlab.com/gitlab-org/gitlab/-/issues/358911). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22456) in GitLab 14.10 [with a flag](../../administration/feature_flags.md) named `environment_multiple_stop_actions`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/358911) in GitLab 15.0. [Feature flag `environment_multiple_stop_actions`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86685) removed. This feature is useful when you need to perform multiple **parallel** stop actions on an environment. @@ -645,17 +642,14 @@ To delete a stopped environment in the GitLab UI: 1. Next to the environment you want to delete, select **Delete environment**. 1. On the confirmation dialog box, select **Delete environment**. -### Prepare an environment without creating a deployment +### Access an environment for preparation or verification purposes > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208655) in GitLab 13.2. -By default, when GitLab CI/CD runs a job for a specific environment, it -triggers a deployment and [(optionally) cancels outdated -deployments](deployment_safety.md#ensure-only-one-deployment-job-runs-at-a-time). +You can define a job that accesses an environment for various purposes, such as verification or preparation. This +effectively bypasses deployment creation, so that you can adjust your CD workflow more accurately. -To use an environment without creating a new deployment, and without -cancelling outdated deployments, append the keyword `action: prepare` to your -job: +To do so, add either `action: prepare`, `action: verify`, or `action: access` to the `environment` section of your job: ```yaml build: @@ -668,8 +662,8 @@ build: url: https://staging.example.com ``` -This gives you access to [environment-scoped variables](#scope-environments-with-specs), -and can be used to [protect builds from unauthorized access](protected_environments.md). +This gives you access to environment-scoped variables, and can be used to protect builds from unauthorized access. Also, +it's effective to avoid the [skip outdated deployment jobs](deployment_safety.md#skip-outdated-deployment-jobs) feature. ### Group similar environments diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index c74af852a9a..2e9e2dd33bf 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -35,7 +35,7 @@ You can also view or fork the complete [example source](https://gitlab.com/gitla } ``` -1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in npm's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#files). +1. Update the `files` key with glob patterns that selects all files that should be included in the published module. More information about `files` can be found [in npm's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#files). 1. Add a `.gitignore` file to the project to avoid committing `node_modules`: diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 23055514839..5f22fb894e5 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -83,4 +83,4 @@ You can check documentation for your specific OS to learn how to find and displa hidden files. If there is no `.gitmodules` file, it's possible the submodule settings are in a -[gitconfig](https://www.atlassian.com/git/tutorials/setting-up-a-repository/git-config) file. +[`git config`](https://www.atlassian.com/git/tutorials/setting-up-a-repository/git-config) file. diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index e589fd8b045..42827dc2d48 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -392,5 +392,5 @@ deploy me: The behavior of deployment jobs can be controlled with [deployment safety](../environments/deployment_safety.md) settings like -[skipping outdated deployment jobs](../environments/deployment_safety.md#prevent-deployments-during-deploy-freeze-windows) +[skipping outdated deployment jobs](../environments/deployment_safety.md#skip-outdated-deployment-jobs) and [ensuring only one deployment job runs at a time](../environments/deployment_safety.md#ensure-only-one-deployment-job-runs-at-a-time). diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 3a302cf352b..83747f36597 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -40,10 +40,10 @@ The following example uses `if` to define that the job runs in only two specific job: script: echo "Hello, Rules!" rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" when: manual allow_failure: true - - if: '$CI_PIPELINE_SOURCE == "schedule"' + - if: $CI_PIPELINE_SOURCE == "schedule" ``` - If the pipeline is for a merge request, the first rule matches, and the job @@ -67,9 +67,9 @@ run them in all other cases: job: script: echo "Hello, Rules!" rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" when: never - - if: '$CI_PIPELINE_SOURCE == "schedule"' + - if: $CI_PIPELINE_SOURCE == "schedule" when: never - when: on_success ``` @@ -118,7 +118,7 @@ For example: docker build: script: docker build -t my-image:$CI_COMMIT_REF_SLUG . rules: - - if: '$VAR == "string value"' + - if: $VAR == "string value" changes: # Include the job and set to when:manual if any of the follow paths match a modified file. - Dockerfile - docker/scripts/* @@ -160,7 +160,7 @@ For example: job: script: echo "This job creates double pipelines!" rules: - - if: '$CUSTOM_VARIABLE == "false"' + - if: $CUSTOM_VARIABLE == "false" when: never - when: always ``` @@ -181,7 +181,7 @@ To avoid duplicate pipelines, you can: job: script: echo "This job does NOT create double pipelines!" rules: - - if: '$CUSTOM_VARIABLE == "true" && $CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CUSTOM_VARIABLE == "true" && $CI_PIPELINE_SOURCE == "merge_request_event" ``` You can also avoid duplicate pipelines by changing the job rules to avoid either push (branch) @@ -195,7 +195,7 @@ without `workflow: rules`: job: script: echo "This job does NOT create double pipelines!" rules: - - if: '$CI_PIPELINE_SOURCE == "push"' + - if: $CI_PIPELINE_SOURCE == "push" when: never - when: always ``` @@ -207,8 +207,8 @@ You should not include both push and merge request pipelines in the same job wit job: script: echo "This job creates double pipelines!" rules: - - if: '$CI_PIPELINE_SOURCE == "push"' - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_PIPELINE_SOURCE == "push" + - if: $CI_PIPELINE_SOURCE == "merge_request_event" ``` Also, do not mix `only/except` jobs with `rules` jobs in the same pipeline. @@ -222,7 +222,7 @@ job-with-no-rules: job-with-rules: script: echo "This job runs in merge request pipelines." rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" ``` For every change pushed to the branch, duplicate pipelines run. One @@ -259,10 +259,10 @@ add the job to any other pipeline type. job: script: echo "Hello, Rules!" rules: - - if: '$CI_PIPELINE_SOURCE == "schedule"' + - if: $CI_PIPELINE_SOURCE == "schedule" when: manual allow_failure: true - - if: '$CI_PIPELINE_SOURCE == "push"' + - if: $CI_PIPELINE_SOURCE == "push" ``` The following example runs the job as a `when: on_success` job in [merge request pipelines](../pipelines/merge_request_pipelines.md) @@ -272,25 +272,25 @@ and scheduled pipelines. It does not run in any other pipeline type. job: script: echo "Hello, Rules!" rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' - - if: '$CI_PIPELINE_SOURCE == "schedule"' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" + - if: $CI_PIPELINE_SOURCE == "schedule" ``` Other commonly used variables for `if` clauses: - `if: $CI_COMMIT_TAG`: If changes are pushed for a tag. - `if: $CI_COMMIT_BRANCH`: If changes are pushed to any branch. -- `if: '$CI_COMMIT_BRANCH == "main"'`: If changes are pushed to `main`. -- `if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'`: If changes are pushed to the default +- `if: $CI_COMMIT_BRANCH == "main"`: If changes are pushed to `main`. +- `if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH`: If changes are pushed to the default branch. Use when you want to have the same configuration in multiple projects with different default branches. -- `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression. +- `if: $CI_COMMIT_BRANCH =~ /regex-expression/`: If the commit branch matches a regular expression. - `if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_COMMIT_TITLE =~ /Merge branch.*/`: If the commit branch is the default branch and the commit message title matches a regular expression. For example, the default commit message for a merge commit starts with `Merge branch`. -- `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/index.md#custom-cicd-variables) +- `if: $CUSTOM_VARIABLE !~ /regex-expression/`: If the [custom variable](../variables/index.md#custom-cicd-variables) `CUSTOM_VARIABLE` does **not** match a regular expression. -- `if: '$CUSTOM_VARIABLE == "value1"'`: If the custom variable `CUSTOM_VARIABLE` is +- `if: $CUSTOM_VARIABLE == "value1"`: If the custom variable `CUSTOM_VARIABLE` is exactly `value1`. ### Variables in `rules:changes` @@ -643,7 +643,7 @@ timed rollout 10%: To stop the active timer of a delayed job, select **Unschedule** (**{time-out}**). This job can no longer be scheduled to run automatically. You can, however, execute the job manually. -To start a delayed job immediately, select **Play** (**{play}**). +To start a delayed job manually, select **Unschedule** (**{time-out}**) to stop the delay timer and then select **Play** (**{play}**). Soon GitLab Runner starts the job. ## Parallelize large jobs diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index d95199a70d0..b1c4c62c465 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -21,7 +21,7 @@ For advanced CI/CD teams, [custom project templates](../../user/admin_area/custo If you have questions that are not answered here, the [GitLab community forum](https://forum.gitlab.com/) can be a great resource. -## `config.yml` vs `gitlab-ci.yml` +## `config.yml` vs `.gitlab-ci.yml` CircleCI's `config.yml` configuration file defines scripts, jobs, and workflows (known as "stages" in GitLab). In GitLab, a similar approach is used with a `.gitlab-ci.yml` file in the root directory of your repository. @@ -166,7 +166,7 @@ job1: script: - make build rules: - - if: '$CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_REF_NAME == "try-schedule-workflow"' + - if: $CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_REF_NAME == "try-schedule-workflow" ``` After the pipeline configuration is saved, you configure the cron schedule in the [GitLab UI](../pipelines/schedules.md#add-a-pipeline-schedule), and can enable or disable schedules in the UI as well. @@ -221,7 +221,7 @@ deploy: script: - echo "Deploy job" rules: - - if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^rc-/' + - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^rc-/ ``` ### Caching diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index 19b1a6cad1f..c59116ea8ed 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -305,7 +305,7 @@ my_job: In GitLab, we use the [`variables` keyword](../yaml/index.md#variables) to define different variables at runtime. These can also be set up through the GitLab UI, under CI/CD settings. See also our [general documentation on variables](../variables/index.md), -including the section on [protected variables](../variables/index.md#protect-a-cicd-variable) which can be used +including the section on [protected variables](../variables/index.md#protected-cicd-variables) which can be used to limit access to certain variables to certain environments or runners: ```yaml diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index d72cb14681d..57a9d7a9358 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -19,6 +19,7 @@ From the pipeline editor page you can: - [Validate](#validate-ci-configuration) your configuration syntax while editing the file. - Do a deeper [lint](#lint-ci-configuration) of your configuration, that verifies it with any configuration added with the [`include`](../yaml/index.md#include) keyword. +- View a [list of the CI/CD configuration added with the `include` keyword](#view-included-cicd-configuration). - See a [visualization](#visualize-ci-configuration) of the current configuration. - View an [expanded](#view-expanded-configuration) version of your configuration. - [Commit](#commit-changes-to-ci-configuration) the changes to a specific branch. @@ -50,6 +51,20 @@ reflected in the CI lint. It displays the same results as the existing [CI Lint  +## View included CI/CD configuration + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7064) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `pipeline_editor_file_tree`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) +named `pipeline_editor_file_tree`. + +You can review configuration added with the [`include`](../yaml/index.md#include) +keyword in the pipeline editor. In the top right, select the file tree (**{file-tree}**) +to see a list of all included configuration files. Selected files open in a new tab +for review. + ## Visualize CI configuration > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241722) in GitLab 13.5. diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index f9d9a4f738b..2b18b1d353b 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -90,6 +90,8 @@ To view CI/CD minutes being used for your group: ## View CI/CD minutes used by a personal namespace +> Displaying shared runners duration [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345795) in GitLab 15.0. + You can view the number of CI/CD minutes being used by a personal namespace: 1. On the top bar, in the top right corner, select your avatar. @@ -103,7 +105,7 @@ These additional CI/CD minutes: - Are used only after the monthly quota included in your subscription runs out. - Are carried over to the next month, if any remain at the end of the month. -- Don't expire. +- Are valid for 12 months from date of purchase or until all minutes are consumed, whichever comes first. Expiry of minutes is not currently enforced. If you use more CI/CD minutes than your monthly quota, when you purchase more, those CI/CD minutes are deducted from your quota. For example, with a GitLab SaaS @@ -191,17 +193,41 @@ The cost factor for a job running on a shared runner is: ### Additional costs on GitLab SaaS -On GitLab SaaS, shared runners can have different cost factors depending on the cost involved -in executing the runner. For example, a high spec shared runner could be set to have a cost factor of `2`. -Conversely, a shared runner that executes jobs for public projects could have a low cost factor, like `0.008`. +GitLab SaaS shared runners have different cost factors, depending on the runner type (Linux, Windows, macOS) and the virtual machine configuration. + +| GitLab SaaS runner type | Virtual machine configuration | CI/CD minutes cost factor | +| :--------- | :------------------- | :--------- | +| Linux OS + Docker executor| 1 vCPU, 3.75 GB RAM |1| +| macOS + shell executor | 4 vCPU, 10 GB RAM| 6 | ### Monthly reset of CI/CD minutes On the first day of each calendar month, the accumulated usage of CI/CD minutes is reset to `0` -for all namespaces that use shared runners. +for all namespaces that use shared runners. This means your full quota is available, and +calculations start again from `0`. + +For example, if you have a monthly quota of `10,000` CI/CD minutes: + +- On **1st April**, you have `10,000` minutes. +- During April, you use only `6,000` of the `10,000` minutes. +- On **1st May**, the accumulated usage of minutes resets to `0`, and you have `10,000` minutes to use again + during May. Usage data for the previous month is kept to show historical view of the consumption over time. +### Monthly rollover of purchased CI/CD minutes + +If you purchase additional CI/CD minutes and don't use the full amount, the remaining amount rolls over to +the next month. + +For example: + +- On **1st April**, you purchase `5,000` CI/CD minutes. +- During April, you use only `3,000` of the `5,000` minutes. +- On **1st May**, the remaining `2,000` minutes roll over and are added to your monthly quota. + +Additional CI/CD minutes are a one-time purchase and do not renew or refresh each month. + ## What happens when you exceed the quota When the quota of CI/CD minutes is used for the current month, GitLab stops diff --git a/doc/ci/pipelines/img/downstream_pipeline_actions.png b/doc/ci/pipelines/img/downstream_pipeline_actions.png Binary files differnew file mode 100644 index 00000000000..4c4384bab57 --- /dev/null +++ b/doc/ci/pipelines/img/downstream_pipeline_actions.png diff --git a/doc/ci/pipelines/img/multi_pipeline_mini_graph.gif b/doc/ci/pipelines/img/multi_pipeline_mini_graph.gif Binary files differdeleted file mode 100644 index de49ba5aa12..00000000000 --- a/doc/ci/pipelines/img/multi_pipeline_mini_graph.gif +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 4d1af735b13..c6142ebefc5 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -449,6 +449,22 @@ Pipeline analytics are available on the [**CI/CD Analytics** page](../../user/an Pipeline status and test coverage report badges are available and configurable for each project. For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges). +### Downstream pipelines + +> Cancel or retry downstream pipelines from the graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. + +In the pipeline graph view, downstream pipelines ([Multi-project pipelines](multi_project_pipelines.md) +and [Parent-child pipelines](parent_child_pipelines.md)) display as a list of cards +on the right of the graph. + +To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**) +on the pipeline's card. + +To retry a failed downstream pipeline, select **Retry** (**{retry}**) +on the pipeline's card. + + + ## Pipelines API GitLab provides API endpoints to: diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 75408de2721..89839de718b 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -20,7 +20,7 @@ Branch pipelines: - Run when you push a new commit to a branch. - Are the default type of pipeline. - Have access to [some predefined variables](../variables/predefined_variables.md). -- Have access to [protected variables](../variables/index.md#protect-a-cicd-variable) and [protected runners](../runners/configure_runners.md#prevent-runners-from-revealing-sensitive-information). +- Have access to [protected variables](../variables/index.md#protected-cicd-variables) and [protected runners](../runners/configure_runners.md#prevent-runners-from-revealing-sensitive-information). Merge request pipelines: @@ -33,7 +33,7 @@ Merge request pipelines: - Do not run by default. The jobs in the CI/CD configuration file [must be configured](#prerequisites) to run in merge request pipelines. - Have access to [more predefined variables](#available-predefined-variables). -- Do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable) or [protected runners](../runners/configure_runners.md#prevent-runners-from-revealing-sensitive-information). +- Do not have access to [protected variables](../variables/index.md#protected-cicd-variables) or [protected runners](../runners/configure_runners.md#prevent-runners-from-revealing-sensitive-information). Both of these types of pipelines can appear on the **Pipelines** tab of a merge request. @@ -127,12 +127,12 @@ Pipelines for forks display with the **fork** badge in the parent project: ### Run pipelines in the parent project **(PREMIUM)** -Project members in the parent project can choose to run a merge request pipeline +Project members in the parent project can trigger a merge request pipeline for a merge request submitted from a fork project. This pipeline: - Is created and runs in the parent (target) project, not the fork (source) project. -- Uses the CI/CD configuration present in the fork project's branch -- Uses the parent project's CI/CD configuration, resources, and project CI/CD variables. +- Uses the CI/CD configuration present in the fork project's branch. +- Uses the parent project's CI/CD settings, resources, and project CI/CD variables. - Uses the permissions of the parent project member that triggers the pipeline. Run pipelines in fork project MRs to ensure that the post-merge pipeline passes in @@ -142,8 +142,12 @@ running the pipeline in the parent project uses the parent project's trusted run WARNING: Fork merge requests can contain malicious code that tries to steal secrets in the parent project when the pipeline runs, even before merge. As a reviewer, carefully -check the changes in the merge request before triggering the pipeline. GitLab shows -a warning that you must accept before you can trigger the pipeline. +check the changes in the merge request before triggering the pipeline. If you trigger +the pipeline by selecting **Run pipeline** or applying a suggestion, GitLab shows +a warning that you must accept before the pipeline runs. If you trigger the pipeline +by using any other method, including the API, [`/rebase` quick action](../../user/project/quick_actions.md#issues-merge-requests-and-epics), +or [**Rebase** option](../../user/project/merge_requests/methods/index.md#rebasing-in-semi-linear-merge-methods), +**no warning displays**. Prerequisites: @@ -152,10 +156,10 @@ Prerequisites: user running the pipeline. Otherwise, the **Pipelines** tab does not display in the merge request. -To run a pipeline in the parent project for a merge request from a fork project: +To use the UI to run a pipeline in the parent project for a merge request from a fork project: 1. In the merge request, go to the **Pipelines** tab. -1. Select **Run pipeline**. You must accept the warning, or the pipeline does not run. +1. Select **Run pipeline**. You must read and accept the warning, or the pipeline does not run. ## Available predefined variables diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 12ea3a70536..d200dde143c 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -24,7 +24,7 @@ requests, each waiting to be merged into the target branch. Many merge requests can be added to the train. Each merge request runs its own merged results pipeline, which includes the changes from all of the other merge requests in *front* of it on the train. -All the pipelines run in parallel, to save time. +All the pipelines run in parallel, to save time. The author of the internal merged result commit is always the user that initiated the merge. If the pipeline for a merge request fails, the breaking changes are not merged, and the target branch is unaffected. The merge request is removed from the train, and all pipelines behind it restart. @@ -73,6 +73,9 @@ To enable merge trains: - Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md). +Merge trains do not work with [Semi-linear history merge requests](../../user/project/merge_requests/methods/index.md#merge-commit-with-semi-linear-history) +or [fast-forward merge requests](../../user/project/merge_requests/methods/index.md#fast-forward-merge). + ## Enable merge trains To enable merge trains for your project: @@ -84,7 +87,6 @@ To enable merge trains for your project: 1. On the left sidebar, select **Settings > General**. 1. Expand **Merge requests**. 1. In the **Merge method** section, verify that **Merge commit** is selected. - You cannot use **Merge commit with semi-linear history** or **Fast-forward merge** with merge trains. 1. In the **Merge options** section, select **Enable merged results pipelines** (if not already selected) and **Enable merge trains**. 1. Select **Save changes**. diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 7df9ea3f72f..9661d2f5263 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -12,7 +12,8 @@ A *merged results pipeline* is a type of [merge request pipeline](merge_request_ GitLab creates an internal commit with the merged results, so the pipeline can run against it. This commit does not exist in either branch, -but you can view it in the pipeline details. +but you can view it in the pipeline details. The author of the internal commit is +always the user that created the merge request. The pipeline runs against the target branch as it exists at the moment you run the pipeline. Over time, while you're working in the source branch, the target branch might change. @@ -35,7 +36,7 @@ To use merged results pipelines: - Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md). - You must not be using [fast forward merges](../../user/project/merge_requests/fast_forward_merge.md). - [An issue exits](https://gitlab.com/gitlab-org/gitlab/-/issues/26996) to change this behavior. + [An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/26996) to change this behavior. ## Enable merged results pipelines diff --git a/doc/ci/pipelines/multi_project_pipelines.md b/doc/ci/pipelines/multi_project_pipelines.md index e6af9292fe1..20184635e4a 100644 --- a/doc/ci/pipelines/multi_project_pipelines.md +++ b/doc/ci/pipelines/multi_project_pipelines.md @@ -308,7 +308,10 @@ When you configure GitLab CI/CD for your project, you can visualize the stages o  -In the merge request, on the **Pipelines** tab, multi-project pipeline mini-graphs are displayed. -They expand and are shown adjacent to each other when hovering (or tapping on touchscreen devices). +## Retry or cancel multi-project pipelines - +If you have permission to trigger pipelines in the downstream project, you can +retry or cancel multi-project pipelines: + +- [In the main graph view](index.md#downstream-pipelines). +- From the downstream pipeline's details page. diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index a3ded24e8c9..3206345d757 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -216,3 +216,10 @@ multi-project pipelines: - [By using the `variable` keyword](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword). - [By using variable inheritance](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). + +## Retry or cancel child pipelines + +You can retry or cancel child pipelines: + +- [In the main graph view](index.md#downstream-pipelines). +- In the child pipeline's details page. diff --git a/doc/ci/pipelines/pipeline_artifacts.md b/doc/ci/pipelines/pipeline_artifacts.md index e9dd1b2a942..3a1367f4a8c 100644 --- a/doc/ci/pipelines/pipeline_artifacts.md +++ b/doc/ci/pipelines/pipeline_artifacts.md @@ -11,7 +11,7 @@ different to [job artifacts](job_artifacts.md) because they are not explicitly m `.gitlab-ci.yml` definitions. Pipeline artifacts are used by the [test coverage visualization feature](../../user/project/merge_requests/test_coverage_visualization.md) -to collect coverage information. It uses the [`artifacts: reports`](../yaml/index.md#artifactsreports) CI/CD keyword. +to collect coverage information. ## Storage diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index b6ea16ae224..2a95a4e1421 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -222,36 +222,49 @@ averaged. To add test coverage results to a merge request using the project's `.gitlab-ci.yml` file, provide a regular expression using the [`coverage`](../yaml/index.md#coverage) keyword. -Setting the regular expression this way takes precedence over project settings. +<!-- start_remove The following content will be removed on remove_date: '2023-08-22' --> -### Add test coverage results using project settings (DEPRECATED) +### Add test coverage results using project settings (removed) -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.9. Replaced by [`coverage` keyword](../yaml/index.md#coverage). +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.8. Replaced by [`coverage` keyword](../yaml/index.md#coverage). +> [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) -in GitLab 14.9, and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. +This feature is in its end-of-life process. It was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) +in GitLab 14.8. The feature is [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. -You can add test coverage results to merge requests using the Project's CI/CD settings: +To migrate from a project setting to the `coverage` keyword, add the [former project setting](#locate-former-project-setting) +to a CI/CD job. For example: -- Set using the GitLab UI: +- A Go test coverage project setting: `coverage: \d+.\d+% of statements`. +- A CI/CD job with `coverage` keyword setting: - 1. On the top bar, select **Menu > Projects** and find your project. - 1. On the left sidebar, select **Settings > CI/CD**. - 1. Expand **General pipelines**. - 1. In the **Test coverage parsing** field, enter a regular expression. Leave blank to disable this feature. + ```yaml + unit-test: + stage: test + coverage: '/coverage: \d+.\d+% of statements/' + script: + - go test -cover + ``` -- Set when [editing a project](../../api/projects.md#edit-project) or [creating a project](../../api/projects.md#create-project) - using the GitLab API with the `build_coverage_regex` attribute: +The `.gitlab-ci.yml` job [`coverage`](../yaml/index.md#coverage) keyword must be: - ```shell - curl --request PUT --header "PRIVATE-TOKEN: <your-token>" \ - --url 'https://gitlab.com/api/v4/projects/<your-project-ID>' \ - --data "build_coverage_regex=<your-regular-expression>" - ``` +- A regular expression starts and ends with the `/` character. +- Defined as single-quoted string. + +You can verify correct syntax using the [pipeline editor](../pipeline_editor/index.md). + +#### Locate former project setting + +To migrate from the project coverage setting to the `coverage` keyword, use the +regular expression displayed in the settings. Available in GitLab 14.10 and earlier: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **General pipelines**. + +The regular expression you need is in the **Test coverage parsing** field. -You can use <https://rubular.com> to test your regular expression. The regular expression returns the **last** -match found in the output. +<!-- end_remove --> ### Test coverage examples @@ -266,6 +279,7 @@ Use this regex for commonly used test tools. - gcovr (C/C++). Example: `^TOTAL.*\s+(\d+\%)$`. - `tap --coverage-report=text-summary` (NodeJS). Example: `^Statements\s*:\s*([^%]+)`. - `nyc npm test` (NodeJS). Example: `All files[^|]*\|[^|]*\s+([\d\.]+)`. +- `jest --ci --coverage` (NodeJS). Example: `All files[^|]*\|[^|]*\s+([\d\.]+)`. - excoveralls (Elixir). Example: `\[TOTAL\]\s+(\d+\.\d+)%`. - `mix test --cover` (Elixir). Example: `\d+.\d+\%\s+\|\s+Total`. - JaCoCo (Java/Kotlin). Example: `Total.*?([0-9]{1,3})%`. diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index fbdf226181b..e35a042a320 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -19,7 +19,7 @@ If you are migrating from another CI/CD tool, view this documentation: - [Migrate from CircleCI](../migration/circleci.md). - [Migrate from Jenkins](../migration/jenkins.md). -> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [First time GitLab & CI/CD](https://www.youtube.com/watch?v=kTNfi5z6Uvk&t=553s). This includes a quick introduction to GitLab, the first steps with CI/CD, building a Go project, running tests, using the CI/CD pipeline editor, detecting secrets and security vulnerabilities and offers more exercises for async practice. +> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [First time GitLab & CI/CD](https://www.youtube.com/watch?v=kTNfi5z6Uvk&t=553s). This includes a quick introduction to GitLab, the first steps with CI/CD, building a Go project, running tests, using the CI/CD pipeline editor, detecting secrets and security vulnerabilities and offers more exercises for asynchronous practice. > - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch [Intro to GitLab CI](https://www.youtube.com/watch?v=l5705U8s_nQ&t=358s). This workshop uses the Web IDE to quickly get going with building source code using CI/CD, and run unit tests. ## CI/CD process overview @@ -27,6 +27,8 @@ If you are migrating from another CI/CD tool, view this documentation: To use GitLab CI/CD: 1. [Ensure you have runners available](#ensure-you-have-runners-available) to run your jobs. + GitLab SaaS provides runners, so if you're using GitLab.com, you can skip this step. + If you don't have a runner, [install GitLab Runner](https://docs.gitlab.com/runner/install/) and [register a runner](https://docs.gitlab.com/runner/register/) for your instance, project, or group. 1. [Create a `.gitlab-ci.yml` file](#create-a-gitlab-ciyml-file) @@ -130,7 +132,7 @@ The pipeline starts when the commit is committed. ```yaml default: - image: ruby:2.7.4 + image: ruby:2.7.5 ``` This command tells the runner to use a Ruby image from Docker Hub diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index 9312f4a8850..e76c4621a0c 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -42,7 +42,7 @@ In this case, the `deploy` jobs across different pipelines could run concurrentl to the `production` environment. Running multiple deployment scripts to the same infrastructure could harm/confuse the instance and leave it in a corrupted state in the worst case. -In order to ensure that a `deploy` job runs once at a time, you can specify +To ensure that a `deploy` job runs once at a time, you can specify [`resource_group` keyword](../yaml/index.md#resource_group) to the concurrency sensitive job: ```yaml @@ -74,27 +74,27 @@ You can choose a process mode to strategically control the job concurrency for y The following modes are supported: - **Unordered:** This is the default process mode that limits the concurrency on running jobs. - It's the easiest option to use and useful when you don't care about the execution order + It's the easiest option to use when you don't care about the execution order of the jobs. It starts processing the jobs whenever a job ready to run. - **Oldest first:** This process mode limits the concurrency of the jobs. When a resource is free, it picks the first job from the list of upcoming jobs (`created`, `scheduled`, or `waiting_for_resource` state) that are sorted by pipeline ID in ascending order. - This mode is useful when you want to ensure that the jobs are executed from the oldest pipeline. - This is less efficient compared to the `unordered` mode in terms of the pipeline efficiency, + This mode is efficient when you want to ensure that the jobs are executed from the oldest pipeline. + It is less efficient compared to the `unordered` mode in terms of the pipeline efficiency, but safer for continuous deployments. - **Newest first:** This process mode limits the concurrency of the jobs. When a resource is free, it picks the first job from the list of upcoming jobs (`created`, `scheduled` or `waiting_for_resource` state) that are sorted by pipeline ID in descending order. - This mode is useful when you want to ensure that the jobs are executed from the newest pipeline and + This mode is efficient when you want to ensure that the jobs are executed from the newest pipeline and cancel all of the old deploy jobs with the [skip outdated deployment jobs](../environments/deployment_safety.md#skip-outdated-deployment-jobs) feature. This is the most efficient option in terms of the pipeline efficiency, but you must ensure that each deployment job is idempotent. ### Change the process mode -To change the process mode of a resource group, you need to use the API and +To change the process mode of a resource group, you must use the API and send a request to [edit an existing resource group](../../api/resource_groups.md#edit-an-existing-resource-group) by specifying the `process_mode`: @@ -145,7 +145,7 @@ Depending on the process mode of the resource group: You can define `resource_group` for downstream pipelines that are sensitive to concurrent executions. The [`trigger` keyword](../yaml/index.md#trigger) can trigger downstream pipelines and the -[`resource_group` keyword](../yaml/index.md#resource_group) can co-exist with it. `resource_group` is useful to control the +[`resource_group` keyword](../yaml/index.md#resource_group) can co-exist with it. `resource_group` is efficient to control the concurrency of deployment pipelines, while other jobs can continue to run concurrently. The following example has two pipeline configurations in a project. When a pipeline starts running, @@ -205,7 +205,7 @@ Read more how you can use GitLab for [safe deployments](../environments/deployme ### Avoid dead locks in pipeline configurations -Since [`oldest_first` process mode](#process-modes) enforces the jobs to be executed in a pipeline order, +Because [`oldest_first` process mode](#process-modes) enforces the jobs to be executed in a pipeline order, there is a case that it doesn't work well with the other CI features. For example, when you run [a child pipeline](../pipelines/parent_child_pipelines.md) @@ -229,10 +229,10 @@ deploy: In a parent pipeline, it runs the `test` job that subsequently runs a child pipeline, and the [`strategy: depend` option](../yaml/index.md#triggerstrategy) makes the `test` job wait until the child pipeline has finished. The parent pipeline runs the `deploy` job in the next stage, that requires a resource from the `production` resource group. -If the process mode is `oldest_first`, it executes the jobs from the oldest pipelines, meaning the `deploy` job is going to be executed next. +If the process mode is `oldest_first`, it executes the jobs from the oldest pipelines, meaning the `deploy` job is executed next. However, a child pipeline also requires a resource from the `production` resource group. -Since the child pipeline is newer than the parent pipeline, the child pipeline +Because the child pipeline is newer than the parent pipeline, the child pipeline waits until the `deploy` job is finished, something that will never happen. In this case, you should specify the `resource_group` keyword in the parent pipeline configuration instead: diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 6b14cea51d6..e7165339ea0 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -148,7 +148,7 @@ different places. ### Determine the IP address of a shared runner -To view the IP address of a shared runner you must have admin access to +To view the IP address of a shared runner you must have administrator access to the GitLab instance. To determine this: 1. On the top bar, select **Menu > Admin**. @@ -302,6 +302,9 @@ globally or for individual jobs: You can also use variables to configure how many times a runner [attempts certain stages of job execution](#job-stages-attempts). +When using the Kubernetes executor, you can use variables to +[override Kubernetes CPU and memory allocations for requests and limits](https://docs.gitlab.com/runner/executors/kubernetes.html#overwriting-container-resources). + ### Git strategy > - Introduced in GitLab 8.9 as an experimental feature. diff --git a/doc/ci/runners/saas/macos/codesigning.md b/doc/ci/runners/saas/macos/codesigning.md new file mode 100644 index 00000000000..4f8316faf17 --- /dev/null +++ b/doc/ci/runners/saas/macos/codesigning.md @@ -0,0 +1,121 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Code signing for SaaS runners on macOS + +> Introduced in GitLab 15.0. + +Before you can integrate GitLab with Apple services, install to a device, or deploy to the Apple App Store, you must [code sign](https://developer.apple.com/support/code-signing/) your application. + +To code sign an iOS project, you need the following files: + +- A certifcate issued by Apple. +- A provisioning profile. + +## Code signing iOS Projects with fastlane + +When you use SaaS runners on macOS, each job runs on a VM. Included in each VM is [fastlane](https://fastlane.tools/), +an open-source solution aimed at simplifying mobile app deployment. + +These steps outline the minimal setup required to use fastlane to code sign your application. Refer to the fastlane [getting started guide](https://docs.fastlane.tools/), [best practices for integrating with GitLab CI](https://docs.fastlane.tools/best-practices/continuous-integration/gitlab/) and the [fastlane code signing getting started guide](https://docs.fastlane.tools/codesigning/getting-started/) for installation instructions, and an overview of how to use fastlane to handle code signing. + +To use fastlane to code sign your application: + +1. At the root of your project repository, on your local development system, run this command: + + ```plaintext + fastlane match init + ``` + + This command creates the `fastlane` directory and adds two files: `Fastfile` and `Appfile`. + +1. Open `Appfile` and edit it to include your Apple ID and app ID. + + ```plaintext + app_identifier("APP IDENTIFIER") # The bundle identifier of your app + + apple_id("APPLE ID") # Your Apple email address + ``` + +1. Open `Fastfile`, which includes the fastlane build steps. + In the following snippet, the steps `get_certificates`, `get_provisioning_profile,match`, `gym`, and + `upload_to_testflight` are fastlane [actions](https://docs.fastlane.tools/actions/). + + ```plaintext + # This file contains the fastlane.tools configuration + # You can find the documentation at https://docs.fastlane.tools + + default_platform(:ios) + + platform :ios do + desc "Build the application" + lane :beta do + increment_build_number( + build_number: latest_testflight_build_number + 1, + xcodeproj: "${PROJECT_NAME}.xcodeproj" + ) + get_certificates + get_provisioning_profile + # match(type: "appstore",read_only: true) + gym + upload_to_testflight + end + end + ``` + +The example configuration also includes an optional `Gymfile`. This file stores configuration +parameters and is used by the fastlane [`gym`](https://docs.fastlane.tools/actions/gym/) action. + +## Using fastlane match + +To simplify the code signing process and implement the +[Code Signing Best Practices Guide](https://codesigning.guide/) recommendations, +use [fastlane match](https://docs.fastlane.tools/actions/match/). + +- Use one code signing identity shared across your team. +- Store the required certificates and provisioning profiles in a separate GitLab project repository. + +Match automatically syncs iOS and macOS keys and provisioning profiles across all team members with access to the GitLab project. Each team member with access to the project can use the credentials for code signing. + +To use fastlane match: + +1. Initialize match in the project repository: + + ```shell + bundle exec fastlane match init + ``` + +1. Select `git` as your storage node. +1. Enter the URL of the GitLab project you plan to use to store your code signing identities. +1. Optional. To create a new certificate and provisioning profile, run: + + ```shell + bundle exec fastlane match development + ``` + +For different code signing identities' storage options, and for a complete step-by-step guide for using match, +refer to the [match documentation](https://docs.fastlane.tools/actions/match/#usage). + +### Environment variables and authentication + +To complete the setup, you must configure environment variables to use with fastlane. The required variables are outlined in the [fastlane documentation](https://docs.fastlane.tools/best-practices/continuous-integration/#environment-variables-to-set). + +To support Apple's two factor authentication requirement, configure these variables: + +- `FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD` and +- `FASTLANE_SESSION` + +To authenticate fastlane with the App Store for the TestFlight upload, configure these variables: + +- `FASTLANE_USER` and +- `FASTLANE_PASSWORD` + +View the [fastlane authentication with Apple Services guide](https://docs.fastlane.tools/getting-started/ios/authentication/) for an overview of authentication options. + +## Related topics + +- [Apple Developer Support - Code Signing](https://developer.apple.com/support/code-signing/) +- [Code Signing Best Practice Guide](https://codesigning.guide/) diff --git a/doc/ci/runners/saas/macos/environment.md b/doc/ci/runners/saas/macos/environment.md index 4d209fe4cd6..edd897e4c23 100644 --- a/doc/ci/runners/saas/macos/environment.md +++ b/doc/ci/runners/saas/macos/environment.md @@ -9,12 +9,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w When you use SaaS runners on macOS: - Each of your jobs runs in a newly provisioned VM, which is dedicated to the specific job. -- The VM is active only for the duration of the job and immediately deleted. +- The VM is active only for the duration of the job and immediately deleted. This means that any changes that your job makes to the virtual machine will not be available to a subsequent job. +- The virtual machine where your job runs has `sudo` access with no password. + +NOTE: +Each time you run a job that requires tooling or dependencies not available in the base image, those items must be added to the newly provisioned build VM. That process will likely increase the total job duration. ## VM types -The virtual machine where your job runs has `sudo` access with no password. -For the [Beta](../../../../policy/alpha-beta-support.md#beta-features), there is only one available machine type, `gbc-macos-large`. +GitLab SaaS provides macOS build machines on Apple servers with Intel x86-64 processors. +The expectation is that virtual machines running on the Apple M1 chip will be available in the second half of 2022. + +At this time there is only one available machine type offered, `gbc-macos-large`. | Instance type | vCPUS | Memory (GB) | | --------- | --- | ------- | @@ -22,23 +28,34 @@ For the [Beta](../../../../policy/alpha-beta-support.md#beta-features), there is ## VM images +### Image update policy + +GitLab expects to release new images based on this cadence: + +macOS updates: + +- **For new OS versions:** When Apple releases a new macOS version to developers (like macOS `12`), GitLab will plan to release an image based on the OS within the next 30 business days. The image is considered `beta` and the contents of the image (including tool versions) are subject to change until the first patch release (`12.1`). The long-term name will not include `beta` (for example, `macos-12-xcode-13`), so customers are moved automatically out of beta over time. GitLab will try to minimize breaking changes between the first two minor versions but makes no guarantees. Tooling often gets critical bug fixes after the first public release of an OS version. + +- **After the first patch release (`12.1`):** + - The image moves to `maintenance` mode. The tools GitLab builds into the image with Homebrew and asdf are frozen. GitLab continues making Xcode updates, security updates, and any non-breaking changes deemed necessary. + - The image for the previous OS version (`11`) moves to `frozen` mode. GitLab then does only unavoidable changes: security updates, runner version upgrades, and setting the production password. + +Both macOS and Xcode follow a yearly release cadence. As time goes on, GitLab increments their versions synchronously (meaning we build macOS 11 with Xcode 12, macOS 12 with Xcode 13, and so on). + +### Available images + You can execute your build on one of the following images. You specify this image in your `.gitlab-ci.yml` file. Each image is running a specific version of macOS and Xcode. -| VM image | Included software | -|---------------------------|--------------------| -| macos-10.13-xcode-7 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | -| macos-10.13-xcode-8 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | -| macos-10.13-xcode-9 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | -| macos-10.14-xcode-10 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/mojave.yml> | -| macos-10.15-xcode-11 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/catalina.yml> | -| macos-11-xcode-12 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/big-sur.yml> | -| macos-11-xcode-13 | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/monterey.yml> - -### Image update policy - -- Support for new macOS versions is planned. -- Additional details on the support policy and image update release process are documented - [in this project](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/55bf59c8fa88712960afff2bf6ecc5daa879a8f5/docs/overview.md#os-images). +| VM image | Status | Included software | +|---------------------------|--------|--------------------| +| `macos-10.13-xcode-7` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | +| `macos-10.13-xcode-8` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | +| `macos-10.13-xcode-9` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | +| `macos-10.14-xcode-10` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/mojave.yml> | +| `macos-10.15-xcode-11` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/catalina.yml> | +| `macos-11-xcode-12` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/big-sur.yml> | +| `macos-12-xcode-13` | `maintenance` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/monterey.yml> | +| (none, awaiting macOS 13) | `beta` | | diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index bad9da960b2..0ab2de36f10 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -4,10 +4,9 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SaaS runners on macOS (beta) **(PREMIUM SAAS)** +# SaaS runners on macOS (Limited Availability) **(PREMIUM SAAS)** -SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta-features) -and shouldn't be relied upon for mission-critical production jobs. +SaaS runners on macOS are now in [Limited Availability](../../../policy/alpha-beta-support.md#beta-features) for approved open source programs and customers in Premium and Ultimate plans. SaaS runners on macOS provide an on-demand macOS build environment integrated with GitLab SaaS [CI/CD](../../../ci/index.md). @@ -15,11 +14,17 @@ Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS of all the capabilities of the GitLab single DevOps platform and not have to manage or operate a build environment. +CI/CD minutes used on GitLab SaaS macOS runners are included in your CI/CD minute consumption totals. CI jobs that run on macOS **will** consume CI minutes at a faster rate than CI jobs on the GitLab SaaS runners on Linux. + +Refer to the CI/CD minutes [cost factor](../../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the GitLab SaaS macOS runners. + ## Quickstart -To start using SaaS runners on macOS, you must submit an access request [issue](https://gitlab.com/gitlab-com/macos-buildcloud-runners-beta/-/issues/new?issuable_template=beta_access_request). After your -access has been granted and your build environment configured, you must configure your -`.gitlab-ci.yml` pipeline file: +To start using SaaS runners on macOS, you must be an active GitLab SaaS Premium or Ultimate customer. Participants in the GitLab Open Source program are also eligible to use the service. + +### Configuring your pipeline + +To start using the SaaS runners on macOS to run your CI jobs, you must configure your `.gitlab-ci.yml` file: 1. Add a `.gitlab-ci.yml` file to your project repository. 1. Specify the [image](macos/environment.md#vm-images) you want to use. @@ -27,7 +32,7 @@ access has been granted and your build environment configured, you must configur The runners automatically run your build. -## Example `.gitlab-ci.yml` file +### Example `.gitlab-ci.yml` file The following sample `.gitlab-ci.yml` file shows how to start using the SaaS runners on macOS: @@ -42,7 +47,7 @@ stages: - test before_script: - - echo "started by ${GITLAB_USER_NAME}" + - echo "started by ${GITLAB_USER_NAME}" build: extends: @@ -60,4 +65,13 @@ test: ``` NOTE: -During the Beta period, the architecture of this solution will change. Rather than the jobs running on a specific VM instance, they will run on an ephemeral VM instance that is created by an autoscaling instance, known as the Runner Manager. We will notify all Beta participants of any downtime required to do this work. +You can specify a different Xcode image to run a job. To do so, replace the value for the `image` keyword with the value of the [virtual machine image name](macos/environment.md#vm-images) from the list of available images. + +## SaaS runners on macOS service level objective + +In SaaS runners on macOS, the objective is to make 90% of CI jobs start executing in 120 seconds or less. The error rate should be less than 0.5%. + +## Known Limitations and Usage Constraints + +- If the VM image does not include the specific software version you need for your job, then the job execution time will increase as the required software needs to be fetched and installed. +- At this time, it is not possible to bring your own OS image. diff --git a/doc/ci/secure_files/index.md b/doc/ci/secure_files/index.md index 3cc38a8b74c..2bf360e69f1 100644 --- a/doc/ci/secure_files/index.md +++ b/doc/ci/secure_files/index.md @@ -47,7 +47,6 @@ The response returns all of the metadata for the file you just uploaded. For exa "name": "myfile.jks", "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac", "checksum_algorithm": "sha256", - "permissions": "read_only", "created_at": "2022-02-22T22:22:22.222Z" } ``` @@ -71,7 +70,6 @@ The response returns an array of all of the secure files in the project. For exa "name": "myfile.jks", "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac", "checksum_algorithm": "sha256", - "permissions": "read_only", "created_at": "2022-02-22T22:22:22.222Z" }] ``` diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index e6406818b4c..e876c6d7326 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -80,7 +80,7 @@ As mentioned before, this feature is designed to provide **network accessible** services. A database is the simplest example of such a service. The services feature is not designed to, and does not, add any software from the -defined `services` image(s) to the job's container. +defined `services` images to the job's container. For example, if you have the following `services` defined in your job, the `php`, `node` or `go` commands are **not** available for your script, and the job fails: diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 81cb924532c..1757543be5b 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -16,7 +16,7 @@ This guide also lists common issues and possible solutions. An early source of problems can be incorrect syntax. The pipeline shows a `yaml invalid` badge and does not start running if any syntax or formatting problems are found. -### Edit `gitlab-ci.yml` with the pipeline editor +### Edit `.gitlab-ci.yml` with the pipeline editor The [pipeline editor](pipeline_editor/index.md) is the recommended editing experience (rather than the single file editor or the Web IDE). It includes: @@ -331,6 +331,12 @@ busy_resources.pluck(:build_id) busy_resources.update_all(build_id: nil) ``` +### Job log slow to update + +When you visit the job log page for a running job, there could be a delay of up to +60 seconds before the log updates. The default refresh time is 60 seconds, but after +the log is viewed in the UI, the following log updates should occur every 3 seconds. + ## How to get help If you are unable to resolve pipeline issues, you can get help from: diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 30d53a340a9..90403351b11 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -168,7 +168,7 @@ To add or update variables in the project settings: - **Type**: [`File` or `Variable`](#cicd-variable-types). - **Environment scope**: Optional. `All`, or specific [environments](../environments/index.md). - **Protect variable** Optional. If selected, the variable is only available - in pipelines that run on protected branches or tags. + in pipelines that run on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). - **Mask variable** Optional. If selected, the variable's **Value** is masked in job logs. The variable fails to save if the value does not meet the [masking requirements](#mask-a-cicd-variable). @@ -367,18 +367,21 @@ a large value to the trace log has the potential to be [revealed](https://gitlab When using GitLab Runner 14.2, only the tail of the variable, characters beyond 4KiB in length, have the potential to be revealed. -### Protect a CI/CD variable +### Protected CI/CD variables -You can protect a project, group or instance CI/CD variable so it is only passed -to pipelines running on [protected branches](../../user/project/protected_branches.md) +You can configure a project, group, or instance CI/CD variable to be available +only to pipelines that run on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). -[Merge request pipelines](../pipelines/merge_request_pipelines.md) do not have access to protected variables. -An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/28002) regarding this limitation. +[Merged results pipelines](../pipelines/merge_request_pipelines.md#types-of-merge-request-pipelines), which run on a +temporary merge commit, not a branch or tag, do not have access to these variables. + +Pipelines that run directly on the merge request's source branch, with no added merge commit, can access +these variables if the source branch is a protected branch. -To protect a variable: +To mark a variable as protected: -1. Go to **Settings > CI/CD** in the project, group or instance admin area. +1. Go to **Settings > CI/CD** in the project, group or instance Admin Area. 1. Expand the **Variables** section. 1. Next to the variable you want to protect, select **Edit**. 1. Select the **Protect variable** checkbox. @@ -729,7 +732,7 @@ the variable can be available for. To learn more about scoping environments, see [Scoping environments with specs](../environments/index.md#scope-environments-with-specs). To learn more about ensuring CI/CD variables are only exposed in pipelines running from protected -branches or tags, see [Protect a CI/CD Variable](#protect-a-cicd-variable). +branches or tags, see [Protected CI/CD variables](#protected-cicd-variables). ## Deployment variables @@ -848,8 +851,8 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_SERVER_PROTOCOL=https ++ export CI_SERVER_NAME=GitLab ++ CI_SERVER_NAME=GitLab -++ export GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal -++ GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal +++ export GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal +++ GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal ++ export CI_PROJECT_ID=17893 ++ CI_PROJECT_ID=17893 ++ export CI_PROJECT_NAME=ci-debug-trace diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 3b504b02fae..f8f775a6df4 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -14,8 +14,6 @@ Some variables are only available with more recent versions of [GitLab Runner](h You can [output the values of all variables available for a job](index.md#list-all-environment-variables) with a `script` command. -There are also [Kubernetes-specific deployment variables (deprecated)](../../user/project/clusters/deploy_to_cluster.md#deployment-variables). - There are also a number of [variables you can use to configure runner behavior](../runners/configure_runners.md#configure-runner-behavior-with-variables) globally or for individual jobs. | Variable | GitLab | Runner | Description | diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index b30b13e1ec0..289e8b91104 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -80,32 +80,30 @@ GitLab can display the results of one or more reports in: - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). -## `artifacts:reports:cobertura` (DEPRECATED) +<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> + +## `artifacts:reports:cobertura` (removed) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.9. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab -14.9 and replaced with `artifacts:reports:coverage_report` in 14.10. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.9 and +[removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. Use `artifacts:reports:coverage_report` +instead. -The `cobertura` report collects [Cobertura coverage XML files](../../user/project/merge_requests/test_coverage_visualization.md). -The collected Cobertura coverage reports upload to GitLab as an artifact. - -GitLab can display the results of one or more reports in the merge request -[diff annotations](../../user/project/merge_requests/test_coverage_visualization.md). - -Cobertura was originally developed for Java, but there are many third-party ports for other languages such as -JavaScript, Python, and Ruby. +<!--- end_remove --> ## `artifacts:reports:coverage_report` -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344533) in GitLab 14.9. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344533) in GitLab 14.10. -Use `coverage_report` to collect coverage report in Cobertura format, similar to `artifacts:reports:cobertura`. +Use `coverage_report` to collect coverage report in Cobertura format. -NOTE: -`artifacts:reports:coverage_report` cannot be used at the same time with `artifacts:reports:cobertura`. +The `cobertura` report collects [Cobertura coverage XML files](../../user/project/merge_requests/test_coverage_visualization.md). + +Cobertura was originally developed for Java, but there are many third-party ports for other languages such as +JavaScript, Python, and Ruby. ```yaml artifacts: diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index e5a543e7130..838710834ba 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -344,7 +344,7 @@ You can only use the following rules with `include` (and only with [certain vari include: - local: builds.yml rules: - - if: '$INCLUDE_BUILDS == "true"' + - if: $INCLUDE_BUILDS == "true" - local: deploys.yml rules: - if: $CI_COMMIT_BRANCH == "main" diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index b335e8e8545..eb587db6d5f 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Keyword reference for the `.gitlab-ci.yml` file **(FREE)** +# `.gitlab-ci.yml` keyword reference **(FREE)** This document lists the configuration options for your GitLab `.gitlab-ci.yml` file. @@ -16,7 +16,7 @@ This document lists the configuration options for your GitLab `.gitlab-ci.yml` f When you are editing your `.gitlab-ci.yml` file, you can validate it with the [CI Lint](../lint.md) tool. -If you are editing this page, make sure you follow the [CI/CD YAML reference style guide](../../development/cicd/cicd_reference_documentation_guide.md). +If you are editing content on this page, follow the [instructions for documenting keywords](../../development/cicd/cicd_reference_documentation_guide.md). ## Keywords @@ -561,7 +561,7 @@ This same warning is displayed when: The default value for `allow_failure` is: - `true` for [manual jobs](../jobs/job_control.md#create-a-job-that-must-be-run-manually). -- `false` for manual jobs that also use [`rules`](#rules). +- `false` for jobs that use `when: manual` inside [`rules`](#rules). - `false` in all other cases. **Keyword type**: Job keyword. You can use it only as part of a job. @@ -1350,8 +1350,6 @@ In this example: **Additional details**: -- Coverage regular expressions set in `gitlab-ci.yml` take precedence over coverage regular expression set in the - [GitLab UI](../pipelines/settings.md#add-test-coverage-results-using-project-settings-deprecated). - If there is more than one matched line in the job output, the last line is used (the first result of reverse search). - If there are multiple matches in a single line, the last match is searched @@ -1561,7 +1559,7 @@ environment. #### `environment:action` -Use the `action` keyword to specify jobs that prepare, start, or stop environments. +Use the `action` keyword to specify how the job interacts with the environment. **Keyword type**: Job keyword. You can use it only as part of a job. @@ -1570,8 +1568,10 @@ Use the `action` keyword to specify jobs that prepare, start, or stop environmen | **Value** | **Description** | |:----------|:----------------| | `start` | Default value. Indicates that the job starts the environment. The deployment is created after the job starts. | -| `prepare` | Indicates that the job is only preparing the environment. It does not trigger deployments. [Read more about preparing environments](../environments/index.md#prepare-an-environment-without-creating-a-deployment). | +| `prepare` | Indicates that the job is only preparing the environment. It does not trigger deployments. [Read more about preparing environments](../environments/index.md#access-an-environment-for-preparation-or-verification-purposes). | | `stop` | Indicates that the job stops a deployment. For more detail, read [Stop an environment](../environments/index.md#stop-an-environment). | +| `verify` | Indicates that the job is only verifying the environment. It does not trigger deployments. [Read more about verifying environments](../environments/index.md#access-an-environment-for-preparation-or-verification-purposes). | +| `access` | Indicates that the job is only accessing the environment. It does not trigger deployments. [Read more about accessing environments](../environments/index.md#access-an-environment-for-preparation-or-verification-purposes). | **Example of `environment:action`**: @@ -3071,12 +3071,12 @@ to configure the job behavior, or with [`workflow`](#workflow) to configure the job: script: echo "Hello, Rules!" rules: - - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME != $CI_DEFAULT_BRANCH' + - if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME != $CI_DEFAULT_BRANCH when: never - - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/' + - if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ when: manual allow_failure: true - - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME' + - if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME ``` **Additional details**: @@ -3090,6 +3090,8 @@ job: - Unlike variables in [`script`](../variables/index.md#use-cicd-variables-in-job-scripts) sections, variables in rules expressions are always formatted as `$VARIABLE`. - You can use `rules:if` with `include` to [conditionally include other configuration files](includes.md#use-rules-with-include). +- In [GitLab 15.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/35438), + variables on the right side of `=~` and `!~` expressions are evaluated as regular expressions. **Related topics**: @@ -3122,7 +3124,7 @@ branch or merge request pipelines. docker build: script: docker build -t my-image:$CI_COMMIT_REF_SLUG . rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" changes: - Dockerfile when: manual @@ -3203,7 +3205,7 @@ job to run before continuing. job: script: echo "Hello, Rules!" rules: - - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH' + - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH when: manual allow_failure: true ``` @@ -3666,6 +3668,10 @@ trigger_job: earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. - In [GitLab 13.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/197140/), you can view which job triggered a downstream pipeline in the [pipeline graph](../pipelines/index.md#visualize-pipelines). +- [Manual pipeline variables](../variables/index.md#override-a-defined-cicd-variable) + and [scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule) + are not passed to downstream pipelines by default. Use [trigger:forward](#triggerforward) + to forward these variables to downstream pipelines. **Related topics**: @@ -3796,7 +3802,11 @@ deploy_review_job: - All YAML-defined variables are also set to any linked [Docker service containers](../services/index.md). - YAML-defined variables are meant for non-sensitive project configuration. Store sensitive information - in [protected variables](../variables/index.md#protect-a-cicd-variable) or [CI/CD secrets](../secrets/index.md). + in [protected variables](../variables/index.md#protected-cicd-variables) or [CI/CD secrets](../secrets/index.md). +- [Manual pipeline variables](../variables/index.md#override-a-defined-cicd-variable) + and [scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule) + are not passed to downstream pipelines by default. Use [trigger:forward](#triggerforward) + to forward these variables to downstream pipelines. **Related topics**: diff --git a/doc/ci/yaml/workflow.md b/doc/ci/yaml/workflow.md index b46d504edfa..a985db14d08 100644 --- a/doc/ci/yaml/workflow.md +++ b/doc/ci/yaml/workflow.md @@ -38,7 +38,7 @@ workflow: rules: - if: $CI_COMMIT_MESSAGE =~ /-draft$/ when: never - - if: '$CI_PIPELINE_SOURCE == "push"' + - if: $CI_PIPELINE_SOURCE == "push" ``` This example has strict rules, and pipelines do **not** run in any other case. @@ -50,9 +50,9 @@ All other pipeline types run. For example: ```yaml workflow: rules: - - if: '$CI_PIPELINE_SOURCE == "schedule"' + - if: $CI_PIPELINE_SOURCE == "schedule" when: never - - if: '$CI_PIPELINE_SOURCE == "push"' + - if: $CI_PIPELINE_SOURCE == "push" when: never - when: always ``` @@ -81,18 +81,20 @@ but does not run pipelines for any other case. It runs: ```yaml workflow: rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' - - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" + - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH ``` -If the pipeline is triggered by: +If GitLab attempts to trigger: -- A merge request, run a merge request pipeline. For example, a merge request pipeline +- A merge request pipeline, start the pipeline. For example, a merge request pipeline can be triggered by a push to a branch with an associated open merge request. -- A change to a branch, but a merge request is open for that branch, do not run a branch pipeline. -- A change to a branch, but without any open merge requests, run a branch pipeline. +- A branch pipeline, but a merge request is open for that branch, do not run the branch pipeline. + For example, a branch pipeline can be triggered by a change to a branch, an API call, + a scheduled pipeline, and so on. +- A branch pipeline, but there is no merge request open for the branch, run the branch pipeline. You can also add a rule to an existing `workflow` section to switch from branch pipelines to merge request pipelines when a merge request is created. |
