diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-09-19 23:18:09 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-09-19 23:18:09 +0000 |
| commit | 6ed4ec3e0b1340f96b7c043ef51d1b33bbe85fde (patch) | |
| tree | dc4d20fe6064752c0bd323187252c77e0a89144b /doc/ci | |
| parent | 9868dae7fc0655bd7ce4a6887d4e6d487690eeed (diff) | |
| download | gitlab-ce-6ed4ec3e0b1340f96b7c043ef51d1b33bbe85fde.tar.gz | |
Add latest changes from gitlab-org/gitlab@15-4-stable-eev15.4.0-rc42
Diffstat (limited to 'doc/ci')
64 files changed, 1580 insertions, 1058 deletions
diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 88d59b1f223..d34f44ea9ba 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -84,7 +84,8 @@ test-job: paths: - .yarn-cache/ script: - - bundle install --path=vendor + - bundle config set --local path 'vendor/ruby' + - bundle install - yarn install --cache-folder .yarn-cache - echo Run tests... ``` @@ -301,8 +302,7 @@ test: If your project uses [pip](https://pip.pypa.io/en/stable/) to install Python dependencies, the following example defines `cache` globally so that -all jobs inherit it. Python libraries are installed in a virtual environment under `venv/`. -pip's cache is defined under `.cache/pip/` and both are cached per-branch: +all jobs inherit it. pip's cache is defined under `.cache/pip/` and is cached per-branch: ```yaml # @@ -317,13 +317,9 @@ variables: # Pip's cache doesn't store the python packages # https://pip.pypa.io/en/stable/reference/pip_install/#caching -# -# If you want to also cache the installed packages, you have to install -# them in a virtualenv and cache it as well. cache: paths: - .cache/pip - - venv/ before_script: - python -V # Print out python version for debugging @@ -358,7 +354,8 @@ cache: before_script: - ruby -v # Print out ruby version for debugging - - bundle install -j $(nproc) --path vendor/ruby # Install dependencies into ./vendor/ruby + - bundle config set --local path 'vendor/ruby' # The location to install the specified gems to + - bundle install -j $(nproc) # Install dependencies into ./vendor/ruby rspec: script: @@ -384,14 +381,16 @@ cache: test_job: stage: test before_script: - - bundle install --without production --path vendor/ruby + - bundle config set --local path 'vendor/ruby' + - bundle install --without production script: - bundle exec rspec deploy_job: stage: production before_script: - - bundle install --without test --path vendor/ruby + - bundle config set --local path 'vendor/ruby' # The location to install the specified gems to + - bundle install --without test script: - bundle exec deploy ``` @@ -472,7 +471,7 @@ and should only be disabled in an environment where all users with Developer rol To use the same cache for all branches: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Clear the **Use separate caches for protected branches** checkbox. @@ -569,7 +568,7 @@ The next time the pipeline runs, the cache is stored in a different location. You can clear the cache in the GitLab UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. In the top right, select **Clear runner caches**. diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index c536ff59b84..cfe7be064fb 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -15,7 +15,8 @@ GitLab CI/CD can be used with Bitbucket Cloud by: To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab, create a project: - 1. On the top menu, select **Projects > Create new project**. + 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **Repository by URL**. 1. Fill in the fields with information from the repository in Bitbucket: diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index a928d315c6b..a48822fc214 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -34,7 +34,8 @@ repositories: `repo` and `admin:repo_hook` so that GitLab can access your project, update commit statuses, and create a web hook to notify GitLab of new commits. 1. In GitLab, create a project: - 1. On the top menu, select **Projects > Create new project**. + 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **GitHub**. 1. For **Personal access token**, paste the token. @@ -61,7 +62,8 @@ To manually enable GitLab CI/CD for your repository: 1. Enter a **Token description** and update the scope to allow `repo` so that GitLab can access your project and update commit statuses. 1. In GitLab, create a project: - 1. On the top menu, select **Projects > Create new project**. + 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository** and **Repository by URL**. 1. In the **Git repository URL** field, enter the HTTPS URL for your GitHub repository. If your project is private, use the personal access token you just created for authentication. diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index e3a8141ed88..7c0889a329a 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -9,8 +9,8 @@ type: index, howto >[Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4642) in GitLab 10.6. -GitLab CI/CD can be used with [GitHub](github_integration.md), [Bitbucket Cloud](bitbucket_integration.md), or any other -Git server. +GitLab CI/CD can be used with [GitHub](github_integration.md), [Bitbucket Cloud](bitbucket_integration.md), +or any other Git server, though there are some [limitations](#limitations). Instead of moving your entire project to GitLab, you can connect your external repository to get the benefits of GitLab CI/CD. @@ -24,7 +24,8 @@ snippets disabled. These features To connect to an external repository: -1. On the top bar, select **Menu > Projects > Create new project**. +1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. +1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **GitHub** or **Repository by URL**. 1. Complete the fields. @@ -86,7 +87,11 @@ The variable names are prefixed with `CI_EXTERNAL_PULL_REQUEST_`. ### Limitations -This feature currently does not support Pull Requests from fork repositories. Any Pull Requests from fork repositories are ignored. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). +This feature does not support: + +- The [manual connection method](github_integration.md#connect-manually) required for GitHub Enterprise. + If the integration is connected manually, external pull requests [do not trigger pipelines](https://gitlab.com/gitlab-org/gitlab/-/issues/323336#note_884820753). +- Pull requests from fork repositories. [Pull Requests from fork repositories are ignored](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). Given that GitLab creates 2 pipelines, if changes are pushed to a remote branch that references an open Pull Request, both contribute to the status of the Pull Request diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md index aea7b492d4e..2d1c3f927e2 100644 --- a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -79,7 +79,7 @@ and [Container Registry](../../../user/packages/container_registry/index.md).  -1. Visit **Packages & Registries > Container Registry**. Make sure the application image has been +1. Visit **Packages and registries > Container Registry**. Make sure the application image has been pushed.  diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 5df396e796e..82d914f0a6a 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -50,6 +50,7 @@ deploy: script: - aws s3 ... - aws create-deployment ... + environment: production ``` GitLab provides a Docker image that includes the AWS CLI: @@ -215,3 +216,14 @@ To deploy to EC2, complete the following steps. - Your built application is pushed to your S3 bucket then and deployed to your EC2 instance, based on the related JSON object's content. The deployment job finishes when the deployment to EC2 is done or has failed. + +## Troubleshooting + +### Error `'ascii' codec can't encode character '\uxxxx'` + +This error can occur when the response from the `aws-cli` utility used by the Cloud Deploy images contains a Unicode character. The Cloud Deploy images we provide do not have a defined locale and default to using ASCII. To resolve this error, add the following CI/CD variable: + +```yaml +variables: + LANG: "UTF-8" +``` diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md new file mode 100644 index 00000000000..901b36afde6 --- /dev/null +++ b/doc/ci/cloud_services/azure/index.md @@ -0,0 +1,157 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Configure OpenID Connect in Azure to retrieve temporary credentials + +This tutorial demonstrates how to use a JSON web token (JWT) in a GitLab CI/CD job +to retrieve temporary credentials from Azure without needing to store secrets. + +To get started, configure OpenID Connect (OIDC) for identity federation between GitLab and Azure. +For more information on using OIDC with GitLab, read [Connect to cloud services](../index.md). + +Prerequisites: + +- Access to an existing Azure Subscription with `Owner` access level. +- Access to the corresponding Azure Active Directory Tenant with at least the `Application Developer` access level. +- A local installation of the [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli). + Alternatively, you can follow all the steps below with the [Azure Cloud Shell](https://shell.azure.com/). +- A GitLab project. + +To complete this tutorial: + +1. [Create Azure AD application and service principal](#create-azure-ad-application-and-service-principal). +1. [Create Azure AD federated identity credentials](#create-azure-ad-federated-identity-credentials). +1. [Grant permissions for the service principal](#grant-permissions-for-the-service-principal). +1. [Retrieve a temporary credential](#retrieve-a-temporary-credential). + +For more information, review Azure's documentation on [Workload identity federation](https://docs.microsoft.com/azure/active-directory/develop/workload-identity-federation). + +## Create Azure AD application and service principal + +To create an [Azure AD application](https://docs.microsoft.com/cli/azure/ad/app?view=azure-cli-latest#az-ad-app-create) +and service principal: + +1. In the Azure CLI, create the AD application: + + ```shell + appId=$(az ad app create --display-name gitlab-oidc --query appId -otsv) + ``` + + Save the `appId` (Application client ID) output, as you need it later + to configure your GitLab CI/CD pipeline. + +1. Create a corresponding [Service Principal](https://docs.microsoft.com/cli/azure/ad/sp?view=azure-cli-latest#az-ad-sp-create): + + ```shell + az ad sp create --id $appId --query appId -otsv + ``` + +Instead of the Azure CLI, you can [use the Azure Portal to create these resources](https://docs.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal). + +## Create Azure AD federated identity credentials + +To create the federated identity credentials for the above Azure AD application: + +```shell +objectId=$(az ad app show --id $appId --query id -otsv) + +cat <<EOF > body.json +{ + "name": "gitlab-federated-identity", + "issuer": "https://gitlab.example.com", + "subject": "project_path:<mygroup>/<myproject>:ref_type:branch:ref:<branch>", + "description": "GitLab service account federated identity", + "audiences": [ + "https://gitlab.example.com" + ] +} +EOF + +az rest --method POST --uri "https://graph.microsoft.com/beta/applications/$objectId/federatedIdentityCredentials" --body @body.json +``` + +For issues related to the values of `issuer`, `subject` or `audiences`, see the +[troubleshooting](#troubleshooting) details. + +Optionally, you can now verify the Azure AD application and the Azure AD federated +identity credentials from the Azure Portal: + +1. Open the [Azure Active Directory App Registration](https://portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps) + view and select the appropriate app registration by searching for the display name `gitlab-oidc`. +1. On the overview page you can verify details like the `Application (client) ID`, + `Object ID`, and `Tenant ID`. +1. Under `Certificates & secrets`, go to `Federated credentials` to review your + Azure AD federated identity credentials. + +## Grant permissions for the service principal + +After you create the credentials, use [`role assignment`](https://docs.microsoft.com/cli/azure/role/assignment?view=azure-cli-latest#az-role-assignment-create) +to grant permissions to the above service principal to access to Azure resources: + +```shell +az role assignment create --assignee $appId --role Reader --scope /subscriptions/<subscription-id> +``` + +You can find your subscription ID in: + +- The [Azure Portal](https://docs.microsoft.com/azure/azure-portal/get-subscription-tenant-id#find-your-azure-subscription). +- The [Azure CLI](https://docs.microsoft.com/cli/azure/manage-azure-subscriptions-azure-cli#get-the-active-subscription). + +## Retrieve a temporary credential + +After you configure the Azure AD application and federated identity credentials, +the CI/CD job can retrieve a temporary credential by using the [Azure CLI](https://docs.microsoft.com/cli/azure/reference-index?view=azure-cli-latest#az-login): + +```yaml +default: + image: mcr.microsoft.com/azure-cli:latest + +variables: + AZURE_CLIENT_ID: "<client-id>" + AZURE_TENANT_ID: "<tenant-id>" + +auth: + script: + - az login --service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID --federated-token $CI_JOB_JWT_V2 + - az account show +``` + +The CI/CD variables are: + +- `AZURE_CLIENT_ID`: The [application client ID you saved earlier](#create-azure-ad-application-and-service-principal). +- `AZURE_TENANT_ID`: Your Azure Active Directory. You can + [find it by using the Azure CLI or Azure Portal](https://docs.microsoft.com/azure/active-directory/fundamentals/active-directory-how-to-find-tenant). +- `CI_JOB_JWT_V2`: The JSON web token is a [predefined CI/CD variable](../../variables/predefined_variables.md). + +## Troubleshooting + +### "No matching federated identity record found" + +If you receive the error `ERROR: AADSTS70021: No matching federated identity record found for presented assertion.` +you should verify: + +- The `Issuer` defined in the Azure AD federated identity credentials, for example + `https://gitlab.com` or your own GitLab URL. +- The `Subject identifier` defined in the Azure AD federated identity credentials, + for example `project_path:<mygroup>/<myproject>:ref_type:branch:ref:<branch>`. + - For the `gitlab-group/gitlab-project` project and `main` branch it would be: + `project_path:gitlab-group/gitlab-project:ref_type:branch:ref:main`. + - The correct values of `mygroup` and `myproject` can be retrieved by checking the URL + when accessing your GitLab project or by selecting the **Clone** option in the project. +- The `Audience` defined in the Azure AD federated identity credentials, for example `https://gitlab.com` + or your own GitLab URL. + +You can review these settings, as well as your `AZURE_CLIENT_ID` and `AZURE_TENANT_ID` +CI/CD variables, from the Azure Portal: + +1. Open the [Azure Active Directory App Registration](https://portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/RegisteredApps) + view and select the appropriate app registration by searching for the display name `gitlab-oidc`. +1. On the overview page you can verify details like the `Application (client) ID`, + `Object ID`, and `Tenant ID`. +1. Under `Certificates & secrets`, go to `Federated credentials` to review your + Azure AD federated identity credentials. + +Review [Connect to cloud services](../index.md) for further details. diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index 1493a930099..93fedb0ffca 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -16,7 +16,7 @@ GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) t - Account on GitLab. - Access to a cloud provider that supports OIDC to configure authorization and create roles. -The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, GCP, and Vault. +The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/index.md). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, Azure, GCP, and Vault. NOTE: Configuring OIDC enables JWT token access to the target environments for all pipelines. @@ -25,8 +25,9 @@ review for the pipeline, focusing on the additional access. You can use the [sof as a starting point, and for more information about supply chain attacks, see [How a DevOps Platform helps protect against supply chain attacks](https://about.gitlab.com/blog/2021/04/28/devops-platform-supply-chain-attacks/). -WARNING: -The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../policy/alpha-beta-support.md#alpha-features) and is not yet suitable for production use. +The `CI_JOB_JWT_V2` variable is available for testing, but the full feature is planned +to be generally available when [issue 360657](https://gitlab.com/gitlab-org/gitlab/-/issues/360657) +is complete. ## Use cases @@ -38,7 +39,7 @@ The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../policy/alpha-b ## How it works -Each job has a JSON web token (JWT) provided as a CI/CD [predefined variable](../variables/predefined_variables.md) named `CI_JOB_JWT` or `CI_JOB_JWT_V2`. This JWT can be used to authenticate with the OIDC-supported cloud provider such as AWS, GCP, or Vault. +Each job has a JSON web token (JWT) provided as a CI/CD [predefined variable](../variables/predefined_variables.md) named `CI_JOB_JWT` or `CI_JOB_JWT_V2`. This JWT can be used to authenticate with the OIDC-supported cloud provider such as AWS, Azure, GCP, or Vault. The following fields are included in the JWT: @@ -112,7 +113,7 @@ sequenceDiagram ``` -1. Create an OIDC identity provider in the cloud (for example, AWS, GCP, Vault). +1. Create an OIDC identity provider in the cloud (for example, AWS, Azure, GCP, Vault). 1. Create a conditional role in the cloud service that filters to a group, project, branch, or tag. 1. The CI/CD job includes a predefined variable `CI_JOB_JWT_V2` that is a JWT token. You can use this token for authorization with your cloud API. 1. The cloud verifies the token, validates the conditional role from the payload, and returns a temporary credential. @@ -138,4 +139,5 @@ To configure the trust between GitLab and OIDC, you must create a conditional ro To connect with your cloud provider, see the following tutorials: - [Configure OpenID Connect in AWS](aws/index.md) +- [Configure OpenID Connect in Azure](azure/index.md) - [Configure OpenID Connect in Google Cloud](google_cloud/index.md) diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index ea4ad25637b..4c9cb2923d7 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -345,7 +345,7 @@ not without its own challenges: root file system, you can use the job's working directory as a mount point for child containers. For example, if you have files you want to share with a child container, you might create a subdirectory under `/builds/$CI_PROJECT_PATH` - and use it as your mount point. For a more detailed explanation, view + and use it as your mount point. For a more detailed explanation, view [issue #41227](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41227). ```yaml @@ -406,7 +406,7 @@ sudo gitlab-runner register -n \ ##### Enable registry mirror for `docker:dind` service When the Docker daemon starts inside of the service container, it uses -the default configuration. You may want to configure a +the default configuration. You may want to configure a [registry mirror](https://docs.docker.com/registry/recipes/mirror/) for performance improvements and to ensure you don't reach Docker Hub rate limits. diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index fdd8b6d38b8..f985b02be33 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -406,7 +406,7 @@ image. This image is private and requires you to log in into a private container To configure access for `<aws_account_id>.dkr.ecr.<region>.amazonaws.com`, follow these steps: -1. Make sure `docker-credential-ecr-login` is available in the GitLab Runner `$PATH`. +1. Make sure [`docker-credential-ecr-login`](https://github.com/awslabs/amazon-ecr-credential-helper) is available in the GitLab Runner `$PATH`. 1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials). Make sure that GitLab Runner can access the credentials. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 712fd7b45d6..ee8d73dd113 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -39,8 +39,6 @@ few important details: GitLab CI/CD. - The entrypoint needs to be [overridden](using_docker_images.md#override-the-entrypoint-of-an-image), otherwise the build script doesn't run. -- A Docker `config.json` file needs to be created with the authentication - information for the desired container registry. In the following example, kaniko is used to: @@ -50,7 +48,7 @@ In the following example, kaniko is used to: The job runs only when a tag is pushed. A `config.json` file is created under `/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the [predefined CI/CD variables](../variables/index.md#predefined-cicd-variables) -GitLab CI/CD provides. +GitLab CI/CD provides. These are automatically read by the Kaniko tool. In the last step, kaniko uses the `Dockerfile` under the root directory of the project, builds the Docker image and pushes it to the @@ -60,13 +58,10 @@ project's Container Registry while tagging it with the Git tag: build: stage: build image: - name: gcr.io/kaniko-project/executor:debug + name: gcr.io/kaniko-project/executor:v1.9.0-debug entrypoint: [""] script: - - mkdir -p /kaniko/.docker - - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 -w 0)\"}}}" > /kaniko/.docker/config.json - - >- - /kaniko/executor + - /kaniko/executor --context "${CI_PROJECT_DIR}" --dockerfile "${CI_PROJECT_DIR}/Dockerfile" --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}" @@ -86,7 +81,6 @@ you must add the corresponding CI/CD variables for authentication to the `config If you use a custom GitLab Runner behind an http(s) proxy, kaniko needs to be set up accordingly. This means: -- Adding the proxy to `/kaniko/.docker/config.json` - Passing the `http_proxy` environment variables as build arguments so the Dockerfile instructions can use the proxy when building the image. @@ -95,25 +89,20 @@ The previous example can be extended as follows: ```yaml build: stage: build + variables: + http_proxy: <your-proxy> + https_proxy: <your-proxy> + no_proxy: <your-no-proxy> image: - name: gcr.io/kaniko-project/executor:debug + name: gcr.io/kaniko-project/executor:v1.9.0-debug entrypoint: [""] script: - - mkdir -p /kaniko/.docker - - |- - KANIKOPROXYBUILDARGS="" - KANIKOCFG="\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}" - if [ "x${http_proxy}" != "x" -o "x${https_proxy}" != "x" ]; then - KANIKOCFG="${KANIKOCFG}, \"proxies\": { \"default\": { \"httpProxy\": \"${http_proxy}\", \"httpsProxy\": \"${https_proxy}\", \"noProxy\": \"${no_proxy}\"}}" - KANIKOPROXYBUILDARGS="--build-arg http_proxy=${http_proxy} --build-arg https_proxy=${https_proxy} --build-arg no_proxy=${no_proxy}" - fi - KANIKOCFG="{ ${KANIKOCFG} }" - echo "${KANIKOCFG}" > /kaniko/.docker/config.json - - >- - /kaniko/executor + - /kaniko/executor --context "${CI_PROJECT_DIR}" + --build-arg http_proxy=$http_proxy + --build-arg https_proxy=$https_proxy + --build-arg no_proxy=$no_proxy --dockerfile "${CI_PROJECT_DIR}/Dockerfile" - "${KANIKOPROXYBUILDARGS}" --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}" rules: - if: $CI_COMMIT_TAG @@ -142,8 +131,6 @@ store: ```yaml before_script: - - mkdir -p /kaniko/.docker - - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json - | echo "-----BEGIN CERTIFICATE----- ... diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index dac52a4540e..bac06972c7b 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -30,7 +30,7 @@ When you disable GitLab CI/CD: To disable GitLab CI/CD in your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. In the **Repository** section, turn off **CI/CD**. @@ -40,7 +40,7 @@ To disable GitLab CI/CD in your project: To enable GitLab CI/CD in your project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. In the **Repository** section, turn on **CI/CD**. diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index 23fa5d45196..26461d9827f 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -65,9 +65,6 @@ co-exist and multiple approval rules takes the precedence over the unified appro #### Unified approval setting -NOTE: -At this time, it is not possible to require approvals for an existing protected environment. The workaround is to unprotect the environment and configure approvals when re-protecting the environment. - There are two ways to configure approvals for a protected environment: 1. Using the [UI](protected_environments.md#protecting-environments) @@ -137,7 +134,7 @@ Prerequisites: To approve or reject a deployment to a protected environment using the UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment's name. 1. In the deployment's row, select **Approval options** (**{thumb-up}**). @@ -169,7 +166,7 @@ curl --data "status=approved&comment=Looks good to me" \ ### Using the UI -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment being deployed to. 1. Look for the `blocked` label. diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 5b2e2045bdc..90efc7ba9ef 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Deployment safety **(FREE)** -[Deployment jobs](../jobs/#deployment-jobs) are a specific kind of CI/CD +[Deployment jobs](../jobs/index.md#deployment-jobs) are a specific kind of CI/CD job. They can be more sensitive than other jobs in a pipeline, and might need to be treated with extra care. GitLab has several features that help maintain deployment security and stability. @@ -66,7 +66,7 @@ For more information, see [Resource Group documentation](../resource_groups/inde ## Skip outdated deployment jobs The effective execution order of pipeline jobs can vary from run to run, which -could cause undesired behavior. For example, a [deployment job](../jobs/#deployment-jobs) +could cause undesired behavior. For example, a [deployment job](../jobs/index.md#deployment-jobs) in a newer pipeline could finish before a deployment job in an older pipeline. This creates a race condition where the older deployment finishes later, overwriting the "newer" deployment. @@ -131,7 +131,7 @@ All users with the Maintainer role for the project have access to production sec that can deploy to a production environment, you can create a separate project and configure a new permission model that isolates the CD permissions from the original project and prevents the original users with the Maintainer role for the project from accessing the production secret and CD configuration. You can -connect the CD project to your development projects by using [multi-project pipelines](../pipelines/multi_project_pipelines.md). +connect the CD project to your development projects by using [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines). ## Protect `.gitlab-ci.yml` from change diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index c3c1e7868fd..11e9fe90e25 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -21,7 +21,7 @@ diagnose if there is a block at a particular point, or if there's a more systemic problem you need to investigate. You can access the dashboard on the top bar by selecting -**Menu > Environments**. +**Main menu > Environments**.  diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 7747f5e9b78..c34a978709b 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -30,7 +30,7 @@ Prerequisites: To view a list of environments and deployments: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. The environments are displayed. @@ -57,7 +57,7 @@ You can create an environment and deployment in the UI or in your `.gitlab-ci.ym In the UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select **New environment**. 1. Enter a name and external URL. @@ -364,7 +364,7 @@ If there is a problem with a deployment, you can retry it or roll it back. To retry or rollback a deployment: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment. 1. To the right of the deployment name: @@ -642,7 +642,7 @@ When the environment is stopped, the system runs `on_stop` actions You can view a deployment's expiration date in the GitLab UI. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the name of the deployment. @@ -652,7 +652,7 @@ In the top left, next to the environment name, the expiration date is displayed. You can manually override a deployment's expiration date. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the deployment name. 1. On the top right, select the thumbtack (**{thumbtack}**). @@ -670,7 +670,7 @@ You can delete [stopped environments](#stop-an-environment) in the GitLab UI or To delete a stopped environment in the GitLab UI: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the **Stopped** tab. 1. Next to the environment you want to delete, select **Delete environment**. @@ -785,7 +785,7 @@ Limitations of GitLab Auto Rollback: GitLab Auto Rollback is turned off by default. To turn it on: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Automatic deployment rollbacks**. 1. Select the checkbox for **Enable automatic rollbacks**. @@ -945,7 +945,7 @@ the `review/feature-1` spec takes precedence over `review/*` and `*` specs. Renaming an environment through the UI is not possible. Instead, you need to delete the old environment and create a new one: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Find the environment and stop it. 1. Delete the environment. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 17eccc38747..e63777dc0e0 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -24,9 +24,13 @@ Maintainer role. ## Protecting environments +Prerequisites: + +- When granting the **Allowed to deploy** permission to a group or sub-group, the user configuring the protected environment must be a **direct member** of the group or sub-group to be added. Otherwise, the group or sub-group will not show up in the dropdown. For more information see [issue #345140](https://gitlab.com/gitlab-org/gitlab/-/issues/345140). + To protect an environment: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Protected environments**. 1. From the **Environment** list, select the environment you want to protect. @@ -238,7 +242,7 @@ To protect a group-level environment, make sure your environments have the corre > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325249) in GitLab 15.1. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Protected environments**. 1. From the **Environment** list, select the [deployment tier of environments](index.md#deployment-tier-of-environments) you want to protect. diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index b083dbb8177..b097f1fac76 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -47,6 +47,7 @@ staging: script: - gem install dpl - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY + environment: staging ``` In the above example we use Dpl to deploy `my-app-staging` to Heroku server with API key stored in `HEROKU_STAGING_API_KEY` secure variable. @@ -70,6 +71,7 @@ staging: - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY only: - main + environment: staging ``` The first line `apt-get update -yq` updates the list of available packages, @@ -93,6 +95,7 @@ staging: - dpl --provider=heroku --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY only: - main + environment: staging production: stage: deploy @@ -101,6 +104,7 @@ production: - dpl --provider=heroku --app=my-app-production --api_key=$HEROKU_PRODUCTION_API_KEY only: - tags + environment: production ``` We created two deploy jobs that are executed on different events: @@ -117,7 +121,7 @@ We also use two secure variables: To store API keys as secure variables: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 4d247a4ff74..5ff99755242 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -228,6 +228,7 @@ deploy_terraform: script: # Your Review App deployment scripts - for a working example please check https://gitlab.com/Flockademic/Flockademic/blob/5a45f1c2412e93810fab50e2dab8949e2d0633c7/.gitlab-ci.yml#L315 - echo + environment: production e2e:firefox: stage: confidence-check services: diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 666c4d444d8..9b9f87fffbb 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -176,7 +176,7 @@ Using phpenv also allows to easily configure the PHP environment with: phpenv config-add my_config.ini ``` -*__Important note:__ It seems `phpenv/phpenv` +**Important note:** It seems `phpenv/phpenv` [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork at [`madumlao/phpenv`](https://github.com/madumlao/phpenv) that tries to bring the project back to life. [`CHH/phpenv`](https://github.com/CHH/phpenv) also diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 2e9e2dd33bf..eaa7d8ebcfa 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -95,7 +95,7 @@ As part of publishing a package, semantic-release increases the version number i 1. Under **select scopes**, select the **api** checkbox. 1. Select **Create project access token**. 1. Copy the value. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. 1. Select **Add variable**. diff --git a/doc/ci/index.md b/doc/ci/index.md index 80752830b85..be7088ab153 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -146,36 +146,30 @@ See also the [Why CI/CD?](https://docs.google.com/presentation/d/1OGgk2Tcxbpl7DJ As GitLab CI/CD has evolved, certain breaking changes have been necessary. -#### 14.0 - -- No breaking changes. - -#### 13.0 - -- [Remove Backported `os.Expand`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4915). -- [Remove Fedora 29 package support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/16158). -- [Remove macOS 32-bit support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25466). -- [Removed `debug/jobs/list?v=1` endpoint](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6361). -- [Remove support for array of strings when defining services for Docker executor](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4922). -- [Remove `--docker-services` flag on register command](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6404). -- [Remove legacy build directory caching](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4180). -- [Remove `FF_USE_LEGACY_VOLUMES_MOUNTING_ORDER` feature flag](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6581). -- [Remove support for Windows Server 1803](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6553). - -#### 12.0 - -- [Use `refspec` to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4069). -- [Old cache configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4070). -- [Old metrics server configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4072). -- [Remove `FF_K8S_USE_ENTRYPOINT_OVER_COMMAND`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4073). -- [Remove Linux distributions that reach EOL](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1130). -- [Update command line API for helper images](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4013). -- [Remove old `git clean` flow](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4175). - -#### 11.0 - -- No breaking changes. - -#### 10.0 - -- No breaking changes. +For GitLab 15.0 and later, all breaking changes are documented on the following pages: + +- [Deprecations](../update/deprecations.md) +- [Removals](../update/removals.md) + +The breaking changes for [GitLab Runner](https://docs.gitlab.com/runner/) in earlier +major version releases are: + +- 14.0: No breaking changes. +- 13.0: + - [Remove Backported `os.Expand`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4915). + - [Remove Fedora 29 package support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/16158). + - [Remove macOS 32-bit support](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25466). + - [Removed `debug/jobs/list?v=1` endpoint](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6361). + - [Remove support for array of strings when defining services for Docker executor](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4922). + - [Remove `--docker-services` flag on register command](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6404). + - [Remove legacy build directory caching](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4180). + - [Remove `FF_USE_LEGACY_VOLUMES_MOUNTING_ORDER` feature flag](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6581). + - [Remove support for Windows Server 1803](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6553). +- 12.0: + - [Use `refspec` to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4069). + - [Old cache configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4070). + - [Old metrics server configuration](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4072). + - [Remove `FF_K8S_USE_ENTRYPOINT_OVER_COMMAND`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4073). + - [Remove Linux distributions that reach EOL](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1130). + - [Update command line API for helper images](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4013). + - [Remove old `git clean` flow](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4175). diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index e6a9f1fa646..03c905184cf 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -18,7 +18,7 @@ taken to protect the users. NOTE: [Shared runners on GitLab.com](../runners/index.md) do not -provide an interactive web terminal. Follow +provide an interactive web terminal. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/24674) for progress on adding support. For groups and projects hosted on GitLab.com, interactive web terminals are available when using your own group or project runner. @@ -27,7 +27,7 @@ terminals are available when using your own group or project runner. Two things need to be configured for the interactive web terminal to work: -- The runner needs to have +- The runner needs to have [`[session_server]` configured properly](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) - If you are using a reverse proxy with your GitLab instance, web terminals need to be [enabled](../../administration/integration/terminal.md#enabling-and-disabling-terminal-support) @@ -54,7 +54,7 @@ Not all executors are NOTE: The `docker` executor does not keep running after the build script is finished. At that point, the terminal automatically -disconnects and does not wait for the user to finish. Please follow +disconnects and does not wait for the user to finish. Please follow [this issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3605) for updates on improving this behavior. diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index 93f22da648a..812683ef2c1 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -24,12 +24,6 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Releases](../../api/releases/index.md) and [Release links](../../api/releases/links.md). - [Terraform plan](../../user/infrastructure/index.md). -NOTE: -There's an open issue, -[GitLab-#333444](https://gitlab.com/gitlab-org/gitlab/-/issues/333444), -which prevents you from using a job token with internal projects. This bug only impacts self-managed -GitLab instances. - The token has the same permissions to access the API as the user that caused the job to run. A user can cause a job to run by pushing a commit, triggering a manual job, being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to @@ -95,7 +89,7 @@ The job token scope is only for controlling access to private projects. ### Configure the job token scope limit -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Token Access**. 1. Toggle **Limit CI_JOB_TOKEN access** to enabled. @@ -121,6 +115,7 @@ trigger_pipeline: - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" rules: - if: $CI_COMMIT_TAG + environment: production ``` If you use the `CI_PIPELINE_SOURCE` [predefined CI/CD variable](../variables/predefined_variables.md) diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index fd6afb1a0ad..806837e3dc8 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -47,7 +47,7 @@ Clicking an individual job shows you its job log, and allows you to: To view the full list of jobs that ran in a project: -1. On the top bar, select **Menu > Projects** and find the project. +1. On the top bar, select **Main menu > Projects** and find the project. 1. On the left sidebar, select **CI/CD > Jobs**. You can filter the list by [job status](#the-order-of-jobs-in-a-pipeline). diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 0d5357e63ad..5a94c2e9bbc 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -243,8 +243,8 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `external` | When you use CI services other than GitLab. | | `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | | `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/merged_results_pipelines.md), and [merge trains](../pipelines/merge_trains.md). | -| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | -| `pipeline` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. | +| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | +| `pipeline` | For [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines) created by [using the API with `CI_JOB_TOKEN`](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. | | `push` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | | `trigger` | For pipelines created by using a [trigger token](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines). | @@ -578,6 +578,8 @@ To run a manual job, you must have permission to merge to the assigned branch: or deployment view. 1. Next to the manual job, select **Play** (**{play}**). +You can also [add custom CI/CD variables when running a manual job](index.md#specifying-variables-when-running-manual-jobs). + ### Protect manual jobs **(PREMIUM)** Use [protected environments](../environments/protected_environments.md) @@ -645,6 +647,7 @@ timed rollout 10%: script: echo 'Rolling out 10% ...' when: delayed start_in: 30 minutes + environment: production ``` To stop the active timer of a delayed job, select **Unschedule** (**{time-out}**). @@ -698,6 +701,7 @@ deploystacks: parallel: matrix: - PROVIDER: [aws, ovh, gcp, vultr] + environment: production/$PROVIDER ``` You can also [create a multi-dimensional matrix](../yaml/index.md#parallelmatrix). @@ -722,6 +726,7 @@ deploystacks: STACK: [monitoring, backup] - PROVIDER: [gcp, vultr] STACK: [data] + environment: $PROVIDER/$STACK ``` This example generates 6 parallel `deploystacks` trigger jobs, each with different values @@ -754,6 +759,7 @@ deploystacks: STACK: [data] tags: - ${PROVIDER}-${STACK} + environment: $PROVIDER/$STACK ``` #### Fetch artifacts from a `parallel:matrix` job @@ -784,6 +790,7 @@ deploy: dependencies: - "ruby: [2.7, aws]" script: echo hello + environment: production ``` Quotes around the `dependencies` entry are required. @@ -957,6 +964,33 @@ For example: Pattern matching is case-sensitive by default. Use the `i` flag modifier to make a pattern case-insensitive. For example: `/pattern/i`. +#### Store the regex pattern in a variable + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35438) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `ci_fix_rules_if_comparison_with_regexp_variable`, disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/359740) and feature flag `ci_fix_rules_if_comparison_with_regexp_variable` removed in GitLab 15.1. + +Variables on the right side of `=~` and `!~` expressions are evaluated as regular expressions. +The regular expression must be enclosed in forward slashes (`/`). For example: + +```yaml +variables: + pattern: '/^ab.*/' + +regex-job1: + variables: + teststring: 'abcde' + script: echo "This job will run, because 'abcde' matches the /^ab.*/ pattern." + rules: + - if: '$teststring =~ $pattern' + +regex-job2: + variables: + teststring: 'fghij' + script: echo "This job will not run, because 'fghi' does not match the /^ab.*/ pattern." + rules: + - if: '$teststring =~ $pattern' +``` + ### Join variable expressions together with `&&` or `||` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/62867) in GitLab 12.0 diff --git a/doc/ci/lint.md b/doc/ci/lint.md index 0811cc87e91..8c64d968b8b 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -24,7 +24,7 @@ configuration added with the [`includes` keyword](yaml/index.md#include). To check CI/CD configuration with the CI lint tool: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. @@ -45,7 +45,7 @@ Prerequisites: To simulate a pipeline: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index 7255d9aec82..efe11466674 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -136,6 +136,7 @@ job3: job4: stage: deploy script: make deploy + environment: production ``` #### Scheduled run @@ -196,6 +197,7 @@ deploy_prod: script: - echo "Deploy to production server" when: manual + environment: production ``` ### Filter job by branch @@ -222,6 +224,7 @@ deploy: - echo "Deploy job" rules: - if: $CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH =~ /^rc-/ + environment: production ``` ### Caching diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index c59116ea8ed..35c5a7e56c8 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -190,7 +190,7 @@ pdf: Additionally, we have package management features like built-in container and package registries that you can leverage. You can see the complete list of packaging features in the -[Packages & Registries](../../user/packages/index.md) documentation. +[Packages and registries](../../user/packages/index.md) documentation. ## Integrated features diff --git a/doc/ci/mobile_devops.md b/doc/ci/mobile_devops.md new file mode 100644 index 00000000000..6eb56434a1b --- /dev/null +++ b/doc/ci/mobile_devops.md @@ -0,0 +1,42 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# Mobile DevOps + +GitLab Mobile DevOps is a collection of features and tools designed for mobile developers +and teams to automate their build and release process using GitLab CI/CD. Mobile DevOps +is an experimental feature developed by [GitLab Incubation Engineering](https://about.gitlab.com/handbook/engineering/incubation/). + +Mobile DevOps is still in development, but you can: + +- [Request a feature](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=feature_request). +- [Report a bug](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=report_bug). +- [Share feedback](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/feedback/-/issues/new?issuable_template=general_feedback). + +## Code Signing + +[Project-level Secure Files](secure_files/index.md) makes it easier to manage key stores, provision profiles, +and signing certificates directly in a GitLab project. + +For a guided walkthrough of this feature, watch the [video demo](https://youtu.be/O7FbJu3H2YM). + +## Review Apps for Mobile + +You can use [Review Apps](review_apps/index.md) to preview changes directly from a merge request. +Review Apps for Mobile brings that capability to mobile developers through an integration +with [Appetize](https://appetize.io/). + +Watch a [video walkthrough](https://youtu.be/X15mI19TXa4) of this feature, or visit the +[setup instructions](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/readme/-/issues/15) +to get started. + +## Mobile SAST + +You can use [Static Application Security Testing (SAST)](../user/application_security/sast/index.md) +to run static analyzers on code to check for known security vulnerabilities. Mobile SAST +expands this functionality for mobile teams with an [experimental SAST feature](../user/application_security/sast/index.md#experimental-features) +based on [Mobile Security Framework (MobSF)](https://github.com/MobSF/Mobile-Security-Framework-MobSF). diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index 0fd8fac7741..87c2b3f1c71 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -160,15 +160,23 @@ checkbox appears. Select it to start a new merge request after you commit the ch ### `Configuration validation currently not available` message -This message is due to a problem with the syntax validation in the pipeline editor. -If GitLab is unable to communicate with the service that validates the syntax, the -information in these sections may not display properly: - -- The syntax status on the **Edit** tab (valid or invalid). -- The **Visualize** tab. -- The **Lint** tab. -- The **View merged YAML** tab. - -You can still work on your CI/CD configuration and commit the changes you made without -any issues. As soon as the service becomes available again, the syntax validation -should display immediately. +This message is caused by a problem validating the syntax in the pipeline editor. +It can happen when: + +- GitLab is unable to communicate with the service that validates the syntax, so the + information in these sections may not display properly: + + - The syntax status on the **Edit** tab (valid or invalid). + - The **Visualize** tab. + - The **Lint** tab. + - The **View merged YAML** tab. + + You can still work on your CI/CD configuration and commit the changes you made without + any issues. As soon as the service becomes available again, the syntax validation + should display immediately. + +- Using [`include`](../yaml/index.md#include), but the included configuration files create a loop. + For example, `.gitlab-ci.yml` includes `file1.yml`, which includes `file2.yml`, + which includes `file1.yml`, creating a loop between `file1.yml` and `file2.yml`. + + Remove one of the `include` lines to eliminate the loop and resolve the issue. diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index 4b7d3845361..02a74883244 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -49,7 +49,7 @@ Prerequisite: To change the default quota that applies to all namespaces: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment**. 1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. @@ -70,7 +70,7 @@ Prerequisite: To set a quota of CI/CD minutes for a namespace: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Groups**. 1. For the group you want to update, select **Edit**. 1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. @@ -95,7 +95,7 @@ Prerequisite: To view CI/CD minutes being used for your group: -1. On the top bar, select **Menu > Groups** and find your group. The group must not be a subgroup. +1. On the top bar, select **Main menu > Groups** and find your group. The group must not be a subgroup. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. Select the **Pipelines** tab. @@ -148,7 +148,7 @@ You can purchase additional CI/CD minutes for your group. You cannot transfer purchased CI/CD minutes from one group to another, so be sure to select the correct group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Usage Quotas**. 1. Select **Pipelines**. 1. Select **Buy additional minutes**. @@ -201,9 +201,12 @@ can be higher than the end-to-end duration of a pipeline. The cost factors for jobs running on shared runners on GitLab.com are: -- `0.008` for public projects, and projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). - For every 125 minutes of job execution time, you use 1 CI/CD minute. - `1` for internal and private projects. +- `0.5` for public projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). +- `0.008` for public forks of public projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). For every 125 minutes of job execution time, + you use 1 CI/CD minute. +- `0.04` for other public projects, after September 1, 2022 (previously `0.008`). + For every 25 minutes of job execution time, you use 1 CI/CD minute. - Calculated differently for [community contributions to GitLab projects](#cost-factor-for-community-contributions-to-gitlab-projects). The cost factors on self-managed instances are: @@ -236,12 +239,14 @@ GitLab administrators can add a namespace to the reduced cost factor ### Additional costs on GitLab SaaS -GitLab SaaS shared runners have different cost factors, depending on the runner type (Linux, Windows, macOS) and the virtual machine configuration. +GitLab SaaS runners have different cost factors, depending on the runner type (Linux, Windows, macOS) and the virtual machine configuration. -| GitLab SaaS runner type | Virtual machine configuration | CI/CD minutes cost factor | +| GitLab SaaS runner type | Machine Type | CI/CD minutes cost factor | | :--------- | :------------------- | :--------- | -| Linux OS + Docker executor| 1 vCPU, 3.75 GB RAM |1| -| macOS + shell executor | 4 vCPU, 10 GB RAM| 6 | +| Linux OS + Docker executor| Small |1| +| Linux OS + Docker executor| Medium |2| +| Linux OS + Docker executor| Large |3| +| macOS + shell executor | Large| 6 | ### Monthly reset of CI/CD minutes diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md new file mode 100644 index 00000000000..6c7ec8e98ec --- /dev/null +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -0,0 +1,647 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Downstream pipelines **(FREE)** + +A downstream pipeline is any GitLab CI/CD pipeline triggered by another pipeline. +A downstream pipeline can be either: + +- A [parent-child pipeline](downstream_pipelines.md#parent-child-pipelines), which is a downstream pipeline triggered + in the same project as the first pipeline. +- A [multi-project pipeline](#multi-project-pipelines), which is a downstream pipeline triggered + in a different project than the first pipeline. + +Parent-child pipelines and multi-project pipelines can sometimes be used for similar purposes, +but there are some key differences. + +Parent-child pipelines: + +- Run under the same project, ref, and commit SHA as the parent pipeline. +- Affect the overall status of the ref the pipeline runs against. For example, + if a pipeline fails for the main branch, it's common to say that "main is broken". + The status of child pipelines don't directly affect the status of the ref, unless the child + pipeline is triggered with [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are automatically canceled if the pipeline is configured with [`interruptible`](../yaml/index.md#interruptible) + when a new pipeline is created for the same ref. +- Display only the parent pipelines in the pipeline index page. Child pipelines are + visible when visiting their parent pipeline's page. +- Are limited to 2 levels of nesting. A parent pipeline can trigger multiple child pipelines, + and those child pipeline can trigger multiple child pipelines (`A -> B -> C`). + +Multi-project pipelines: + +- Are triggered from another pipeline, but the upstream (triggering) pipeline does + not have much control over the downstream (triggered) pipeline. However, it can + choose the ref of the downstream pipeline, and pass CI/CD variables to it. +- Affect the overall status of the ref of the project it runs in, but does not + affect the status of the triggering pipeline's ref, unless it was triggered with + [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are not automatically canceled in the downstream project when using [`interruptible`](../yaml/index.md#interruptible) + if a new pipeline runs for the same ref in the upstream pipeline. They can be + automatically canceled if a new pipeline is triggered for the same ref on the downstream project. +- Multi-project pipelines are standalone pipelines because they are normal pipelines + that happened to be triggered by an external project. They are all visible on the pipeline index page. +- Are independent, so there are no nesting limits. + +## Multi-project pipelines + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. + +You can set up [GitLab CI/CD](../index.md) across multiple projects, so that a pipeline +in one project can trigger a downstream pipeline in another project. You can visualize the entire pipeline +in one place, including all cross-project interdependencies. + +For example, you might deploy your web application from three different projects in GitLab. +Each project has its own build, test, and deploy process. With multi-project pipelines you can +visualize the entire pipeline, including all build and test stages for all three projects. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see the [Multi-project pipelines demo](https://www.youtube.com/watch?v=g_PIwBM1J84). + +Multi-project pipelines are also useful for larger products that require cross-project interdependencies, like those +with a [microservices architecture](https://about.gitlab.com/blog/2016/08/16/trends-in-version-control-land-microservices/). +Learn more in the [Cross-project Pipeline Triggering and Visualization demo](https://about.gitlab.com/learn/) +at GitLab@learn, in the Continuous Integration section. + +If you trigger a pipeline in a downstream private project, on the upstream project's pipelines page, +you can view: + +- The name of the project. +- The status of the pipeline. + +If you have a public project that can trigger downstream pipelines in a private project, +make sure there are no confidentiality problems. + +### Trigger a multi-project pipeline from a job in your `.gitlab-ci.yml` file + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. + +When you use the [`trigger`](../yaml/index.md#trigger) keyword to create a multi-project +pipeline in your `.gitlab-ci.yml` file, you create what is called a *trigger job*. For example: + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + variables: + ENVIRONMENT: staging + stage: deploy + trigger: my/deployment +``` + +In this example, after the `rspec` job succeeds in the `test` stage, +the `staging` trigger job starts. The initial status of this +job is `pending`. + +GitLab then creates a downstream pipeline in the +`my/deployment` project and, as soon as the pipeline is created, the +`staging` job succeeds. The full path to the project is `my/deployment`. + +You can view the status for the pipeline, or you can display +[the downstream pipeline's status instead](#mirror-the-status-of-a-downstream-pipeline-in-the-trigger-job). + +The user that creates the upstream pipeline must be able to create pipelines in the +downstream project (`my/deployment`) too. If the downstream project is not found, +or the user does not have [permission](../../user/permissions.md) to create a pipeline there, +the `staging` job is marked as _failed_. + +#### Specify a downstream pipeline branch + +You can specify a branch name for the downstream pipeline to use. +GitLab uses the commit on the head of the branch to +create the downstream pipeline. + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + stage: deploy + trigger: + project: my/deployment + branch: stable-11-2 +``` + +Use: + +- The `project` keyword to specify the full path to a downstream project. + In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367660), variable expansion is + supported. +- The `branch` keyword to specify the name of a branch in the project specified by `project`. + In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is + supported. + +Pipelines triggered on a protected branch in a downstream project use the [role](../../user/permissions.md) +of the user that ran the trigger job in the upstream project. If the user does not +have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See +[pipeline security for protected branches](index.md#pipeline-security-on-protected-branches). + +#### Use `rules` or `only`/`except` with multi-project pipelines + +You can use CI/CD variables or the [`rules`](../yaml/index.md#rulesif) keyword to +[control job behavior](../jobs/job_control.md) for multi-project pipelines. When a +downstream pipeline is triggered with the [`trigger`](../yaml/index.md#trigger) keyword, +the value of the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) +is `pipeline` for all its jobs. + +If you use [`only/except`](../yaml/index.md#only--except) to control job behavior, use the +[`pipelines`](../yaml/index.md#onlyrefs--exceptrefs) keyword. + +### Trigger a multi-project pipeline by using the API + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) to GitLab Free in 12.4. + +When you use the [`CI_JOB_TOKEN` to trigger pipelines](../jobs/ci_job_token.md), +GitLab recognizes the source of the job token. The pipelines become related, +so you can visualize their relationships on pipeline graphs. + +These relationships are displayed in the pipeline graph by showing inbound and +outbound connections for upstream and downstream pipeline dependencies. + +When using: + +- CI/CD variables or [`rules`](../yaml/index.md#rulesif) to control job behavior, the value of + the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) is + `pipeline` for multi-project pipeline triggered through the API with `CI_JOB_TOKEN`. +- [`only/except`](../yaml/index.md#only--except) to control job behavior, use the + `pipelines` keyword. + +## Parent-child pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7. + +As pipelines grow more complex, a few related problems start to emerge: + +- The staged structure, where all steps in a stage must be completed before the first + job in next stage begins, causes arbitrary waits, slowing things down. +- Configuration for the single global pipeline becomes very long and complicated, + making it hard to manage. +- Imports with [`include`](../yaml/index.md#include) increase the complexity of the configuration, and create the potential + for namespace collisions where jobs are unintentionally duplicated. +- Pipeline UX can become unwieldy with so many jobs and stages to work with. + +Additionally, sometimes the behavior of a pipeline needs to be more dynamic. The ability +to choose to start sub-pipelines (or not) is a powerful ability, especially if the +YAML is dynamically generated. + + + +Similarly to [multi-project pipelines](#multi-project-pipelines), a pipeline can trigger a +set of concurrently running downstream child pipelines, but in the same project: + +- Child pipelines still execute each of their jobs according to a stage sequence, but + would be free to continue forward through their stages without waiting for unrelated + jobs in the parent pipeline to finish. +- The configuration is split up into smaller child pipeline configurations. Each child pipeline contains only relevant steps which are + easier to understand. This reduces the cognitive load to understand the overall configuration. +- Imports are done at the child pipeline level, reducing the likelihood of collisions. + +Child pipelines work well with other GitLab CI/CD features: + +- Use [`rules: changes`](../yaml/index.md#ruleschanges) to trigger pipelines only when + certain files change. This is useful for monorepos, for example. +- Since the parent pipeline in `.gitlab-ci.yml` and the child pipeline run as normal + pipelines, they can have their own behaviors and sequencing in relation to triggers. + +See the [`trigger`](../yaml/index.md#trigger) keyword documentation for full details on how to +include the child pipeline configuration. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8KpBSqZNbk). + +NOTE: +The artifact containing the generated YAML file must not be larger than 5MB. + +### Trigger a parent-child pipeline + +The simplest case is [triggering a child pipeline](../yaml/index.md#trigger) using a +local YAML file to define the pipeline configuration. In this case, the parent pipeline +triggers the child pipeline, and continues without waiting: + +```yaml +microservice_a: + trigger: + include: path/to/microservice_a.yml +``` + +You can include multiple files when defining a child pipeline. The child pipeline's +configuration is composed of all configuration files merged together: + +```yaml +microservice_a: + trigger: + include: + - local: path/to/microservice_a.yml + - template: Security/SAST.gitlab-ci.yml +``` + +In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/205157), +you can use [`include:file`](../yaml/index.md#includefile) to trigger child pipelines +with a configuration file in a different project: + +```yaml +microservice_a: + trigger: + include: + - project: 'my-group/my-pipeline-library' + ref: 'main' + file: '/path/to/child-pipeline.yml' +``` + +The maximum number of entries that are accepted for `trigger:include` is three. + +### Merge request child pipelines + +To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md) we need to: + +- Set the trigger job to run on merge requests: + +```yaml +# parent .gitlab-ci.yml +microservice_a: + trigger: + include: path/to/microservice_a.yml + rules: + - if: $CI_MERGE_REQUEST_ID +``` + +- Configure the child pipeline by either: + + - Setting all jobs in the child pipeline to evaluate in the context of a merge request: + + ```yaml + # child path/to/microservice_a.yml + workflow: + rules: + - if: $CI_MERGE_REQUEST_ID + + job1: + script: ... + + job2: + script: ... + ``` + + - Alternatively, setting the rule per job. For example, to create only `job1` in + the context of merge request pipelines: + + ```yaml + # child path/to/microservice_a.yml + job1: + script: ... + rules: + - if: $CI_MERGE_REQUEST_ID + + job2: + script: ... + ``` + +### Dynamic child pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9. + +Instead of running a child pipeline from a static YAML file, you can define a job that runs +your own script to generate a YAML file, which is then used to trigger a child pipeline. + +This technique can be very powerful in generating pipelines targeting content that changed or to +build a matrix of targets and architectures. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM). + +We also have an example project using +[Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) +which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. +You could use a similar process for other templating languages like +[Dhall](https://dhall-lang.org/) or [ytt](https://get-ytt.io/). + +The artifact path is parsed by GitLab, not the runner, so the path must match the +syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows +runner for testing, the path separator for the trigger job would be `/`. Other CI/CD +configuration for jobs, like scripts, that use the Windows runner would use `\`. + +For example, to trigger a child pipeline from a dynamically generated configuration file: + +```yaml +generate-config: + stage: build + script: generate-ci-config > generated-config.yml + artifacts: + paths: + - generated-config.yml + +child-pipeline: + stage: test + trigger: + include: + - artifact: generated-config.yml + job: generate-config +``` + +The `generated-config.yml` is extracted from the artifacts and used as the configuration +for triggering the child pipeline. + +In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail. +This is [resolved](https://gitlab.com/gitlab-org/gitlab/-/issues/209070) in GitLab 12.10. + +### Nested child pipelines + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29651) in GitLab 13.4. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243747) in GitLab 13.5. + +Parent and child pipelines were introduced with a maximum depth of one level of child +pipelines, which was later increased to two. A parent pipeline can trigger many child +pipelines, and these child pipelines can trigger their own child pipelines. It's not +possible to trigger another level of child pipelines. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Nested Dynamic Pipelines](https://youtu.be/C5j3ju9je2M). + +## View a downstream pipeline + +In the [pipeline graph view](index.md#view-full-pipeline-graph), downstream pipelines display +as a list of cards on the right of the graph. + +### Retry a downstream pipeline + +> - Retry from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. +> - Retry from graph view [generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. + +To retry a completed downstream pipeline, select **Retry** (**{retry}**): + +- From the downstream pipeline's details page. +- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). + +### Cancel a downstream pipeline + +> - Retry from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. +> - Retry from graph view [generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. + +To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**): + +- From the downstream pipeline's details page. +- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). + +### Mirror the status of a downstream pipeline in the trigger job + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11238) in GitLab Premium 12.3. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. + +You can mirror the pipeline status from the triggered pipeline to the source trigger job +by using [`strategy: depend`](../yaml/index.md#triggerstrategy): + +::Tabs + +:::TabTitle Multi-project pipeline + +```yaml +trigger_job: + trigger: + project: my/project + strategy: depend +``` + +:::TabTitle Parent-child pipeline + +```yaml +trigger_job: + trigger: + include: + - local: path/to/child-pipeline.yml + strategy: depend +``` + +::EndTabs + +### View multi-project pipelines in pipeline graphs **(PREMIUM)** + +When you trigger a multi-project pipeline, the downstream pipeline displays +to the right of the [pipeline graph](index.md#visualize-pipelines). + + + +In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipeline +displays to the right of the mini graph. + + + +## Pass artifacts to a downstream pipeline + +You can pass artifacts to a downstream pipeline by using [`needs:project`](../yaml/index.md#needsproject). + +1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. +1. Trigger the downstream pipeline with a trigger job: + + ```yaml + build_artifacts: + stage: build + script: + - echo "This is a test artifact!" >> artifact.txt + artifacts: + paths: + - artifact.txt + + deploy: + stage: deploy + trigger: my/downstream_project + ``` + +1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline + by using `needs:project`. Set `job` to the job in the upstream pipeline to fetch artifacts from, + `ref` to the branch, and `artifacts: true`. + + ```yaml + test: + stage: test + script: + - cat artifact.txt + needs: + - project: my/upstream_project + job: build_artifacts + ref: main + artifacts: true + ``` + +### Pass artifacts from a Merge Request pipeline + +When you use `needs:project` to [pass artifacts to a downstream pipeline](#pass-artifacts-to-a-downstream-pipeline), +the `ref` value is usually a branch name, like `main` or `development`. + +For merge request pipelines, the `ref` value is in the form of `refs/merge-requests/<id>/head`, +where `id` is the merge request ID. You can retrieve this ref with the [`CI_MERGE_REQUEST_REF_PATH`](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) +CI/CD variable. Do not use a branch name as the `ref` with merge request pipelines, +because the downstream pipeline attempts to fetch artifacts from the latest branch pipeline. + +To fetch the artifacts from the upstream `merge request` pipeline instead of the `branch` pipeline, +pass this variable to the downstream pipeline using variable inheritance: + +1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. +1. In the job that triggers the downstream pipeline, pass the `$CI_MERGE_REQUEST_REF_PATH` variable by using + [variable inheritance](#pass-yaml-defined-cicd-variables): + + ```yaml + build_artifacts: + stage: build + script: + - echo "This is a test artifact!" >> artifact.txt + artifacts: + paths: + - artifact.txt + + upstream_job: + variables: + UPSTREAM_REF: $CI_MERGE_REQUEST_REF_PATH + trigger: + project: my/downstream_project + branch: my-branch + ``` + +1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline + by using `needs:project`. Set the `ref` to the `UPSTREAM_REF` variable, and `job` + to the job in the upstream pipeline to fetch artifacts from: + + ```yaml + test: + stage: test + script: + - cat artifact.txt + needs: + - project: my/upstream_project + job: build_artifacts + ref: $UPSTREAM_REF + artifacts: true + ``` + +This method works for fetching artifacts from a regular merge request parent pipeline, +but fetching artifacts from [merge results](merged_results_pipelines.md) pipelines is not supported. + +## Pass CI/CD variables to a downstream pipeline + +You can pass CI/CD variables to a downstream pipeline with a few different methods, +based on where the variable is created or defined. + +### Pass YAML-defined CI/CD variables + +You can use the `variables` keyword to pass CI/CD variables to a downstream pipeline, +just like you would for any other job. + +For example, in a [multi-project pipeline](#multi-project-pipelines): + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + variables: + ENVIRONMENT: staging + stage: deploy + trigger: my/deployment +``` + +The `ENVIRONMENT` variable is passed to every job defined in a downstream +pipeline. It is available as a variable when GitLab Runner picks a job. + +In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline +that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` +job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. + +```yaml +variables: + MY_VARIABLE: my-value + +trigger-downstream: + variables: + ENVIRONMENT: something + trigger: my/project +``` + +### Prevent global variables from being passed + +You can stop global variables from reaching the downstream pipeline by using the [`inherit:variables` keyword](../yaml/index.md#inheritvariables). +For example, in a [multi-project pipeline](#multi-project-pipelines): + +```yaml +variables: + MY_GLOBAL_VAR: value + +trigger-downstream: + inherit: + variables: false + variables: + MY_LOCAL_VAR: value + trigger: my/project +``` + +In this example, the `MY_GLOBAL_VAR` variable is not available in the triggered pipeline. + +### Pass a predefined variable + +You might want to pass some information about the upstream pipeline using predefined variables. +To do that, you can use interpolation to pass any variable. For example, +in a [multi-project pipeline](#multi-project-pipelines): + +```yaml +downstream-job: + variables: + UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME + trigger: my/project +``` + +In this scenario, the `UPSTREAM_BRANCH` variable with the value of the upstream pipeline's +`$CI_COMMIT_REF_NAME` is passed to `downstream-job`. It is available in the +context of all downstream builds. + +You cannot use this method to forward [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables) +to a downstream pipeline, as they are not available in trigger jobs. + +Upstream pipelines take precedence over downstream ones. If there are two +variables with the same name defined in both upstream and downstream projects, +the ones defined in the upstream project take precedence. + +### Pass dotenv variables created in a job **(PREMIUM)** + +You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) +and [`needs:project`](../yaml/index.md#needsproject). + +For example, in a [multi-project pipeline](#multi-project-pipelines): + +1. Save the variables in a `.env` file. +1. Save the `.env` file as a `dotenv` report. +1. Trigger the downstream pipeline. + + ```yaml + build_vars: + stage: build + script: + - echo "BUILD_VERSION=hello" >> build.env + artifacts: + reports: + dotenv: build.env + + deploy: + stage: deploy + trigger: my/downstream_project + ``` + +1. Set the `test` job in the downstream pipeline to inherit the variables from the `build_vars` + job in the upstream project with `needs`. The `test` job inherits the variables in the + `dotenv` report and it can access `BUILD_VERSION` in the script: + + ```yaml + test: + stage: test + script: + - echo $BUILD_VERSION + needs: + - project: my/upstream_project + job: build_vars + ref: master + artifacts: true + ``` diff --git a/doc/ci/pipelines/img/downstream_pipeline_actions.png b/doc/ci/pipelines/img/downstream_pipeline_actions.png Binary files differdeleted file mode 100644 index 4c4384bab57..00000000000 --- a/doc/ci/pipelines/img/downstream_pipeline_actions.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 08264170d52..2696d3adabd 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -57,44 +57,10 @@ Pipelines can be configured in many different ways: already been merged into the target branch. - [Merge trains](../pipelines/merge_trains.md) use merged results pipelines to queue merges one after the other. -- [Parent-child pipelines](parent_child_pipelines.md) break down complex pipelines +- [Parent-child pipelines](downstream_pipelines.md#parent-child-pipelines) break down complex pipelines into one parent pipeline that can trigger multiple child sub-pipelines, which all run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. -- [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. - -### How parent-child pipelines compare to multi-project pipelines - -Parent-child pipelines and multi-project pipelines can sometimes be used for similar -purposes, but there are some key differences: - -Parent-child pipelines: - -- Run under the same project, ref, and commit SHA as the parent pipeline. -- Affect the overall status of the ref the pipeline runs against. For example, - if a pipeline fails for the main branch, it's common to say that "main is broken". - The status of child pipelines don't directly affect the status of the ref, unless the child - pipeline is triggered with [`strategy:depend`](../yaml/index.md#triggerstrategy). -- Are automatically canceled if the pipeline is configured with [`interruptible`](../yaml/index.md#interruptible) - when a new pipeline is created for the same ref. -- Display only the parent pipelines in the pipeline index page. Child pipelines are - visible when visiting their parent pipeline's page. -- Are limited to 2 levels of nesting. A parent pipeline can trigger multiple child pipelines, - and those child pipeline can trigger multiple child pipelines (`A -> B -> C`). - -Multi-project pipelines: - -- Are triggered from another pipeline, but the upstream (triggering) pipeline does - not have much control over the downstream (triggered) pipeline. However, it can - choose the ref of the downstream pipeline, and pass CI/CD variables to it. -- Affect the overall status of the ref of the project it runs in, but does not - affect the status of the triggering pipeline's ref, unless it was triggered with - [`strategy:depend`](../yaml/index.md#triggerstrategy). -- Are not automatically canceled in the downstream project when using [`interruptible`](../yaml/index.md#interruptible) - if a new pipeline runs for the same ref in the upstream pipeline. They can be - automatically canceled if a new pipeline is triggered for the same ref on the downstream project. -- Multi-project pipelines are standalone pipelines because they are normal pipelines - that happened to be triggered by an external project. They are all visible on the pipeline index page. -- Are independent, so there are no nesting limits. +- [Multi-project pipelines](downstream_pipelines.md#multi-project-pipelines) combine pipelines for different projects together. ## Configure a pipeline @@ -175,7 +141,7 @@ operation of the pipeline. To execute a pipeline manually: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. 1. Select **Run pipeline**. 1. In the **Run for branch name or tag** field, select the branch or tag to run the pipeline for. @@ -288,7 +254,7 @@ page, then selecting **Delete**.  Deleting a pipeline does not automatically delete its -[child pipelines](parent_child_pipelines.md). +[child pipelines](downstream_pipelines.md#parent-child-pipelines). See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/39503) for details. @@ -323,6 +289,34 @@ preserving deployment keys and other credentials from being unintentionally accessed. To ensure that jobs intended to be executed on protected runners do not use regular runners, they must be tagged accordingly. +## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.8. + +You can trigger a pipeline in your project whenever a pipeline finishes for a new +tag in a different project. + +Prerequisites: + +- The upstream project must be [public](../../user/public_access.md). +- The user must have the Developer role + in the upstream project. + +To trigger the pipeline when the upstream project is rebuilt: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Pipeline subscriptions**. +1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. + For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. +1. Select **Subscribe**. + +Any pipelines that complete successfully for new tags in the subscribed project +now trigger a pipeline on the current project's default branch. The maximum +number of upstream pipeline subscriptions is 2 by default, for both the upstream and +downstream projects. On self-managed instances, an administrator can change this +[limit](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project). + ### How pipeline duration is calculated Total running time for a given pipeline excludes retries and pending @@ -388,8 +382,8 @@ You can group the jobs by: - [Job dependencies](#view-job-dependencies-in-the-pipeline-graph), which arranges jobs based on their [`needs`](../yaml/index.md#needs) dependencies. -[Multi-project pipeline graphs](multi_project_pipelines.md#multi-project-pipeline-visualization) help -you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)** +[Multi-project pipeline graphs](downstream_pipelines.md#view-multi-project-pipelines-in-pipeline-graphs) help +you visualize the entire pipeline, including all cross-project inter-dependencies. If a stage contains more than 100 jobs, only the first 100 jobs are listed in the pipeline graph. The remaining jobs still run as normal. To see the jobs: @@ -403,8 +397,9 @@ pipeline graph. The remaining jobs still run as normal. To see the jobs: > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/328538) in GitLab 14.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328538) in GitLab 14.2. -You can arrange jobs in the pipeline graph based on their [`needs`](../yaml/index.md#needs) -dependencies. +To arrange jobs in the pipeline graph based on their [`needs`](../yaml/index.md#needs) +dependencies, select **Job dependencies** in the **Group jobs by** section. This option +is available for pipelines with 3 or more jobs with `needs` job dependencies. Jobs in the leftmost column run first, and jobs that depend on them are grouped in the next columns. @@ -455,25 +450,6 @@ Pipeline analytics are available on the [**CI/CD Analytics** page](../../user/an Pipeline status and test coverage report badges are available and configurable for each project. For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges). -### Downstream pipelines - -In the pipeline graph view, downstream pipelines ([Multi-project pipelines](multi_project_pipelines.md) -and [Parent-child pipelines](parent_child_pipelines.md)) display as a list of cards -on the right of the graph. - -#### Cancel or retry downstream pipelines from the graph view - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. -> - [Generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. - -To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**) -on the pipeline's card. - -To retry a failed downstream pipeline, select **Retry** (**{retry}**) -on the pipeline's card. - - - ## Pipelines API GitLab provides API endpoints to: diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index c8babe3320d..f30ae32efb2 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -305,7 +305,7 @@ the artifact. ## How searching for job artifacts works In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201784), artifacts -for [parent and child pipelines](parent_child_pipelines.md) are searched in hierarchical +for [parent and child pipelines](downstream_pipelines.md#parent-child-pipelines) are searched in hierarchical order from parent to child. For example, if both parent and child pipelines have a job with the same name, the job artifact from the parent pipeline is returned. @@ -388,7 +388,7 @@ Keeping the latest artifacts can use a large amount of storage space in projects with a lot of jobs or large artifacts. If the latest artifacts are not needed in a project, you can disable this behavior to save space: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Artifacts**. 1. Clear the **Keep artifacts from most recent successful jobs** checkbox. @@ -451,3 +451,63 @@ test-job: reports: dotenv: build.env ``` + +### Job artifacts are not expired + +If some job artifacts are not expiring as expected, check if the +[**Keep artifacts from most recent successful jobs**](#keep-artifacts-from-most-recent-successful-jobs) +setting is enabled. + +When this setting is enabled, job artifacts from the latest successful pipeline +of each ref do not expire and are not deleted. + +### Error message `This job could not start because it could not retrieve the needed artifacts.` + +A job configured with [`needs:artifacts`](../yaml/index.md#needsartifacts) keyword +fails to start and returns this error message if: + +- The job's dependencies cannot be found. +- The job cannot access the relevant resources due to insufficient permissions. + +The troubleshooting steps to follow are determined by the syntax used in the job configuration. + +#### Job configured with `needs:project` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:project`](../yaml/index.md#needsproject), with a configuration similar to: + +```yaml +rspec: + needs: + - project: org/another-project + job: dependency-job + ref: master + artifacts: true +``` + +To troubleshoot this job, verify that: + +- Project `org/another-project` is in a group with a Premium subscription plan. +- The user running the job has permissions to access resources in `org/another-project`. +- The `project`, `job`, and `ref` combination exists and results in the desired dependency. +- Any variables in use evaluate to the correct values. + +#### Job configured with `needs:pipeline:job` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:pipeline:job`](../yaml/index.md#needspipelinejob), with a configuration similar to: + +```yaml +rspec: + needs: + - pipeline: $UPSTREAM_PIPELINE_ID + job: dependency-job + artifacts: true +``` + +To troubleshoot this job, verify that: + +- The `$UPSTREAM_PIPELINE_ID` CI/CD variable is available in the current pipeline's + parent-child pipeline hierarchy. +- The `pipeline` and `job` combination exists and resolves to an existing pipeline. +- `dependency-job` has run and finished successfully. diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 5ba489c9830..f6c93356046 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -218,3 +218,28 @@ is not considered successful if: When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) feature and both pipelines types are present, the merge request pipelines are checked, not the branch pipelines. + +### `An error occurred while trying to run a new pipeline for this merge request.` + +This error can happen when you select **Run pipeline** in a merge request, but the +project does not have merge request pipelines enabled anymore. + +Some possible reasons for this error message: + +- The project does not have merge request pipelines enabled, has no pipelines listed + in the **Pipelines** tab, and you select **Run pipelines**. +- The project used to have merge request pipelines enabled, but the configuration + was removed. For example: + + 1. The project has merge request pipelines enabled in the `.gitlab-ci.yml` configuration + file when the merge request is created. + 1. The **Run pipeline** options is available in the merge request's **Pipelines** tab, + and selecting **Run pipeline** at this point likely does not cause any errors. + 1. The project's `.gitlab-ci.yml` file is changed to remove the merge request pipelines configuration. + 1. The branch is rebased to bring the updated configuration into the merge request. + 1. Now the pipeline configuration no longer supports merge request pipelines, + but you select **Run pipeline** to run a merge request pipeline. + +If **Run pipeline** is available, but the project does not have merge request pipelines +enabled, do not use this option. You can push a commit or rebase the branch to trigger +new branch pipelines. diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 2882cd378aa..88ab6163f3c 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -59,7 +59,7 @@ changes that are included in the target branch, and the `C` changes that are fro the merge request already in the train. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -Watch this video for a demonstration on +Watch this video for a demonstration on [how parallel execution of merge trains can prevent commits from breaking the default branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ). ## Prerequisites @@ -81,9 +81,8 @@ To enable merge trains for your project: 1. If you are on a self-managed GitLab instance, ensure the [feature flag](#merge-trains-feature-flag) is set correctly. 1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) so that the pipeline or individual jobs run for merge requests. -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. 1. In the **Merge method** section, verify that **Merge commit** is selected. 1. In the **Merge options** section, select **Enable merged results pipelines** (if not already selected) and **Enable merge trains**. 1. Select **Save changes**. @@ -210,7 +209,7 @@ If it succeeds after a retry, the merge request is not removed from the merge tr Sometimes the **Start/Add to merge train** button is not available and the merge request says, "The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure." -This issue occurs when [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +This issue occurs when [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) is enabled in **Settings > General > Merge requests**. This option requires that you run a new successful pipeline before you can re-add a merge request to a merge train. diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 777871a7c5f..7209a6b9a77 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -41,10 +41,9 @@ To use merged results pipelines: To enable merged results pipelines in a project, you must have at least the Maintainer role: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. Select **Enable merged results pipelines**. +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge options** section, select **Enable merged results pipelines**. 1. Select **Save changes**. WARNING: diff --git a/doc/ci/pipelines/multi_project_pipelines.md b/doc/ci/pipelines/multi_project_pipelines.md index a71af78f410..824f1445818 100644 --- a/doc/ci/pipelines/multi_project_pipelines.md +++ b/doc/ci/pipelines/multi_project_pipelines.md @@ -1,419 +1,11 @@ --- -stage: Verify -group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'downstream_pipelines.md' +remove_date: '2022-11-31' --- -# Multi-project pipelines **(FREE)** +This document was moved to [another location](downstream_pipelines.md). -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. - -You can set up [GitLab CI/CD](../index.md) across multiple projects, so that a pipeline -in one project can trigger a pipeline in another project. You can visualize the entire pipeline -in one place, including all cross-project interdependencies. - -For example, you might deploy your web application from three different projects in GitLab. -Each project has its own build, test, and deploy process. With multi-project pipelines you can -visualize the entire pipeline, including all build and test stages for all three projects. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see the [Multi-project pipelines demo](https://www.youtube.com/watch?v=g_PIwBM1J84). - -Multi-project pipelines are also useful for larger products that require cross-project interdependencies, like those -with a [microservices architecture](https://about.gitlab.com/blog/2016/08/16/trends-in-version-control-land-microservices/). -Learn more in the [Cross-project Pipeline Triggering and Visualization demo](https://about.gitlab.com/learn/) -at GitLab@learn, in the Continuous Integration section. - -If you trigger a pipeline in a downstream private project, on the upstream project's pipelines page, -you can view: - -- The name of the project. -- The status of the pipeline. - -If you have a public project that can trigger downstream pipelines in a private project, -make sure there are no confidentiality problems. - -## Create multi-project pipelines - -To create multi-project pipelines, you can: - -- [Define them in your `.gitlab-ci.yml` file](#define-multi-project-pipelines-in-your-gitlab-ciyml-file). -- [Use the API](#create-multi-project-pipelines-by-using-the-api). - -### Define multi-project pipelines in your `.gitlab-ci.yml` file - -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. - -When you use the [`trigger`](../yaml/index.md#trigger) keyword to create a multi-project -pipeline in your `.gitlab-ci.yml` file, you create what is called a *trigger job*. For example: - -```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - variables: - ENVIRONMENT: staging - stage: deploy - trigger: my/deployment -``` - -In this example, after the `rspec` job succeeds in the `test` stage, -the `staging` trigger job starts. The initial status of this -job is `pending`. - -GitLab then creates a downstream pipeline in the -`my/deployment` project and, as soon as the pipeline is created, the -`staging` job succeeds. The full path to the project is `my/deployment`. - -You can view the status for the pipeline, or you can display -[the downstream pipeline's status instead](#mirror-status-of-a-triggered-pipeline-in-the-trigger-job). - -The user that creates the upstream pipeline must be able to create pipelines in the -downstream project (`my/deployment`) too. If the downstream project is not found, -or the user does not have [permission](../../user/permissions.md) to create a pipeline there, -the `staging` job is marked as _failed_. - -#### Trigger job configuration limitations - -Trigger jobs can use only a limited set of the GitLab CI/CD [configuration keywords](../yaml/index.md). -The keywords available for use in trigger jobs are: - -- [`trigger`](../yaml/index.md#trigger) -- [`stage`](../yaml/index.md#stage) -- [`allow_failure`](../yaml/index.md#allow_failure) -- [`rules`](../yaml/index.md#rules) -- [`only` and `except`](../yaml/index.md#only--except) -- [`when`](../yaml/index.md#when) (only with a value of `on_success`, `on_failure`, or `always`) -- [`extends`](../yaml/index.md#extends) -- [`needs`](../yaml/index.md#needs), but not [`needs:project`](../yaml/index.md#needsproject) - -Trigger jobs cannot use [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables). - -#### Specify a downstream pipeline branch - -You can specify a branch name for the downstream pipeline to use. -GitLab uses the commit on the head of the branch to -create the downstream pipeline. - -```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - stage: deploy - trigger: - project: my/deployment - branch: stable-11-2 -``` - -Use: - -- The `project` keyword to specify the full path to a downstream project. - In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367660), variable expansion is - supported. -- The `branch` keyword to specify the name of a branch in the project specified by `project`. - In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is - supported. - -Pipelines triggered on a protected branch in a downstream project use the [role](../../user/permissions.md) -of the user that ran the trigger job in the upstream project. If the user does not -have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See -[pipeline security for protected branches](index.md#pipeline-security-on-protected-branches). - -#### Pass CI/CD variables to a downstream pipeline by using the `variables` keyword - -Sometimes you might want to pass CI/CD variables to a downstream pipeline. -You can do that by using the `variables` keyword, just like you would for any other job. - -```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - variables: - ENVIRONMENT: staging - stage: deploy - trigger: my/deployment -``` - -The `ENVIRONMENT` variable is passed to every job defined in a downstream -pipeline. It is available as a variable when GitLab Runner picks a job. - -In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline -that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` -job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. - -```yaml -variables: - MY_VARIABLE: my-value - -trigger-downstream: - variables: - ENVIRONMENT: something - trigger: my/project -``` - -You can stop global variables from reaching the downstream pipeline by using the [`inherit:variables` keyword](../yaml/index.md#inheritvariables). -In this example, the `MY_GLOBAL_VAR` variable is not available in the triggered pipeline: - -```yaml -variables: - MY_GLOBAL_VAR: value - -trigger-downstream: - inherit: - variables: false - variables: - MY_LOCAL_VAR: value - trigger: my/project -``` - -You might want to pass some information about the upstream pipeline using, for -example, predefined variables. In order to do that, you can use interpolation -to pass any variable. For example: - -```yaml -downstream-job: - variables: - UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME - trigger: my/project -``` - -In this scenario, the `UPSTREAM_BRANCH` variable with the value of the upstream pipeline's -`$CI_COMMIT_REF_NAME` is passed to `downstream-job`. It is available in the -context of all downstream builds. - -You cannot use this method to forward [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables) -to a downstream pipeline, as they are not available in trigger jobs. - -Upstream pipelines take precedence over downstream ones. If there are two -variables with the same name defined in both upstream and downstream projects, -the ones defined in the upstream project take precedence. - -#### Pass CI/CD variables to a downstream pipeline by using variable inheritance **(PREMIUM)** - -You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) and [`needs:project`](../yaml/index.md#needsproject). - -In the upstream pipeline: - -1. Save the variables in a `.env` file. -1. Save the `.env` file as a `dotenv` report. -1. Trigger the downstream pipeline. - - ```yaml - build_vars: - stage: build - script: - - echo "BUILD_VERSION=hello" >> build.env - artifacts: - reports: - dotenv: build.env - - deploy: - stage: deploy - trigger: my/downstream_project - ``` - -1. Set the `test` job in the downstream pipeline to inherit the variables from the `build_vars` - job in the upstream project with `needs`. The `test` job inherits the variables in the - `dotenv` report and it can access `BUILD_VERSION` in the script: - - ```yaml - test: - stage: test - script: - - echo $BUILD_VERSION - needs: - - project: my/upstream_project - job: build_vars - ref: master - artifacts: true - ``` - -#### Pass artifacts to a downstream pipeline - -You can pass artifacts to a downstream pipeline by using [`needs:project`](../yaml/index.md#needsproject). - -1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. -1. Trigger the downstream pipeline with a trigger job: - - ```yaml - build_artifacts: - stage: build - script: - - echo "This is a test artifact!" >> artifact.txt - artifacts: - paths: - - artifact.txt - - deploy: - stage: deploy - trigger: my/downstream_project - ``` - -1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline - by using `needs:project`. Set `job` to the job in the upstream pipeline to fetch artifacts from, - `ref` to the branch, and `artifacts: true`. - - ```yaml - test: - stage: test - script: - - cat artifact.txt - needs: - - project: my/upstream_project - job: build_artifacts - ref: main - artifacts: true - ``` - -#### Pass artifacts to a downstream pipeline from a Merge Request pipeline - -When you use `needs:project` to [pass artifacts to a downstream pipeline](#pass-artifacts-to-a-downstream-pipeline), -the `ref` value is usually a branch name, like `main` or `development`. - -For merge request pipelines, the `ref` value is in the form of `refs/merge-requests/<id>/head`, -where `id` is the merge request ID. You can retrieve this ref with the [`CI_MERGE_REQUEST_REF_PATH`](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) -CI/CD variable. Do not use a branch name as the `ref` with merge request pipelines, -because the downstream pipeline attempts to fetch artifacts from the latest branch pipeline. - -To fetch the artifacts from the upstream `merge request` pipeline instead of the `branch` pipeline, -pass this variable to the downstream pipeline using variable inheritance: - -1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. -1. In the job that triggers the downstream pipeline, pass the `$CI_MERGE_REQUEST_REF_PATH` variable by using - [variable inheritance](#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword): - - ```yaml - build_artifacts: - stage: build - script: - - echo "This is a test artifact!" >> artifact.txt - artifacts: - paths: - - artifact.txt - - upstream_job: - variables: - UPSTREAM_REF: $CI_MERGE_REQUEST_REF_PATH - trigger: - project: my/downstream_project - branch: my-branch - ``` - -1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline - by using `needs:project`. Set the `ref` to the `UPSTREAM_REF` variable, and `job` - to the job in the upstream pipeline to fetch artifacts from: - - ```yaml - test: - stage: test - script: - - cat artifact.txt - needs: - - project: my/upstream_project - job: build_artifacts - ref: UPSTREAM_REF - artifacts: true - ``` - -This method works for fetching artifacts from a regular merge request parent pipeline, -but fetching artifacts from [merge results](merged_results_pipelines.md) pipelines is not supported. - -#### Use `rules` or `only`/`except` with multi-project pipelines - -You can use CI/CD variables or the [`rules`](../yaml/index.md#rulesif) keyword to -[control job behavior](../jobs/job_control.md) for multi-project pipelines. When a -downstream pipeline is triggered with the [`trigger`](../yaml/index.md#trigger) keyword, -the value of the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) -is `pipeline` for all its jobs. - -If you use [`only/except`](../yaml/index.md#only--except) to control job behavior, use the -[`pipelines`](../yaml/index.md#onlyrefs--exceptrefs) keyword. - -#### Mirror status of a triggered pipeline in the trigger job - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11238) in GitLab Premium 12.3. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. - -You can mirror the pipeline status from the triggered pipeline to the source trigger job -by using [`strategy: depend`](../yaml/index.md#triggerstrategy). For example: - -```yaml -trigger_job: - trigger: - project: my/project - strategy: depend -``` - -### Create multi-project pipelines by using the API - -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) to GitLab Free in 12.4. - -When you use the [`CI_JOB_TOKEN` to trigger pipelines](../jobs/ci_job_token.md), -GitLab recognizes the source of the job token. The pipelines become related, -so you can visualize their relationships on pipeline graphs. - -These relationships are displayed in the pipeline graph by showing inbound and -outbound connections for upstream and downstream pipeline dependencies. - -When using: - -- CI/CD variables or [`rules`](../yaml/index.md#rulesif) to control job behavior, the value of - the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md) is - `pipeline` for multi-project pipeline triggered through the API with `CI_JOB_TOKEN`. -- [`only/except`](../yaml/index.md#only--except) to control job behavior, use the - `pipelines` keyword. - -## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.8. - -You can trigger a pipeline in your project whenever a pipeline finishes for a new -tag in a different project. - -Prerequisites: - -- The upstream project must be [public](../../user/public_access.md). -- The user must have the Developer role - in the upstream project. - -To trigger the pipeline when the upstream project is rebuilt: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. -1. Expand **Pipeline subscriptions**. -1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. - For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. -1. Select **Subscribe**. - -Any pipelines that complete successfully for new tags in the subscribed project -now trigger a pipeline on the current project's default branch. The maximum -number of upstream pipeline subscriptions is 2 by default, for both the upstream and -downstream projects. On self-managed instances, an administrator can change this -[limit](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project). - -## Multi-project pipeline visualization **(PREMIUM)** - -When your pipeline triggers a downstream pipeline, the downstream pipeline displays -to the right of the [pipeline graph](index.md#visualize-pipelines). - - - -In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipeline -displays to the right of the mini graph. - - - -## Retry or cancel multi-project pipelines - -If you have permission to trigger pipelines in the downstream project, you can -retry or cancel multi-project pipelines: - -- [In the main graph view](index.md#downstream-pipelines). -- From the downstream pipeline's details page. +<!-- This redirect file can be deleted after <2022-11-31>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index 3fd739087ec..be8ed8ba6d7 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -1,228 +1,11 @@ --- -stage: Verify -group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'downstream_pipelines.md' +remove_date: '2022-12-05' --- -# Parent-child pipelines **(FREE)** +This document was moved to [another location](downstream_pipelines.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16094) in GitLab 12.7. - -As pipelines grow more complex, a few related problems start to emerge: - -- The staged structure, where all steps in a stage must be completed before the first - job in next stage begins, causes arbitrary waits, slowing things down. -- Configuration for the single global pipeline becomes very long and complicated, - making it hard to manage. -- Imports with [`include`](../yaml/index.md#include) increase the complexity of the configuration, and create the potential - for namespace collisions where jobs are unintentionally duplicated. -- Pipeline UX can become unwieldy with so many jobs and stages to work with. - -Additionally, sometimes the behavior of a pipeline needs to be more dynamic. The ability -to choose to start sub-pipelines (or not) is a powerful ability, especially if the -YAML is dynamically generated. - - - -Similarly to [multi-project pipelines](multi_project_pipelines.md), a pipeline can trigger a -set of concurrently running child pipelines, but within the same project: - -- Child pipelines still execute each of their jobs according to a stage sequence, but - would be free to continue forward through their stages without waiting for unrelated - jobs in the parent pipeline to finish. -- The configuration is split up into smaller child pipeline configurations. Each child pipeline contains only relevant steps which are - easier to understand. This reduces the cognitive load to understand the overall configuration. -- Imports are done at the child pipeline level, reducing the likelihood of collisions. - -Child pipelines work well with other GitLab CI/CD features: - -- Use [`rules: changes`](../yaml/index.md#ruleschanges) to trigger pipelines only when - certain files change. This is useful for monorepos, for example. -- Since the parent pipeline in `.gitlab-ci.yml` and the child pipeline run as normal - pipelines, they can have their own behaviors and sequencing in relation to triggers. - -See the [`trigger`](../yaml/index.md#trigger) keyword documentation for full details on how to -include the child pipeline configuration. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8KpBSqZNbk). - -NOTE: -The artifact containing the generated YAML file must not be larger than 5MB. - -## Examples - -The simplest case is [triggering a child pipeline](../yaml/index.md#trigger) using a -local YAML file to define the pipeline configuration. In this case, the parent pipeline -triggers the child pipeline, and continues without waiting: - -```yaml -microservice_a: - trigger: - include: path/to/microservice_a.yml -``` - -You can include multiple files when defining a child pipeline. The child pipeline's -configuration is composed of all configuration files merged together: - -```yaml -microservice_a: - trigger: - include: - - local: path/to/microservice_a.yml - - template: Security/SAST.gitlab-ci.yml -``` - -In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/205157), -you can use [`include:file`](../yaml/index.md#includefile) to trigger child pipelines -with a configuration file in a different project: - -```yaml -microservice_a: - trigger: - include: - - project: 'my-group/my-pipeline-library' - ref: 'main' - file: '/path/to/child-pipeline.yml' -``` - -The maximum number of entries that are accepted for `trigger:include` is three. - -Similar to [multi-project pipelines](multi_project_pipelines.md#mirror-status-of-a-triggered-pipeline-in-the-trigger-job), -we can set the parent pipeline to depend on the status of the child pipeline upon completion: - -```yaml -microservice_a: - trigger: - include: - - local: path/to/microservice_a.yml - - template: Security/SAST.gitlab-ci.yml - strategy: depend -``` - -## Merge request child pipelines - -To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md) we need to: - -- Set the trigger job to run on merge requests: - -```yaml -# parent .gitlab-ci.yml -microservice_a: - trigger: - include: path/to/microservice_a.yml - rules: - - if: $CI_MERGE_REQUEST_ID -``` - -- Configure the child pipeline by either: - - - Setting all jobs in the child pipeline to evaluate in the context of a merge request: - - ```yaml - # child path/to/microservice_a.yml - workflow: - rules: - - if: $CI_MERGE_REQUEST_ID - - job1: - script: ... - - job2: - script: ... - ``` - - - Alternatively, setting the rule per job. For example, to create only `job1` in - the context of merge request pipelines: - - ```yaml - # child path/to/microservice_a.yml - job1: - script: ... - rules: - - if: $CI_MERGE_REQUEST_ID - - job2: - script: ... - ``` - -## Dynamic child pipelines - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9. - -Instead of running a child pipeline from a static YAML file, you can define a job that runs -your own script to generate a YAML file, which is then used to trigger a child pipeline. - -This technique can be very powerful in generating pipelines targeting content that changed or to -build a matrix of targets and architectures. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Create child pipelines using dynamically generated configurations](https://youtu.be/nMdfus2JWHM). - -We also have an example project using -[Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) -which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. -You could use a similar process for other templating languages like -[Dhall](https://dhall-lang.org/) or [ytt](https://get-ytt.io/). - -The artifact path is parsed by GitLab, not the runner, so the path must match the -syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows -runner for testing, the path separator for the trigger job would be `/`. Other CI/CD -configuration for jobs, like scripts, that use the Windows runner would use `\`. - -In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail. -This is [resolved](https://gitlab.com/gitlab-org/gitlab/-/issues/209070) in GitLab 12.10. - -### Dynamic child pipeline example - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9. - -You can trigger a child pipeline from a [dynamically generated configuration file](../pipelines/parent_child_pipelines.md#dynamic-child-pipelines): - -```yaml -generate-config: - stage: build - script: generate-ci-config > generated-config.yml - artifacts: - paths: - - generated-config.yml - -child-pipeline: - stage: test - trigger: - include: - - artifact: generated-config.yml - job: generate-config -``` - -The `generated-config.yml` is extracted from the artifacts and used as the configuration -for triggering the child pipeline. - -## Nested child pipelines - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29651) in GitLab 13.4. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243747) in GitLab 13.5. - -Parent and child pipelines were introduced with a maximum depth of one level of child -pipelines, which was later increased to two. A parent pipeline can trigger many child -pipelines, and these child pipelines can trigger their own child pipelines. It's not -possible to trigger another level of child pipelines. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Nested Dynamic Pipelines](https://youtu.be/C5j3ju9je2M). - -## Pass CI/CD variables to a child pipeline - -You can pass CI/CD variables to a downstream pipeline using the same methods as -multi-project pipelines: - -- [By using the `variable` keyword](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword). -- [By using variable inheritance](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). - -## Retry or cancel child pipelines - -You can retry or cancel child pipelines: - -- [In the main graph view](index.md#downstream-pipelines). -- In the child pipeline's details page. +<!-- This redirect file can be deleted after <2022-12-05>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index 3ff22a16900..a02ac7ba067 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -84,12 +84,14 @@ deploy_a: script: - echo "This job deploys something. It will only run when all jobs in the" - echo "test stage complete." + environment: production deploy_b: stage: deploy script: - echo "This job deploys something else. It will only run when all jobs in the" - echo "test stage complete. It will start at about the same time as deploy_a." + environment: production ``` ## Directed Acyclic Graph Pipelines @@ -151,18 +153,20 @@ deploy_a: script: - echo "Since build_a and test_a run quickly, this deploy job can run much earlier." - echo "It does not need to wait for build_b or test_b." + environment: production deploy_b: stage: deploy needs: [test_b] script: - echo "Since build_b and test_b run slowly, this deploy job will run much later." + environment: production ``` ## Child / Parent Pipelines In the examples above, it's clear we've got two types of things that could be built independently. -This is an ideal case for using [Child / Parent Pipelines](parent_child_pipelines.md)) via +This is an ideal case for using [Child / Parent Pipelines](downstream_pipelines.md#parent-child-pipelines)) via the [`trigger` keyword](../yaml/index.md#trigger). It separates out the configuration into multiple files, keeping things very simple. You can also combine this with: @@ -237,6 +241,7 @@ deploy_a: needs: [test_a] script: - echo "This job deploys something." + environment: production ``` Example child `b` pipeline configuration, located in `/b/.gitlab-ci.yml`, making @@ -266,6 +271,7 @@ deploy_b: needs: [test_b] script: - echo "This job deploys something else." + environment: production ``` It's also possible to set jobs to run before or after triggering child pipelines, diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index ad43895d7ef..72711f9b9dd 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -187,7 +187,7 @@ shouldn't run, saving pipeline resources. In a basic configuration, jobs always wait for all other jobs in earlier stages to complete before running. This is the simplest configuration, but it's also the slowest in most cases. [Directed Acyclic Graphs](../directed_acyclic_graph/index.md) and -[parent/child pipelines](parent_child_pipelines.md) are more flexible and can +[parent/child pipelines](downstream_pipelines.md#parent-child-pipelines) are more flexible and can be more efficient, but can also make pipelines harder to understand and analyze. ### Caching diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 8ab80e3798a..897caa340f9 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -27,7 +27,7 @@ Otherwise, the pipeline is not created. No error message is displayed. To add a pipeline schedule: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Schedules**. 1. Select **New schedule** and fill in the form. - **Interval Pattern**: Select one of the preconfigured intervals, or enter a custom @@ -45,7 +45,7 @@ To add a pipeline schedule: The owner of a pipeline schedule can edit it: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. In the left sidebar, select **CI/CD > Schedules**. 1. Next to the schedule, select **Edit** (**{pencil}**) and fill in the form. @@ -58,7 +58,7 @@ of the schedule. To trigger a pipeline schedule manually, so that it runs immediately instead of the next scheduled time: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Schedules**. 1. On the right of the list, for the pipeline you want to run, select **Play** (**{play}**). @@ -74,7 +74,7 @@ including [protected environments](../environments/protected_environments.md) an To take ownership of a pipeline created by a different user: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Schedules**. 1. On the right of the list, for the pipeline you want to become owner of, select **Take ownership**. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 34eae9828dd..d663dea7de8 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -25,7 +25,7 @@ For public and internal projects, you can change who can see your: To change the visibility of your pipelines and related features: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Select or clear the **Public pipelines** checkbox. @@ -57,7 +57,7 @@ This setting has no effect when: To change the pipeline visibility for non-project members: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. For **CI/CD**, choose: @@ -73,7 +73,7 @@ is selected. You can set pending or running pipelines to cancel automatically when a new pipeline runs on the same branch. You can enable this in the project settings: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General Pipelines**. 1. Select the **Auto-cancel redundant pipelines** checkbox. @@ -94,7 +94,7 @@ newer one, which may not be what you want. To avoid this scenario: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Select the **Skip outdated deployment jobs** checkbox. @@ -130,7 +130,7 @@ directory. However, you can specify an alternate filename path, including locati To customize the path: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **CI/CD configuration file** field, enter the filename. If the file: @@ -179,7 +179,7 @@ able to edit it. You can choose how your repository is fetched from GitLab when a job runs. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Under **Git strategy**, select an option: @@ -200,7 +200,7 @@ in the `.gitlab-ci.yml` file. You can limit the number of changes that GitLab CI/CD fetches when it clones a repository. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Under **Git strategy**, under **Git shallow clone**, enter a value. @@ -217,7 +217,7 @@ in the `.gitlab-ci.yml` file. You can define how long a job can run before it times out. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **Timeout** field, enter the number of minutes, or a human-readable value like `2 hours`. @@ -282,7 +282,7 @@ You can verify correct syntax using the [pipeline editor](../pipeline_editor/ind To migrate from the project coverage setting to the `coverage` keyword, use the regular expression displayed in the settings. Available in GitLab 14.10 and earlier: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. @@ -326,7 +326,7 @@ Use this regex for commonly used test tools. To see the evolution of your project code coverage over time, you can view a graph or download a CSV file with this data. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Repository**. The historic data for each job is listed in the dropdown above the graph. @@ -392,7 +392,7 @@ Support for [`semver`](https://semver.org/) sorting is tracked [in this issue](h You can view the exact link for your badges. Then you can embed the badge in your HTML or Markdown pages. -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **Pipeline status**, **Coverage report**, or **Latest release** sections, view the URLs for the images. diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index 0369824c92e..2b44cf3b898 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -109,6 +109,7 @@ To create a `.gitlab-ci.yml` file: stage: deploy script: - echo "This job deploys something from the $CI_COMMIT_BRANCH branch." + environment: production ``` `$GITLAB_USER_LOGIN` and `$CI_COMMIT_BRANCH` are diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index e76c4621a0c..dff52a742a8 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -171,6 +171,7 @@ deploy: include: deploy.gitlab-ci.yml strategy: depend resource_group: AWS-production + environment: production ``` ```yaml @@ -187,6 +188,7 @@ provision: deployment: stage: deploy script: echo "Deploying..." + environment: production ``` You must define [`strategy: depend`](../yaml/index.md#triggerstrategy) @@ -208,7 +210,7 @@ Read more how you can use GitLab for [safe deployments](../environments/deployme Because [`oldest_first` process mode](#process-modes) enforces the jobs to be executed in a pipeline order, there is a case that it doesn't work well with the other CI features. -For example, when you run [a child pipeline](../pipelines/parent_child_pipelines.md) +For example, when you run [a child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) that requires the same resource group with the parent pipeline, a dead lock could happen. Here is an example of a _bad_ setup: @@ -224,6 +226,7 @@ deploy: stage: deploy script: echo resource_group: production + environment: production ``` In a parent pipeline, it runs the `test` job that subsequently runs a child pipeline, @@ -250,4 +253,5 @@ deploy: stage: deploy script: echo resource_group: production + environment: production ``` diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 6dd03033926..9bafb69482b 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -76,7 +76,7 @@ Prerequisite: To use the Review Apps template: -1. On the top bar, select **Menu > Projects** and find the project you want to create a Review App job for. +1. On the top bar, select **Main menu > Projects** and find the project you want to create a Review App job for. 1. On the left sidebar, select **Deployments > Environments**. 1. Select **Enable Review Apps**. 1. Copy the provided code snippet and paste it into your diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 3efa697bf2f..9d26ec63f96 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -151,7 +151,7 @@ different places. To view the IP address of a shared runner you must have administrator access to the GitLab instance. To determine this: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Overview > Runners**. 1. Find the runner in the table and view the **IP Address** column. @@ -859,7 +859,7 @@ You can clean up group runners that have been inactive for more than three month Group runners are those that were created at the group level. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable stale runner cleanup** toggle. @@ -903,8 +903,8 @@ The version of GitLab Runner used by your runners should be To determine which runners need to be upgraded: 1. View the list of runners: - - For a group, on the top bar, select **Menu > Groups** and on the left sidebar, select **CI/CD > Runners**. - - For the instance, select **Menu > Admin** and on the left sidebar, select **Runners**. + - For a group, on the top bar, select **Main menu > Groups**, find your group, and on the left sidebar select **CI/CD > Runners**. + - For the instance, select **Main menu > Admin** and on the left sidebar, select **Runners**. 1. Above the list of runners, view the status: - **Outdated - recommended**: The runner does not have the latest `PATCH` version, which may make it vulnerable @@ -912,3 +912,43 @@ To determine which runners need to be upgraded: - **Outdated - available**: Newer versions are available but upgrading is not critical. 1. Filter the list by status to view which individual runners need to be upgraded. + +## Authentication token security + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 15.3 [with a flag](../../administration/feature_flags.md) named `enforce_runner_token_expires_at`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to +[enable the feature flag](../../administration/feature_flags.md) named `enforce_runner_token_expires_at`. +On GitLab.com, this feature is not available. + +Each runner has an [authentication token](../../api/runners.md#registration-and-authentication-tokens) +to connect with the GitLab instance. + +To help prevent the token from being compromised, you can have the +token rotate automatically at specified intervals. When the tokens are rotated, +they are updated for each runner, regardless of the runner's status (`online` or `offline`). + +No manual intervention should be required, and no running jobs should be affected. + +If you need to manually update the authentication token, you can run a +command to [reset the token](https://docs.gitlab.com/runner/commands/#gitlab-runner-reset-token). + +### Automatically rotate authentication tokens + +You can specify an interval for authentication tokens to rotate. +This rotation helps ensure the security of the tokens assigned to your runners. + +Prerequisites: + +- Ensure your runners are using [GitLab Runner 15.3 or later](https://docs.gitlab.com/runner/#gitlab-runner-versions). + +To automatically rotate runner authentication tokens: + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment** +1. Set a **Runners expiration** time for runners, leave empty for no expiration. +1. Select **Save**. + +Before the interval expires, runners automatically request a new authentication token. diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index f69d1f0f730..1de24efa8aa 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -7,8 +7,8 @@ type: reference # Runner SaaS **(FREE SAAS)** -If you use GitLab SaaS (GitLab.com), your CI jobs automatically run on runners provided by GitLab. -No configuration is required. Your jobs can run on: +If you use GitLab SaaS (GitLab.com), your [untagged](../../ci/runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) CI jobs automatically run in containers on the Linux Runners. +As long as shared runners are enabled for your project, no configuration is required. Your jobs can run on: - [Linux runners](saas/linux_saas_runner.md). - [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index 9bd0b52f423..f8a33ea6861 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -28,7 +28,7 @@ If you are using a self-managed instance of GitLab: going to your project's **Settings > CI/CD**, expanding **Runners**, and selecting **Show runner installation instructions**. These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html). -- The administrator can also configure a maximum number of shared runner +- The administrator can also configure a maximum number of shared runner [CI/CD minutes for each group](../pipelines/cicd_minutes.md#set-the-quota-of-cicd-minutes-for-a-specific-namespace). If you are using GitLab.com: @@ -51,7 +51,7 @@ For existing projects, an administrator must To enable shared runners for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable shared runners for this project** toggle. @@ -60,7 +60,7 @@ To enable shared runners for a project: To enable shared runners for a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable shared runners for this group** toggle. @@ -73,7 +73,7 @@ or group. To disable shared runners for a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Shared runners** area, turn off the **Enable shared runners for this project** toggle. @@ -87,7 +87,7 @@ Shared runners are automatically disabled for a project: To disable shared runners for a group: -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn off the **Enable shared runners for this group** toggle. @@ -170,7 +170,7 @@ You must have the Owner role for the group. To create a group runner: 1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/). -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **CI/CD > Runners**. 1. Note the URL and token. 1. [Register the runner](https://docs.gitlab.com/runner/register/). @@ -183,7 +183,7 @@ You can view and manage all runners for a group, its subgroups, and projects. You can do this for your self-managed GitLab instance or for GitLab.com. You must have the Owner role for the group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **CI/CD > Runners**. From this page, you can edit, pause, and remove runners from the group, its subgroups, and projects. @@ -193,7 +193,7 @@ From this page, you can edit, pause, and remove runners from the group, its subg You can pause or remove a group runner for your self-managed GitLab instance or for GitLab.com. You must have the Owner role for the group. -1. On the top bar, select **Menu > Groups** and find your group. +1. On the top bar, select **Main menu > Groups** and find your group. 1. On the left sidebar, select **CI/CD > Runners**. 1. Select **Pause** or **Remove runner**. - If you pause a group runner that is used by multiple projects, the runner pauses for all projects. @@ -229,7 +229,7 @@ Prerequisite: To create a specific runner: 1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/). -1. On the top bar, select **Menu > Projects** and find the project where you want to use the runner. +1. On the top bar, select **Main menu > Projects** and find the project where you want to use the runner. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Specific runners** section, note the URL and token. @@ -250,7 +250,7 @@ You must have at least the Maintainer role for: To enable a specific runner for a project: -1. On the top bar, select **Menu > Projects** and find the project where you want to enable the runner. +1. On the top bar, select **Main menu > Projects** and find the project where you want to enable the runner. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Specific runners** area, by the runner you want, select **Enable for this project**. @@ -269,7 +269,7 @@ but can also be changed later. To lock or unlock a specific runner: -1. On the top bar, select **Menu > Projects** and find the project where you want to enable the runner. +1. On the top bar, select **Main menu > Projects** and find the project where you want to enable the runner. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Runners**. 1. Find the specific runner you want to lock or unlock. Make sure it's enabled. You cannot lock shared or group runners. diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index e96e89b47e5..a7d1b8722a5 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -4,43 +4,97 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SaaS runners on Linux **(FREE SAAS)** +# SaaS runners on Linux -SaaS runners on Linux are autoscaled ephemeral Google Cloud Platform virtual machines. +When you run jobs on SaaS runners on Linux, the runners are on auto-scaled ephemeral virtual machine (VM) instances. +Each VM uses the Google Container-Optimized OS (COS) and the latest version of Docker Engine. +The default region for the VMs is `us-east1`. -Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available on GitLab.com. +## Machine types available for private projects (x86-64) -GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our [Open Source](https://about.gitlab.com/solutions/open-source/join/), [Education](https://about.gitlab.com/solutions/education/), and [Startups](https://about.gitlab.com/solutions/startups/) programs. For private projects, GitLab offers various [plans](https://about.gitlab.com/pricing/), starting with a Free tier. +For the SaaS runners on Linux, GitLab offers a range of machine types for use in private projects. +For Free, Premium, and Ultimate plan customers, jobs on these instances consume the CI/CD minutes allocated to your namespace. -All your CI/CD jobs run on [n1-standard-1 instances](https://cloud.google.com/compute/docs/machine-types) with 3.75GB of RAM, Google COS and the latest Docker Engine -installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default -region of the VMs is US East1. -Each instance is used only for one job. This ensures that any sensitive data left on the system can't be accessed by other people's CI/CD jobs. +| | Small | Medium | Large | +|-------------------|---------------------------|---------------------------|--------------------------| +| Specs | 1 vCPU, 3.75GB RAM | 2 vCPUs, 8GB RAM | 4 vCPUs, 16GB RAM | +| GitLab CI/CD tags | `saas-linux-small-amd64` | `saas-linux-medium-amd64` | `saas-linux-large-amd64` | +| Subscription | Free, Premium, Ultimate | Free, Premium, Ultimate | Premium, Ultimate | -NOTE: -The final disk space your jobs can use will be less than 25GB. Some disk space allocated to the instance will be occupied by the operating system, the Docker image, and a copy of your cloned repository. +The `small` machine type is the default. Your job runs on this machine type if you don't specify +a [tags:](../../yaml/index.md#tags) keyword in your `.gitlab-ci.yml` file. + +CI/CD jobs that run on `medium` and `large` machine types **will** consume CI minutes at a different rate than CI/CD jobs on the `small` machine type. + +Refer to the CI/CD minutes [cost factor](../../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the machine type based on size. + +## Example of how to tag a job -The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don't run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. +To use a machine type other than `small`, add a `tags:` keyword to your job. +For example: -Jobs handled by shared runners on GitLab.com (`shared-runners-manager-X.gitlab.com`) -**time out after 3 hours**, regardless of the timeout configured in a -project. Check issue [#4010](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4010) and [#4070](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4070) for the reference. +```yaml +stages: + - Prebuild + - Build + - Unit Test -Jobs handled by shared runners on Windows and macOS on GitLab.com **time out after 1 hour** while this service is in the [Beta](../../../policy/alpha-beta-support.md#beta-features) stage. +job_001: + stage: Prebuild + script: + - echo "this job runs on the default (small) instance" -Below are the runners' settings. +job_002: + tags: [ saas-linux-medium-amd64 ] + stage: Build + script: + - echo "this job runs on the medium instance" + + +job_003: + tags: [ saas-linux-large-amd64 ] + stage: Unit Test + script: + - echo "this job runs on the large instance" + +``` -| Setting | GitLab.com | Default | -| ----------- | ----------------- | ---------- | -| Executor | `docker+machine` | - | -| Default Docker image | `ruby:2.5` | - | -| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | +## SaaS runners for GitLab projects -These runners share a [distributed cache](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) through use of a Google Cloud Storage (GCS) bucket. Cache contents not updated within the last 14 days are automatically removed through use of an [object lifecycle management policy](https://cloud.google.com/storage/docs/lifecycle). +The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for +GitLab projects and related community forks. These runners are backed by a Google Compute +`n1-standard-2` machine type and do not run untagged jobs. Unlike the machine types used +for private projects, each virtual machine is re-used up to 40 times. + +## SaaS runners on Linux settings + +Below are the settings for SaaS runners on Linux. + +| Setting | GitLab.com | Default | +|-------------------------------------------------------------------------|------------------|---------| +| Executor | `docker+machine` | - | +| Default Docker image | `ruby:2.5` | - | +| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | + +- **Cache**: These runners share a + [distributed cache](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) + that's stored in a Google Cloud Storage (GCS) bucket. Cache contents not updated within + the last 14 days are automatically removed, based on the + [object lifecycle management policy](https://cloud.google.com/storage/docs/lifecycle). + +- **Timeout settings**: Jobs handled by the SaaS Runners on Linux + **time out after 3 hours**, regardless of the timeout configured in a + project. For details, see issues [#4010](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4010) + and [#4070](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4070). + +NOTE: +The final disk space your jobs can use will be less than 25GB. Some disk space +allocated to the instance will be occupied by the operating system, the Docker image, +and a copy of your cloned repository. ## Pre-clone script -With SaaS runners on Linux, you can run commands in a CI +With SaaS runners on Linux, you can run commands in a CI/CD job before the runner attempts to run `git init` and `git fetch` to download a GitLab repository. The [`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) @@ -55,12 +109,13 @@ To use this feature, define a [CI/CD variable](../../../ci/variables/index.md#cu `CI_PRE_CLONE_SCRIPT` that contains a bash script. NOTE: -The `CI_PRE_CLONE_SCRIPT` variable does not work on GitLab SaaS Windows or macOS Runners. +The `CI_PRE_CLONE_SCRIPT` variable does not work on GitLab SaaS Windows or macOS runners. ### Pre-clone script example This example was used in the `gitlab-org/gitlab` project until November 2021. -The project no longer uses this optimization because the [pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache) +The project no longer uses this optimization because the +[pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache) lets Gitaly serve the full CI/CD fetch traffic. See [Git fetch caching](../../../development/pipelines.md#git-fetch-caching). The `CI_PRE_CLONE_SCRIPT` was defined as a project CI/CD variable: diff --git a/doc/ci/secure_files/index.md b/doc/ci/secure_files/index.md index 8e141c62ef6..ff5c29cdb06 100644 --- a/doc/ci/secure_files/index.md +++ b/doc/ci/secure_files/index.md @@ -23,7 +23,7 @@ plain text and binary file types. You can manage secure files in the project settings, or with the [secure files API](../../api/secure_files.md). Secure files can be [downloaded and used by CI/CD jobs](#use-secure-files-in-cicd-jobs) -by using the [load-secure-files](https://gitlab.com/gitlab-org/incubation-engineering/devops-for-mobile-apps/load-secure-files) +by using the [load-secure-files](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/load-secure-files) tool. NOTE: @@ -34,7 +34,7 @@ Additional features and capabilities are planned. To add a secure file to a project: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. In the **Secure Files** section, select **Expand**. 1. Select **Upload File**. @@ -43,7 +43,7 @@ To add a secure file to a project: ## Use secure files in CI/CD jobs -To use your secure files in a CI/CD job, you must use the [`load-secure-files`](https://gitlab.com/gitlab-org/incubation-engineering/devops-for-mobile-apps/load-secure-files) +To use your secure files in a CI/CD job, you must use the [`load-secure-files`](https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/load-secure-files) tool to download the files in the job. After they are downloaded, you can use them with your other script commands. @@ -59,5 +59,5 @@ test: variables: SECURE_FILES_DOWNLOAD_PATH: './where/files/should/go/' script: - - curl --silent "https://gitlab.com/gitlab-org/incubation-engineering/devops-for-mobile-apps/load-secure-files/-/raw/main/installer" | bash + - curl --silent "https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/load-secure-files/-/raw/main/installer" | bash ``` diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index ba3f806c96e..9b2bcc39b3e 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -8,9 +8,12 @@ type: index # Services **(FREE)** -The `services` keyword defines a Docker image that runs during a `job` -linked to the Docker image that the image keyword defines. This allows -you to access the service image during build time. +When you configure CI/CD, you specify an image, which is used to create the container +where your jobs will run. To specify this image, you use the `image` keyword. + +You can specify an additional image by using the `services` keyword. This additional +image is used to create another container, which is available to the first container. +The two containers have access to one another and can communicate when running the job. The service image can run any application, but the most common use case is to run a database container, for example: diff --git a/doc/ci/testing/unit_test_reports.md b/doc/ci/testing/unit_test_reports.md index 6294996c703..28356a62c99 100644 --- a/doc/ci/testing/unit_test_reports.md +++ b/doc/ci/testing/unit_test_reports.md @@ -156,7 +156,7 @@ If parsing JUnit report XML results in an error, an indicator is shown next to t  -For test case parsing limits, see [Max test cases per unit test report](../../user/gitlab_com/#gitlab-cicd). +For test case parsing limits, see [Max test cases per unit test report](../../user/gitlab_com/index.md#gitlab-cicd). GitLab does not parse very [large nodes](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html#parse-options) of JUnit reports. There is [an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/268035) open to make this optional. diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index 2c916ba092c..36d136727b0 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -13,7 +13,7 @@ to the [pipeline triggers API endpoint](../../api/pipeline_triggers.md). When authenticating with the API, you can use: - A [trigger token](#create-a-trigger-token) to trigger a branch or tag pipeline. -- A [CI/CD job token](../jobs/ci_job_token.md) to trigger a [multi-project pipeline](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api). +- A [CI/CD job token](../jobs/ci_job_token.md) to [trigger a multi-project pipeline](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api). ## Create a trigger token @@ -26,7 +26,7 @@ Prerequisite: To create a trigger token: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Pipeline triggers**. 1. Enter a description and select **Add trigger**. @@ -88,6 +88,7 @@ trigger_pipeline: - 'curl --fail --request POST --form token=$MY_TRIGGER_TOKEN --form ref=main "https://gitlab.example.com/api/v4/projects/123456/trigger/pipeline"' rules: - if: $CI_COMMIT_TAG + environment: production ``` In this example: @@ -152,7 +153,7 @@ users with the Owner and Maintainer role can view the values. To revoke a trigger token: -1. On the top bar, select **Menu > Projects** and find your project. +1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Pipeline triggers**. 1. To the left of the trigger token you want to revoke, select **Revoke** (**{remove}**). @@ -169,7 +170,7 @@ To [configure when to run jobs](../jobs/job_control.md) in triggered pipelines: | `$CI_PIPELINE_SOURCE` value | `only`/`except` keywords | Trigger method | |-----------------------------|--------------------------|---------------------| | `trigger` | `triggers` | In pipelines triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using a [trigger token](#create-a-trigger-token). | -| `pipeline` | `pipelines` | In [multi-project pipelines](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api) triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using the [`$CI_JOB_TOKEN`](../jobs/ci_job_token.md), or by using the [`trigger`](../yaml/index.md#trigger) keyword in the CI/CD configuration file. | +| `pipeline` | `pipelines` | In [multi-project pipelines](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api) triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using the [`$CI_JOB_TOKEN`](../jobs/ci_job_token.md), or by using the [`trigger`](../yaml/index.md#trigger) keyword in the CI/CD configuration file. | Additionally, the `$CI_PIPELINE_TRIGGERED` predefined CI/CD variable is set to `true` in pipelines triggered with a trigger token. diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 0230aaf7113..8a03ec0b56f 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -40,15 +40,26 @@ is at: https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/editor/schema/ci.json. ``` -The schema rules for custom YAML tags like `!reference` will not work until the -custom tags are set in the editor settings. For example, in VS Code, you can set -`vscode-yaml` to parse `customTags`: - -```json -"yaml.customTags": [ - "!reference sequence" -] -``` +The schema rules for custom YAML tags like `!reference` are treated as invalid by your editor until the custom tags are +set in your editor settings. For example: + +- In VS Code, you can set `vscode-yaml` to parse `customTags` in your `settings.json` file: + + ```json + "yaml.customTags": [ + "!reference sequence" + ] + ``` + +- In Sublime Text, if you are using the `LSP-yaml` package, you can set `customTags` in your `LSP-yaml` user settings: + + ```json + { + "settings": { + "yaml.customTags": ["!reference sequence"] + } + } + ``` To see the full list of custom tags covered by the CI/CD schema, check the latest version of the schema, linked above. @@ -87,11 +98,11 @@ and [templates](examples/index.md#cicd-templates). Some pipeline types have their own detailed usage guides that you should read if you are using that type: -- [Multi-project pipelines](pipelines/multi_project_pipelines.md): Have your pipeline trigger +- [Multi-project pipelines](pipelines/downstream_pipelines.md#multi-project-pipelines): Have your pipeline trigger a pipeline in a different project. -- [Parent/child pipelines](pipelines/parent_child_pipelines.md): Have your main pipeline trigger +- [Parent/child pipelines](pipelines/downstream_pipelines.md#parent-child-pipelines): Have your main pipeline trigger and run separate pipelines in the same project. You can also - [dynamically generate the child pipeline's configuration](pipelines/parent_child_pipelines.md#dynamic-child-pipelines) + [dynamically generate the child pipeline's configuration](pipelines/downstream_pipelines.md#dynamic-child-pipelines) at runtime. - [Merge request pipelines](pipelines/merge_request_pipelines.md): Run a pipeline in the context of a merge request. @@ -259,7 +270,7 @@ are enabled, the button is either **Add to merge train** or **Add to merge train #### "A CI/CD pipeline must run and be successful before merge" message -This message is shown if the [Pipelines must succeed](../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +This message is shown if the [Pipelines must succeed](../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) setting is enabled in the project and a pipeline has not yet run successfully. This also applies if the pipeline has not been created yet, or if you are waiting for an external CI service. If you don't use pipelines for your project, then you @@ -316,11 +327,16 @@ To reduce the configuration size, you can: [merged YAML](pipeline_editor/index.md#view-expanded-configuration) tab. Look for duplicated configuration that can be removed or simplified. - Move long or repeated `script` sections into standalone scripts in the project. -- Use [parent and child pipelines](pipelines/parent_child_pipelines.md) to move some +- Use [parent and child pipelines](pipelines/downstream_pipelines.md#parent-child-pipelines) to move some work to jobs in an independent child pipeline. On a self-managed instance, you can [increase the size limits](../administration/instance_limits.md#maximum-size-and-depth-of-cicd-configuration-yaml-files). +### Error 500 when editing the `.gitlab-ci.yml` file + +A [loop of included configuration files](pipeline_editor/index.md#configuration-validation-currently-not-available-message) +can cause a `500` error when editing the `.gitlab-ci.yml` file with the [web editor](../user/project/repository/web_editor.md). + ## Pipeline warnings Pipeline configuration warnings are shown when you: diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 72df8d56815..25178903c9a 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -239,7 +239,7 @@ You can define instance variables via the UI or [API](../../api/instance_level_c To add an instance variable: -1. On the top bar, select **Menu > Admin**. +1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD** and expand the **Variables** section. 1. Select the **Add variable** button, and fill in the details: @@ -592,6 +592,7 @@ deploy: BUILD_VARIABLE: value_from_deploy_job script: - echo "$BUILD_VARIABLE" # Output is: 'value_from_build_job' due to precedence + environment: production ``` The [`dependencies`](../yaml/index.md#dependencies) or @@ -616,12 +617,19 @@ deploy_one: - echo "$BUILD_VERSION" # Output is: 'hello' dependencies: - build + environment: + name: customer1 + deployment_tier: production + deploy_two: stage: deploy script: - echo "$BUILD_VERSION" # Output is empty dependencies: [] + environment: + name: customer2 + deployment_tier: production deploy_three: stage: deploy @@ -629,6 +637,10 @@ deploy_three: - echo "$BUILD_VERSION" # Output is: 'hello' needs: - build + environment: + name: customer3 + deployment_tier: production + deploy_four: stage: deploy @@ -637,6 +649,9 @@ deploy_four: needs: job: build artifacts: true + environment: + name: customer4 + deployment_tier: production deploy_five: stage: deploy @@ -645,9 +660,12 @@ deploy_five: needs: job: build artifacts: false + environment: + name: customer5 + deployment_tier: production ``` -[Multi-project pipelines](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance) +[Multi-project pipelines](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job) can also inherit variables from their upstream pipelines. ## CI/CD variable precedence @@ -695,8 +713,8 @@ You can override the value of a variable when you: 1. Run a job manually in the UI. 1. Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). 1. Trigger a pipeline by using [the API](../triggers/index.md#pass-cicd-variables-in-the-api-call). -1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword) - or [by using variable inheritance](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). +1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/downstream_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline) + or [by using variable inheritance](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job). The pipeline variables declared in these events take [priority over other variables](#cicd-variable-precedence). diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index f8900501244..77005e1cec4 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -66,8 +66,8 @@ as it can cause the pipeline to behave unexpectedly. | `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. | | `CI_JOB_JWT` | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | | `CI_JOB_JWT_V1` | 14.6 | all | The same value as `CI_JOB_JWT`. | -| `CI_JOB_JWT_V2` | 14.6 | all | [**alpha:**](../../policy/alpha-beta-support.md#alpha-features) A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). | -| `CI_JOB_MANUAL` | 8.12 | all | `true` if a job was started manually. | +| `CI_JOB_JWT_V2` | 14.6 | all | A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). **Note:** The `CI_JOB_JWT_V2` variable is available for testing, but the full feature is planned to be generally available when [issue 360657](https://gitlab.com/gitlab-org/gitlab/-/issues/360657) is complete.| +| `CI_JOB_MANUAL` | 8.12 | all | Only available if the job was started manually. `true` when available. | | `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job. | | `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the job's stage. | | `CI_JOB_STATUS` | all | 13.5 | The status of the job as each runner stage is executed. Use with [`after_script`](../yaml/index.md#after_script). Can be `success`, `failed`, or `canceled`. | @@ -117,6 +117,9 @@ as it can cause the pipeline to behave unexpectedly. | `CI_SERVER_PORT` | 12.8 | all | The port of the GitLab instance URL, without host or protocol. For example `8080`. | | `CI_SERVER_PROTOCOL` | 12.8 | all | The protocol of the GitLab instance URL, without host or port. For example `https`. | | `CI_SERVER_REVISION` | all | all | GitLab revision that schedules jobs. | +| `CI_SERVER_TLS_CA_FILE` | all | all | File containing the TLS CA certificate to verify the GitLab server when `tls-ca-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | +| `CI_SERVER_TLS_CERT_FILE` | all | all | File containing the TLS certificate to verify the GitLab server when `tls-cert-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | +| `CI_SERVER_TLS_KEY_FILE` | all | all | File containing the TLS key to verify the GitLab server when `tls-key-file` set in [runner settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | | `CI_SERVER_URL` | 12.7 | all | The base URL of the GitLab instance, including protocol and port. For example `https://gitlab.example.com:8080`. | | `CI_SERVER_VERSION_MAJOR` | 11.4 | all | The major version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MAJOR` is `13`. | | `CI_SERVER_VERSION_MINOR` | 11.4 | all | The minor version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MINOR` is `6`. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index db83f2a14f3..6c1737f7c65 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -30,9 +30,10 @@ There are two places defined variables can be used. On the: | [`cache:key`](../yaml/index.md#cachekey) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | | [`environment:name`](../yaml/index.md#environmentname) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | [`environment:url`](../yaml/index.md#environmenturl) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | +| [`environment:auto_stop_in`](../yaml/index.md#environmentauto_stop_in)| yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/> The value of the variable being substituted should be a period of time in a human readable natural language form. See [possible inputs](../yaml/index.md#environmentauto_stop_in) for more information.| | [`except:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | [`image`](../yaml/index.md#image) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | -| [`include`](../yaml/index.md#include) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>Predefined project variables are supported: `GITLAB_FEATURES`, `CI_DEFAULT_BRANCH`, and all variables that start with `CI_PROJECT_` (for example `CI_PROJECT_NAME`). | +| [`include`](../yaml/index.md#include) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>See [Use variables with include](../yaml/includes.md#use-variables-with-include) for more information on supported variables. | | [`only:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | [`resource_group`](../yaml/index.md#resource_group) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/>- `CI_ENVIRONMENT_URL`<br/>- [Persisted variables](#persisted-variables). | | [`rules:if`](../yaml/index.md#rulesif) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 61ef8bbfab7..56a9da7cb84 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -80,20 +80,6 @@ GitLab can display the results of one or more reports in: - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). -<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> - -## `artifacts:reports:cobertura` (removed) - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.7. -> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. - -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.7 and -[removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. Use `artifacts:reports:coverage_report` -instead. - -<!--- end_remove --> - ## `artifacts:reports:coverage_report` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344533) in GitLab 14.10. @@ -113,7 +99,8 @@ artifacts: path: coverage/cobertura-coverage.xml ``` -The collected coverage report is uploaded to GitLab as an artifact. +The collected coverage report is uploaded to GitLab as an artifact. You can use +only one report per job. GitLab can display the results of coverage report in the merge request [diff annotations](../testing/test_coverage_visualization.md). @@ -160,6 +147,30 @@ GitLab can display the results of one or more reports in: - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). - The [security dashboard](../../user/application_security/security_dashboard/index.md). +## `artifacts:reports:cyclonedx` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360766) in GitLab 15.3 + +This report is a Software Bill of Materials describing the components of a project +following the [cyclonedx](https://cyclonedx.org/docs/1.4) protocol format. + +You can specify multiple cyclonedx reports per job. These can be either supplied +as a list of filenames, a filename pattern, or both: + +- List of filenames: `cyclonedx: [gl-sbom-npm-npm.cdx.json, gl-sbom-bundler-gem.cdx.json]`. +- A filename pattern: `cyclonedx: gl-sbom-*.json`. +- Combination of both of the above: `cyclonedx: [gl-sbom-*.json, my-cyclonedx.json]`. + +Below is an example of a job exposing cyclonedx artifacts: + +```yaml +artifacts: + reports: + cyclonedx: + - gl-sbom-npm-npm.cdx.json + - gl-sbom-bundler-gem.cdx.json +``` + ## `artifacts:reports:dast` **(ULTIMATE)** The `dast` report collects [DAST vulnerabilities](../../user/application_security/dast/index.md). The collected DAST @@ -183,7 +194,7 @@ GitLab can display the results of one or more reports in: - The pipeline [**Security** tab](../../user/application_security/vulnerability_report/pipeline.md#view-vulnerabilities-in-a-pipeline). - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). -- The [dependency list](../../user/application_security/dependency_list/). +- The [dependency list](../../user/application_security/dependency_list/index.md). ## `artifacts:reports:dotenv` @@ -329,27 +340,3 @@ GitLab can display the results of one or more reports in the merge request [terraform widget](../../user/infrastructure/iac/mr_integration.md#output-terraform-plan-information-into-a-merge-request). For more information, see [Output `terraform plan` information into a merge request](../../user/infrastructure/iac/mr_integration.md). - -## `artifacts:reports:cyclonedx` - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360766) in GitLab 15.3 - -This report is a Software Bill of Materials describing the components of a project -following the [cyclonedx](https://cyclonedx.org/docs/1.4) protocol format. - -You can specify multiple cyclonedx reports per job. These can be either supplied -as a list of filenames, a filename pattern, or both: - -- List of filenames: `cyclonedx: [gl-sbom-npm-npm.cdx.json, gl-sbom-bundler-gem.cdx.json]`. -- A filename pattern: `cyclonedx: gl-sbom-*.json`. -- Combination of both of the above: `cyclonedx: [gl-sbom-*.json, my-cyclonedx.json]`. - -Below is an example of a job exposing cyclonedx artifacts: - -```yaml -artifacts: - reports: - cyclonedx: - - gl-sbom-npm-npm.cdx.json - - gl-sbom-bundler-gem.cdx.json -``` diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 0b434e7013c..8aa3ebd4759 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -156,6 +156,12 @@ the time limit to resolve all files is 30 seconds. - You can override included configuration by having the same job name or global keyword in the `.gitlab-ci.yml` file. The two configurations are merged together, and the configuration in the `.gitlab-ci.yml` file takes precedence over the included configuration. +- If you rerun a: + - Job, the `include` files are not fetched again. All jobs in a pipeline use the configuration + fetched when the pipeline was created. Any changes to the source `include` files + do not affect job reruns. + - Pipeline, the `include` files are fetched again. If they changed after the last + pipeline run, the new pipeline uses the changed configuration. **Related topics**: @@ -607,6 +613,7 @@ job3: stage: deploy script: - deploy_to_staging + environment: staging ``` In this example, `job1` and `job2` run in parallel: @@ -1384,7 +1391,7 @@ In this example: for the coverage number. - If there are multiple coverage numbers found in the matched fragment, the first number is used. - Leading zeros are removed. -- Coverage output from [child pipelines](../pipelines/parent_child_pipelines.md) +- Coverage output from [child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines) is not recorded or displayed. Check [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/280818) for more details. @@ -1478,6 +1485,7 @@ test linux: deploy: stage: deploy script: make deploy + environment: production ``` In this example, two jobs have artifacts: `build osx` and `build linux`. When `test osx` is executed, @@ -1632,6 +1640,7 @@ these are all equivalent: - `168 hours` - `7 days` - `one week` +- `never` **Example of `environment:auto_stop_in`**: @@ -1896,13 +1905,10 @@ image: #### `image:pull_policy` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21619) in GitLab 15.1 [with a flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.2. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.4. [Feature flag `ci_docker_image_pull_policy`](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) removed. > - Requires GitLab Runner 15.1 or later. -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. -The feature is not ready for production use. - The pull policy that the runner uses to fetch the Docker image. **Keyword type**: Job keyword. You can use it only as part of a job or in the [`default` section](#default). @@ -2123,6 +2129,7 @@ mac:rspec: production: stage: deploy script: echo "Running production..." + environment: production ``` This example creates four paths of execution: @@ -2286,14 +2293,14 @@ build_job: **Related topics**: -- To download artifacts between [parent-child pipelines](../pipelines/parent_child_pipelines.md), +- To download artifacts between [parent-child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines), use [`needs:pipeline:job`](#needspipelinejob). #### `needs:pipeline:job` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255983) in GitLab 13.7. -A [child pipeline](../pipelines/parent_child_pipelines.md) can download artifacts from a job in +A [child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) can download artifacts from a job in its parent pipeline or another child pipeline in the same parent-child pipeline hierarchy. **Keyword type**: Job keyword. You can use it only as part of a job. @@ -2385,12 +2392,14 @@ deploy-job: - job: test-job2 optional: true - job: test-job1 + environment: production review-job: stage: deploy needs: - job: test-job2 optional: true + environment: review ``` In this example: @@ -2472,7 +2481,7 @@ when to add jobs to pipelines. | `external` | When you use CI services other than GitLab. | | `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). | | `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/merged_results_pipelines.md), and [merge trains](../pipelines/merge_trains.md). | - | `pipelines` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](#trigger) keyword. | + | `pipelines` | For [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines) created by [using the API with `CI_JOB_TOKEN`](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api), or the [`trigger`](#trigger) keyword. | | `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedules` | For [scheduled pipelines](../pipelines/schedules.md). | | `tags` | When the Git reference for a pipeline is a tag. | @@ -2620,7 +2629,7 @@ docker build: **Related topics**: - [`only: changes` and `except: changes` examples](../jobs/job_control.md#onlychanges--exceptchanges-examples). -- If you use `changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds), +- If you use `changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge), you should [also use `only:merge_requests`](../jobs/job_control.md#use-onlychanges-with-merge-request-pipelines). - [Jobs or pipelines can run unexpectedly when using `only: changes`](../jobs/job_control.md#jobs-or-pipelines-run-unexpectedly-when-using-changes). @@ -2671,6 +2680,7 @@ pages: - public rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + environment: production ``` This example moves all files from the root of the project to the `public/` directory. @@ -2752,6 +2762,7 @@ deploystacks: STACK: [monitoring, backup, app] - PROVIDER: [gcp, vultr] STACK: [data, processing] + environment: $PROVIDER/$STACK ``` The example generates 10 parallel `deploystacks` jobs, each with different values @@ -3224,8 +3235,7 @@ job: - Unlike variables in [`script`](../variables/index.md#use-cicd-variables-in-job-scripts) sections, variables in rules expressions are always formatted as `$VARIABLE`. - You can use `rules:if` with `include` to [conditionally include other configuration files](includes.md#use-rules-with-include). -- In [GitLab 15.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/35438), - variables on the right side of `=~` and `!~` expressions are evaluated as regular expressions. +- CI/CD variables on the right side of `=~` and `!~` expressions are [evaluated as regular expressions](../jobs/job_control.md#store-the-regex-pattern-in-a-variable). **Related topics**: @@ -3643,12 +3653,9 @@ in that container. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21619) in GitLab 15.1 [with a flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.2. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) in GitLab 15.4. [Feature flag `ci_docker_image_pull_policy`](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) removed. > - Requires GitLab Runner 15.1 or later. -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. - The pull policy that the runner uses to fetch the Docker image. **Keyword type**: Job keyword. You can use it only as part of a job or in the [`default` section](#default). @@ -3725,6 +3732,7 @@ job4: stage: deploy script: - echo "This job deploys the code. It runs when the test stage completes." + environment: production ``` **Additional details**: @@ -3881,54 +3889,46 @@ test: ### `trigger` -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8997) in GitLab Premium 11.8. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. +Use `trigger` to declare that a job is a "trigger job" which starts a +[downstream pipeline](../pipelines/downstream_pipelines.md) that is either: + +- [A multi-project pipeline](../pipelines/downstream_pipelines.md#multi-project-pipelines). +- [A child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines). -Use `trigger` to start a downstream pipeline that is either: +Trigger jobs can use only a limited set of GitLab CI/CD configuration keywords. +The keywords available for use in trigger jobs are: -- [A multi-project pipeline](../pipelines/multi_project_pipelines.md). -- [A child pipeline](../pipelines/parent_child_pipelines.md). +- [`trigger`](#trigger). +- [`stage`](#stage). +- [`allow_failure`](#allow_failure). +- [`rules`](#rules). +- [`only` and `except`](#only--except). +- [`when`](#when) (only with a value of `on_success`, `on_failure`, or `always`). +- [`extends`](#extends). +- [`needs`](#needs), but not [`needs:project`](#needsproject). **Keyword type**: Job keyword. You can use it only as part of a job. **Possible inputs**: -- For multi-project pipelines, path to the downstream project. CI/CD variables - [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) - in GitLab 15.3 and later. -- For child pipelines, path to the child pipeline CI/CD configuration file. +- For multi-project pipelines, the path to the downstream project. CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) + in GitLab 15.3 and later, but not [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables). + Alternatively, use [`trigger:project](#triggerproject). +- For child pipelines, use [`trigger:include`](#triggerinclude). -**Example of `trigger` for multi-project pipeline**: +**Example of `trigger`**: ```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - stage: deploy - trigger: my/deployment -``` - -**Example of `trigger` for child pipelines**: - -```yaml -trigger_job: - trigger: - include: path/to/child-pipeline.yml +trigger-multi-project-pipeline: + trigger: my-group/my-project ``` **Additional details**: -- Jobs with `trigger` can only use a [limited set of keywords](../pipelines/multi_project_pipelines.md#define-multi-project-pipelines-in-your-gitlab-ciyml-file). - For example, you can't run commands with [`script`](#script), [`before_script`](#before_script), - or [`after_script`](#after_script). Also, [`environment`](#environment) is not supported with `trigger`. - You [cannot use the API to start `when:manual` trigger jobs](https://gitlab.com/gitlab-org/gitlab/-/issues/284086). - In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201938), you can use [`when:manual`](#when) in the same job as `trigger`. In GitLab 13.4 and earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. -- In [GitLab 13.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/197140/), you can - view which job triggered a downstream pipeline in the [pipeline graph](../pipelines/index.md#visualize-pipelines). - [Manual pipeline variables](../variables/index.md#override-a-defined-cicd-variable) and [scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule) are not passed to downstream pipelines by default. Use [trigger:forward](#triggerforward) @@ -3938,12 +3938,75 @@ trigger_job: **Related topics**: -- [Multi-project pipeline configuration examples](../pipelines/multi_project_pipelines.md#define-multi-project-pipelines-in-your-gitlab-ciyml-file). -- [Child pipeline configuration examples](../pipelines/parent_child_pipelines.md#examples). +- [Multi-project pipeline configuration examples](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-from-a-job-in-your-gitlab-ciyml-file). - To run a pipeline for a specific branch, tag, or commit, you can use a [trigger token](../triggers/index.md) to authenticate with the [pipeline triggers API](../../api/pipeline_triggers.md). The trigger token is different than the `trigger` keyword. +#### `trigger:include` + +Use `trigger:include` to declare that a job is a "trigger job" which starts a +[child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines). + +Use `trigger:include:artifact` to trigger a [dynamic child pipeline](../pipelines/downstream_pipelines.md#dynamic-child-pipelines). + +**Keyword type**: Job keyword. You can use it only as part of a job. + +**Possible inputs**: + +- The path to the child pipeline's configuration file. + +**Example of `trigger:include`**: + +```yaml +trigger-child-pipeline: + trigger: + include: path/to/child-pipeline.gitlab-ci.yml +``` + +**Related topics**: + +- [Child pipeline configuration examples](../pipelines/downstream_pipelines.md#trigger-a-parent-child-pipeline). + +#### `trigger:project` + +Use `trigger:project` to declare that a job is a "trigger job" which starts a +[multi-project pipeline](../pipelines/downstream_pipelines.md#multi-project-pipelines). + +By default, the multi-project pipeline triggers for the default branch. Use `trigger:branch` +to specify a different branch. + +**Keyword type**: Job keyword. You can use it only as part of a job. + +**Possible inputs**: + +- The path to the downstream project. CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) + in GitLab 15.3 and later, but not [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables). + +**Example of `trigger:project`**: + +```yaml +trigger-multi-project-pipeline: + trigger: + project: my-group/my-project +``` + +**Example of `trigger:project` for a different branch**: + +```yaml +trigger-multi-project-pipeline: + trigger: + project: my-group/my-project + branch: development +``` + +**Related topics**: + +- [Multi-project pipeline configuration examples](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-from-a-job-in-your-gitlab-ciyml-file). +- To run a pipeline for a specific branch, tag, or commit, you can also use a [trigger token](../triggers/index.md) + to authenticate with the [pipeline triggers API](../../api/pipeline_triggers.md). + The trigger token is different than the `trigger` keyword. + #### `trigger:strategy` Use `trigger:strategy` to force the `trigger` job to wait for the downstream pipeline to complete @@ -3984,8 +4047,8 @@ successfully complete before starting. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/355572) in GitLab 15.1. [Feature flag `ci_trigger_forward_variables`](https://gitlab.com/gitlab-org/gitlab/-/issues/355572) removed. Use `trigger:forward` to specify what to forward to the downstream pipeline. You can control -what is forwarded to both [parent-child pipelines](../pipelines/parent_child_pipelines.md) -and [multi-project pipelines](../pipelines/multi_project_pipelines.md). +what is forwarded to both [parent-child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines) +and [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines). **Possible inputs**: @@ -4062,6 +4125,7 @@ deploy_job: stage: deploy script: - deploy-script --url $DEPLOY_SITE --path "/" + environment: production deploy_review_job: stage: deploy @@ -4069,6 +4133,7 @@ deploy_review_job: REVIEW_PATH: "/review" script: - deploy-review-script --url $DEPLOY_SITE --path $REVIEW_PATH + environment: production ``` **Additional details**: @@ -4161,6 +4226,7 @@ deploy_job: script: - make deploy when: manual + environment: production cleanup_job: stage: cleanup @@ -4194,20 +4260,6 @@ In this example, the script: The following keywords are deprecated. -<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> - -### Globally-defined `types` (removed) - -The `types` keyword was deprecated in GitLab 9.0, and [removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). -Use [`stages`](#stages) instead. - -### Job-defined `type` (removed) - -The `type` keyword was deprecated in GitLab 9.0, and [removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). -Use [`stage`](#stage) instead. - -<!--- end_remove --> - ### Globally-defined `image`, `services`, `cache`, `before_script`, `after_script` Defining `image`, `services`, `cache`, `before_script`, and diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index f1cdcf57e64..bd8d7f02a17 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -244,6 +244,7 @@ pages-job: stage: deploy script: - curl --header 'PRIVATE-TOKEN: ${PRIVATE_TOKEN}' "https://gitlab.example.com/api/v4/projects" + environment: production ``` The YAML parser thinks the `:` defines a YAML keyword, and outputs the @@ -257,6 +258,7 @@ pages-job: stage: deploy script: - 'curl --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "https://gitlab.example.com/api/v4/projects"' + environment: production ``` ### Job does not fail when using `&&` in a script diff --git a/doc/ci/yaml/workflow.md b/doc/ci/yaml/workflow.md index 743a2639c0c..d3e815c742d 100644 --- a/doc/ci/yaml/workflow.md +++ b/doc/ci/yaml/workflow.md @@ -176,7 +176,7 @@ include: If a merge request displays `Checking pipeline status.`, but the message never goes away (the "spinner" never stops spinning), it might be due to `workflow:rules`. -This issue can happen if a project has [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +This issue can happen if a project has [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) enabled, but the `workflow:rules` prevent a pipeline from running for the merge request. For example, with this workflow, merge requests cannot be merged, because no |
