diff options
Diffstat (limited to 'doc/ci')
40 files changed, 975 insertions, 39 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 0ff12f16d9d..bd29824781b 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -48,7 +48,7 @@ into more features: | [Introduction to pipelines and jobs](pipelines.md) | Provides an overview of GitLab CI/CD and jobs. | | [CI/CD Variables](variables/README.md) | How environment variables can be configured and made available in pipelines. | | [Where variables can be used](variables/where_variables_can_be_used.md) | A deeper look into where and how CI/CD variables can be used. | -| [User](../user/permissions.md#gitlab-cicd-permissions) and [job](../user/permissions.md#job-permissions) permissions | Learn about the access levels a user can have for performing certain CI actions. | +| [User](../user/permissions.md#gitlab-cicd-permissions) and [job](../user/permissions.md#job-permissions) permissions | Learn about the access levels a user can have for performing certain CI actions. | | [Configuring GitLab Runners](runners/README.md) | Documentation for configuring [GitLab Runner](https://docs.gitlab.com/runner/). | | [Introduction to environments and deployments](environments.md) | Learn how to separate your jobs into environments and use them for different purposes like testing, building and, deploying. | | [Job artifacts](../user/project/pipelines/job_artifacts.md) | Learn about the output of jobs. | @@ -61,6 +61,10 @@ into more features: | [Connecting GitLab with a Kubernetes cluster](../user/project/clusters/index.md) | Integrate one or more Kubernetes clusters to your project. | | [ChatOps](chatops/README.md) | Trigger CI jobs from chat, with results sent back to the channel. | | [Interactive web terminals](interactive_web_terminal/index.md) | Open an interactive web terminal to debug the running jobs. | +| [Review Apps](review_apps/index.md) | Configure GitLab CI/CD to preview code changes in a per-branch basis. | +| [Deploy Boards](../user/project/deploy_boards.md) **[PREMIUM]** | Check the current health and status of each CI/CD environment running on Kubernetes. | +| [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) **[PREMIUM]** | Get the benefits of GitLab CI/CD combined with repositories in GitHub and BitBucket Cloud. | +| [Protected environments](environments/protected_environments.md) **[PREMIUM]** | Ensure that only people with the right privileges can deploy to an environment. | ### GitLab Pages diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md new file mode 100644 index 00000000000..b20bf1ced7e --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -0,0 +1,147 @@ +# Using GitLab CI/CD with a Bitbucket Cloud repository **[PREMIUM]** + +GitLab CI/CD can be used with Bitbucket Cloud by creating a +[CI/CD project](../../user/project/ci_cd_for_external_repo.md) and connecting +your Git repository via URL. + +1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and + create the project. + +  + + GitLab will import the repository and enable [Pull Mirroring][pull-mirroring]. + +1. In GitLab create a + [Personal Access Token](../../user/profile/personal_access_tokens.md) + with `api` scope. This will be used to authenticate requests from the web + hook that will be created in Bitbucket to notify GitLab of new commits. + +1. In Bitbucket from **Settings > Webhooks** create a new web hook to notify + GitLab of new commits. + + The web hook URL should be set to the GitLab API to trigger pull mirroring, + using the Personal Access Token we just generated for authentication. + + ``` + https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> + ``` + + The web hook Trigger should be set to 'Repository Push'. + +  + + After saving, test the web hook by pushing a change to your Bitbucket + repository. + +1. In Bitbucket create an **App Password** from **Bitbucket Settings > App + Passwords** to authenticate the build status script setting commit build + statuses in Bitbucket. Repository write permissions are required. + +  + +1. In GitLab from **Settings > CI/CD > Environment variables** add variables to allow + communication with Bitbucket via the Bitbucket API. + + `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above + + `BITBUCKET_USERNAME`: the username of the Bitbucket account + + `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ + + `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ + +1. In Bitbucket add a script to push the pipeline status to Bitbucket. + + > Note: changes made in GitLab will be overwritten by any changes made + upstream in Bitbucket. + + Create a file `build_status` and insert the script below and run + `chmod +x build_status` in your terminal to make the script executable. + + ```bash + #!/usr/bin/env bash + + # Push GitLab CI/CD build status to Bitbucket Cloud + + if [ -z "$BITBUCKET_ACCESS_TOKEN" ]; then + echo "ERROR: BITBUCKET_ACCESS_TOKEN is not set" + exit 1 + fi + if [ -z "$BITBUCKET_USERNAME" ]; then + echo "ERROR: BITBUCKET_USERNAME is not set" + exit 1 + fi + if [ -z "$BITBUCKET_NAMESPACE" ]; then + echo "Setting BITBUCKET_NAMESPACE to $CI_PROJECT_NAMESPACE" + BITBUCKET_NAMESPACE=$CI_PROJECT_NAMESPACE + fi + if [ -z "$BITBUCKET_REPOSITORY" ]; then + echo "Setting BITBUCKET_REPOSITORY to $CI_PROJECT_NAME" + BITBUCKET_REPOSITORY=$CI_PROJECT_NAME + fi + + BITBUCKET_API_ROOT="https://api.bitbucket.org/2.0" + BITBUCKET_STATUS_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/commit/$CI_COMMIT_SHA/statuses/build" + BITBUCKET_KEY="ci/gitlab-ci/$CI_JOB_NAME" + + case "$BUILD_STATUS" in + running) + BITBUCKET_STATE="INPROGRESS" + BITBUCKET_DESCRIPTION="The build is running!" + ;; + passed) + BITBUCKET_STATE="SUCCESSFUL" + BITBUCKET_DESCRIPTION="The build passed!" + ;; + failed) + BITBUCKET_STATE="FAILED" + BITBUCKET_DESCRIPTION="The build failed." + ;; + esac + + echo "Pushing status to $BITBUCKET_STATUS_API..." + curl --request POST $BITBUCKET_STATUS_API \ + --user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \ + --header "Content-Type:application/json" \ + --silent \ + --data "{ \"state\": \"$BITBUCKET_STATE\", \"key\": \"$BITBUCKET_KEY\", \"description\": + \"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }" + ``` + +1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push + pipeline success and failures to Bitbucket. + + ``` + stages: + - test + - ci_status + + unit-tests: + script: + - echo "Success. Add your tests!" + + success: + stage: ci_status + before_script: + - "" + after_script: + - "" + script: + - BUILD_STATUS=passed BUILD_KEY=push ./build_status + when: on_success + + failure: + stage: ci_status + before_script: + - "" + after_script: + - "" + script: + - BUILD_STATUS=failed BUILD_KEY=push ./build_status + when: on_failure + ``` + +GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines +configured in `.gitlab-ci.yml` and push the status to Bitbucket. + +[pull-mirroring]: ../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md new file mode 100644 index 00000000000..1a3bb87514f --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -0,0 +1,111 @@ +# Using GitLab CI/CD with a GitHub repository **[PREMIUM]** + +GitLab CI/CD can be used with **GitHub.com** and **GitHub Enterprise** by +creating a [CI/CD project](../../user/project/ci_cd_for_external_repo.md) to connect your GitHub repository to +GitLab. + +NOTE: **Note:** +To use **GitHub Enterprise** with **GitLab.com** you should use the +manual method. + +## Connect with GitHub integration + +If the [GitHub integration](../../integration/github.md) has been enabled by your GitLab +administrator: + +NOTE: **Note:** +Due to a 10-token limitation on the [GitHub OAuth Implementation](https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/#creating-multiple-tokens-for-oauth-apps), +if you import more than 10 times, your oldest imported project's token will be +revoked. See issue [#9147](https://gitlab.com/gitlab-org/gitlab-ee/issues/9147) +for more information. + +1. In GitLab create a **CI/CD for external repo** project and select + **GitHub**. + +  + +1. Once authenticated, you will be redirected to a list of your repositories to + connect. Click **Connect** to select the repository. + +  + +1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). + +GitLab will import the project, enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), enable +[GitHub project integration](../../user/project/integrations/github.md), and create a web hook +on GitHub to notify GitLab of new commits. + +## Connect with Personal Access Token + +NOTE: **Note:** Personal access tokens can only be used to connect GitHub.com +repositories to GitLab. + +If you are not using the [GitHub integration](../../integration/github.md), you can +still perform a one-off authorization with GitHub to grant GitLab access your +repositories: + +1. Open https://github.com/settings/tokens/new to create a **Personal Access + Token**. This token with be used to access your repository and push commit + statuses to GitHub. + + The `repo` and `admin:repo_hook` should be enable to allow GitLab access to + your project, update commit statuses, and create a web hook to notify + GitLab of new commits. + +1. In GitLab create a **CI/CD for external repo** project and select + **GitHub**. + +  + +1. Paste the token into the **Personal access token** field and click **List + Repositories**. Click **Connect** to select the repository. + +1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). + +GitLab will import the project, enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), enable +[GitHub project integration](../../user/project/integrations/github.md), and create a web hook +on GitHub to notify GitLab of new commits. + +## Connect manually + +If the [GitHub integration](../../integration/github.md) is not enabled, or is enabled +for a different GitHub instance, you GitLab CI/CD can be manually enabled for +your repository. + +1. In GitHub open https://github.com/settings/tokens/new create a **Personal + Access Token.** GitLab will use this token to access your repository and + push commit statuses. + + Enter a **Token description** and update the scope to allow: + + `repo` so that GitLab can access your project and update commit statuses + +1. In GitLab create a **CI/CD project** using the Git URL option and the HTTPS + URL for your GitHub repository. If your project is private, use the personal + access token you just created for authentication. + + GitLab will automatically configure polling-based pull mirroring. + +1. Still in GitLab, enable the [GitHub project integration](../../user/project/integrations/github.md) + from **Settings > Integrations.** + + Check the **Active** checkbox to enable the integration, paste your + personal access token and HTTPS repository URL into the form, and **Save.** + +1. Still in GitLab create a **Personal Access Token** with `API` scope to + authenticate the GitHub web hook notifying GitLab of new commits. + +1. In GitHub from **Settings > Webhooks** create a web hook to notify GitLab of + new commits. + + The web hook URL should be set to the GitLab API to + [trigger pull mirroring](../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter), + using the GitLab personal access token we just created. + + ``` + https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> + ``` + +  + +1. In GitHub add a `.gitlab-ci.yml` to configure GitLab CI/CD. diff --git a/doc/ci/ci_cd_for_external_repos/img/bitbucket_app_password.png b/doc/ci/ci_cd_for_external_repos/img/bitbucket_app_password.png Binary files differnew file mode 100644 index 00000000000..ac5be3c3058 --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/bitbucket_app_password.png diff --git a/doc/ci/ci_cd_for_external_repos/img/bitbucket_webhook.png b/doc/ci/ci_cd_for_external_repos/img/bitbucket_webhook.png Binary files differnew file mode 100644 index 00000000000..0a3476d9035 --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/bitbucket_webhook.png diff --git a/doc/ci/ci_cd_for_external_repos/img/ci_cd_for_external_repo.png b/doc/ci/ci_cd_for_external_repos/img/ci_cd_for_external_repo.png Binary files differnew file mode 100644 index 00000000000..f068688146b --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/ci_cd_for_external_repo.png diff --git a/doc/ci/ci_cd_for_external_repos/img/external_repository.png b/doc/ci/ci_cd_for_external_repos/img/external_repository.png Binary files differnew file mode 100644 index 00000000000..b850d91f56b --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/external_repository.png diff --git a/doc/ci/ci_cd_for_external_repos/img/github_omniauth.png b/doc/ci/ci_cd_for_external_repos/img/github_omniauth.png Binary files differnew file mode 100644 index 00000000000..71a3a057a41 --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/github_omniauth.png diff --git a/doc/ci/ci_cd_for_external_repos/img/github_omniauth_list.png b/doc/ci/ci_cd_for_external_repos/img/github_omniauth_list.png Binary files differnew file mode 100644 index 00000000000..3f2059504f5 --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/github_omniauth_list.png diff --git a/doc/ci/ci_cd_for_external_repos/img/github_push_webhook.png b/doc/ci/ci_cd_for_external_repos/img/github_push_webhook.png Binary files differnew file mode 100644 index 00000000000..e8c17d664e1 --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/github_push_webhook.png diff --git a/doc/ci/ci_cd_for_external_repos/img/github_repo_list.png b/doc/ci/ci_cd_for_external_repos/img/github_repo_list.png Binary files differnew file mode 100644 index 00000000000..73579dd3cf1 --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/img/github_repo_list.png diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md new file mode 100644 index 00000000000..e57be551bef --- /dev/null +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -0,0 +1,30 @@ +# GitLab CI/CD for external repositories **[PREMIUM]** + +NOTE: **Note:** +To [promote its release](https://about.gitlab.com/2018/03/22/gitlab-10-6-released/#gitlab-cicd-for-external-repos), GitLab.com users are receiving this feature for free through March 22, 2019. + +>[Introduced][ee-4642] in [GitLab Premium][eep] 10.6. + +GitLab CI/CD can be used with GitHub or any other Git server. +Instead of moving your entire project to GitLab, you can connect your +external repository to get the benefits of GitLab CI/CD. + +- [GitHub](github_integration.md) +- [Bitbucket Cloud](bitbucket_integration.md) + +Connecting an external repository will set up [repository mirroring][mirroring] +and create a lightweight project where issues, merge requests, wiki, and +snippets disabled. These features +[can be re-enabled later][settings]. + +1. From your GitLab dashboard click **New project** +1. Switch to the **CI/CD for external repo** tab +1. Choose **GitHub** or **Repo by URL** +1. The next steps are similar to the [import flow](../../user/project/import/index.md) + + + +[ee-4642]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4642 +[eep]: https://about.gitlab.com/pricing/ +[mirroring]: ../../workflow/repository_mirroring.md +[settings]: ../../user/project/settings/index.md#sharing-and-permissions diff --git a/doc/ci/environments.md b/doc/ci/environments.md index fe66f7e3c28..3327445724d 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -598,6 +598,38 @@ fetch line: fetch = +refs/environments/*:refs/remotes/origin/environments/* ``` +## Scoping environments with specs **[PREMIUM]** + +Some GitLab [Enterprise Edition](https://about.gitlab.com/pricing/) features can behave differently for each [Environment](#introduction-to-environments-and-deployments). +For example, you can [create a secret variable to be injected only into a production environment](variables/README.md#limiting-environment-scopes-of-variables-premium). +In most cases, these features use the _environment specs_ mechanism, which offers +an efficient way to implement scoping within each environment group. + +Let's say there are four environments: + +- `production` +- `staging` +- `review/feature-1` +- `review/feature-2` + +Each environment can be matched with the following environment spec: + +| Environment Spec | `production` | `staging` | `review/feature-1` | `review/feature-2` | +| ---------------- | ---------- | ------- | ---------------- | ---------------- | +| * | Matched | Matched | Matched | Matched | +| production | Matched | | | | +| staging | | Matched | | | +| review/* | | | Matched | Matched | +| review/feature-1 | | | Matched | | + +As you can see, you can use specific matching for selecting a particular environment, +and also use wildcard matching (`*`) for selecting a particular environment group, +such as [Review apps](review_apps/index.md) (`review/*`). + +NOTE: **Note:** +The most _specific_ spec takes precedence over the other wildcard matching. +In this case, `review/feature-1` spec takes precedence over `review/*` and `*` specs. + ## Limitations 1. You are limited to use only the [CI predefined variables][variables] in the @@ -611,6 +643,7 @@ Below are some links you may find interesting: - [The `.gitlab-ci.yml` definition of environments](yaml/README.md#environment) - [A blog post on Deployments & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) - [Review Apps - Use dynamic environments to deploy your code for every branch](review_apps/index.md) +- [Deploy Boards for your applications running on Kubernetes](../user/project/deploy_boards.md) [Pipelines]: pipelines.md [jobs]: yaml/README.md#jobs diff --git a/doc/ci/environments/img/protected_environments_form.png b/doc/ci/environments/img/protected_environments_form.png Binary files differnew file mode 100644 index 00000000000..7c04903a264 --- /dev/null +++ b/doc/ci/environments/img/protected_environments_form.png diff --git a/doc/ci/environments/img/protected_environments_list.png b/doc/ci/environments/img/protected_environments_list.png Binary files differnew file mode 100644 index 00000000000..a1b9cc1e587 --- /dev/null +++ b/doc/ci/environments/img/protected_environments_list.png diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md new file mode 100644 index 00000000000..23df3f84039 --- /dev/null +++ b/doc/ci/environments/protected_environments.md @@ -0,0 +1,48 @@ +# Protected Environments **[PREMIUM]** + +> [Introduced][6303] in [GitLab Premium][ee] 11.3. + +## Overview + +[Environments](../environments.md) can be used for different scopes, some of +them are just for testing while others are for production. As deploy jobs could +be raised by different users with different roles, it is very important that +specific environments are "protected" to avoid unauthorized people to affect them. + +By default, a protected environment does one thing: it ensures that only people +with the right privileges can deploy to it, thus keeping it safe. + +NOTE: **Note**: +A GitLab admin is always allowed to use environments, even if they are protected. + +To protect, update, or unprotect an environment, you need to have at least +[Maintainer permissions](../../user/permissions.md). + +## Configuring protected environments + +To protect an environment: + +1. Navigate to your project's **Settings âž” CI/CD**. +1. Scroll to find the **Protected Environments** section. +1. From the **Environment** dropdown menu, select the environment you want to protect and + click **Protect**. +1. In the "Allowed to Deploy" dropdown menu, you can select the role and/or the + users and/or the groups you want to have deploy access. There are some + considerations to have in mind: + - There are two roles to choose from: + - **Maintainers**: will allow access to all maintainers in the project. + - **Developers**: will allow access to all maintainers and all developers in the project. + - You can only select groups that are associated with the project. + - Only users that have at least Developer permission level will appear on + the "Allowed to Deploy" dropdown menu. + +1. Once done, the protected environment will appear in the "Protected Environments" + list. + +Maintainers can update existing protected environments at any time +by changing the access on "Allowed to Deploy" dropdown menu. Similarly, +to unprotect a protected environment, Maintainers need to click the +**Unprotect** button of the respective environment. + +[ee]: https://about.gitlab.com/pricing/ +[6303]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6303 diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index a1c997d1de6..f33c641bbea 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -18,30 +18,30 @@ Examples are available in several forms. As a collection of: The following table lists examples for different use cases: -| Use case | Resource | -|:-----------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------| -| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](browser_performance.md). | -| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | -| Code quality analysis | [Analyze your project's Code Quality](code_quality.md). **[STARTER]** | -| Container scanning | [Container Scanning with GitLab CI/CD](container_scanning.md). | -| Dependency scanning | [Dependency Scanning with GitLab CI/CD](https://docs.gitlab.com/ee/ci/examples/dependency_scanning.html). **[ULTIMATE]** | -| Deployment with `dpl` | [Using `dpl` as deployment tool](deployment/README.md). | -| Dynamic application<br>security testing (DAST) | [Dynamic Application Security Testing with GitLab CI/CD](dast.md) **[ULTIMATE]** | -| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | -| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | -| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example. | -| Java | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | -| JUnit | [JUnit test reports](../junit_test_reports.md). | -| License management | [Dependencies license management with GitLab CI/CD](https://docs.gitlab.com/ee/ci/examples/license_management.html) **[ULTIMATE]** | -| Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | -| PHP | [Testing PHP projects](php.md). | -| PHP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | -| Python | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | -| Ruby | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | -| Scala | [Test and deploy a Scala application to Heroku](test-scala-application.md). | -| Static application<br>security testing (SAST) | [Static Application Security Testing with GitLab CI/CD](https://docs.gitlab.com/ee/ci/examples/sast.html) **[ULTIMATE]** | -| Testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | +| Use case | Resource | +|:-----------------------------------------------|:---------------------------------------------------------------------------------------------------------------------| +| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](browser_performance.md). | +| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | +| Code quality analysis | [Analyze your project's Code Quality](code_quality.md). **[STARTER]** | +| Container scanning | [Container Scanning with GitLab CI/CD](container_scanning.md). | +| Dependency scanning | [Dependency Scanning with GitLab CI/CD](dependency_scanning.md). **[ULTIMATE]** | +| Deployment with `dpl` | [Using `dpl` as deployment tool](deployment/README.md). | +| Dynamic application<br>security testing (DAST) | [Dynamic Application Security Testing with GitLab CI/CD](dast.md) **[ULTIMATE]** | +| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | +| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | +| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example. | +| Java | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | +| JUnit | [JUnit test reports](../junit_test_reports.md). | +| License management | [Dependencies license management with GitLab CI/CD](license_management.md) **[ULTIMATE]** | +| Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | +| PHP | [Testing PHP projects](php.md). | +| PHP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | +| PHP | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| Python | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | +| Ruby | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | +| Scala | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| Static application<br>security testing (SAST) | [Static Application Security Testing with GitLab CI/CD](sast.md) **[ULTIMATE]** | +| Testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | ### Contributing examples diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index b47038011de..7bec8d94bfc 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -56,7 +56,7 @@ provide a list of URLs to test, please consult TIP: **Tip:** For [GitLab Premium](https://about.gitlab.com/pricing/) users, key metrics are automatically extracted and shown right in the merge request widget. -[Learn more on Browser Performance Testing in merge requests](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html). +[Learn more on Browser Performance Testing in merge requests](../../user/project/merge_requests/browser_performance_testing.md). ## Performance testing on Review Apps diff --git a/doc/ci/examples/code_quality.md b/doc/ci/examples/code_quality.md index 3e7d6e7e3f7..a6fa6979190 100644 --- a/doc/ci/examples/code_quality.md +++ b/doc/ci/examples/code_quality.md @@ -43,7 +43,7 @@ Due to implementation limitations we always take the latest Code Quality artifac TIP: **Tip:** For [GitLab Starter][ee] users, this information will be automatically extracted and shown right in the merge request widget. -[Learn more on Code Quality in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html). +[Learn more on Code Quality in merge requests](../../user/project/merge_requests/code_quality.html). ## Previous job definitions diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md index e8e9c73d1b2..5ef41d498cc 100644 --- a/doc/ci/examples/container_scanning.md +++ b/doc/ci/examples/container_scanning.md @@ -62,7 +62,7 @@ in our case its named `clair-whitelist.yml`. TIP: **Tip:** For [GitLab Ultimate][ee] users, this information will be automatically extracted and shown right in the merge request widget. -[Learn more on Container Scanning in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html). +[Learn more on Container Scanning in merge requests](../../user/project/merge_requests/container_scanning.html). CAUTION: **Caution:** Starting with GitLab 11.5, Container Scanning feature is licensed under the name `container_scanning`. diff --git a/doc/ci/examples/dast.md b/doc/ci/examples/dast.md index ab0ca13d2cf..52c4147e245 100644 --- a/doc/ci/examples/dast.md +++ b/doc/ci/examples/dast.md @@ -17,7 +17,7 @@ It can be very useful combined with [Review Apps](../review_apps/index.md). ## Example First, you need GitLab Runner with -[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). +[docker executor](https://docs.gitlab.com/runner/executors/docker.html). Once you set up the Runner, add a new job to `.gitlab-ci.yml` that generates the expected report: @@ -72,7 +72,7 @@ to learn more about authentication settings. TIP: **Tip:** For [GitLab Ultimate][ee] users, this information will be automatically extracted and shown right in the merge request widget. -[Learn more on DAST in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html). +[Learn more on DAST in merge requests](../../user/project/merge_requests/dast.md). ## Previous job definitions diff --git a/doc/ci/examples/dependency_scanning.md b/doc/ci/examples/dependency_scanning.md new file mode 100644 index 00000000000..48ddea73203 --- /dev/null +++ b/doc/ci/examples/dependency_scanning.md @@ -0,0 +1,98 @@ +# Dependency Scanning with GitLab CI/CD **[ULTIMATE]** + +CAUTION: **Caution:** +The job definition shown below is supported on GitLab 11.5 and later versions. +It also requires the GitLab Runner 11.5 or later. +For earlier versions, use the [previous job definitions](#previous-job-definitions). + +This example shows how to run Dependency Scanning on your +project's dependencies by using GitLab CI/CD. + + +First, you need GitLab Runner with +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). + +Once you set up the Runner, add a new job to `.gitlab-ci.yml` that +generates the expected report: + +```yaml +dependency_scanning: + image: docker:stable + variables: + DOCKER_DRIVER: overlay2 + allow_failure: true + services: + - docker:stable-dind + script: + - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') + - docker run + --env DEP_SCAN_DISABLE_REMOTE_CHECKS="${DEP_SCAN_DISABLE_REMOTE_CHECKS:-false}" + --volume "$PWD:/code" + --volume /var/run/docker.sock:/var/run/docker.sock + "registry.gitlab.com/gitlab-org/security-products/dependency-scanning:$SP_VERSION" /code + artifacts: + reports: + dependency_scanning: gl-dependency-scanning-report.json +``` + +The above example will create a `dependency_scanning` job in your CI/CD pipeline +and scan your dependencies for possible vulnerabilities. The report will be saved as a +[Dependency Scanning report artifact](../../ci/yaml/README.md#artifactsreportsdependency_scanning-ultimate) +that you can later download and analyze. +Due to implementation limitations we always take the latest Dependency Scanning artifact available. + +The results are sorted by the priority of the vulnerability: + +1. High +1. Medium +1. Low +1. Unknown +1. Everything else + +Behind the scenes, the [GitLab Dependency Scanning Docker image](https://gitlab.com/gitlab-org/security-products/dependency-scanning) +is used to detect the languages/package managers and in turn runs the matching scan tools. + +Some security scanners require to send a list of project dependencies to GitLab +central servers to check for vulnerabilities. To learn more about this or to +disable it, check the [GitLab Dependency Scanning documentation](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks). + +TIP: **Tip:** +For [GitLab Ultimate][ee] users, this information will +be automatically extracted and shown right in the merge request widget. +[Learn more on Dependency Scanning in merge requests](../../user/project/merge_requests/dependency_scanning.md). + +## Supported languages and package managers + +See [the full list of supported languages and package managers](../../user/project/merge_requests/dependency_scanning.md#supported-languages-and-dependency-managers). + +## Previous job definitions + +CAUTION: **Caution:** +Before GitLab 11.5, Dependency Scanning job and artifact had to be named specifically +to automatically extract report data and show it in the merge request widget. +While these old job definitions are still maintained they have been deprecated +and may be removed in next major release, GitLab 12.0. +You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. + +For GitLab 11.4 and earlier, the job should look like: + +```yaml +dependency_scanning: + image: docker:stable + variables: + DOCKER_DRIVER: overlay2 + allow_failure: true + services: + - docker:stable-dind + script: + - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') + - docker run + --env DEP_SCAN_DISABLE_REMOTE_CHECKS="${DEP_SCAN_DISABLE_REMOTE_CHECKS:-false}" + --volume "$PWD:/code" + --volume /var/run/docker.sock:/var/run/docker.sock + "registry.gitlab.com/gitlab-org/security-products/dependency-scanning:$SP_VERSION" /code + artifacts: + paths: [gl-dependency-scanning-report.json] +``` + +[ee]: https://about.gitlab.com/pricing/ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/variables_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/variables_page.png Binary files differindex 4675e20ef79..edeaa011ada 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/variables_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/variables_page.png diff --git a/doc/ci/examples/license_management.md b/doc/ci/examples/license_management.md new file mode 100644 index 00000000000..c45f1c0404a --- /dev/null +++ b/doc/ci/examples/license_management.md @@ -0,0 +1,100 @@ +# Dependencies license management with GitLab CI/CD **[ULTIMATE]** + +CAUTION: **Caution:** +The job definition shown below is supported on GitLab 11.5 and later versions. +It also requires the GitLab Runner 11.5 or later. +For earlier versions, use the [previous job definitions](#previous-job-definitions). + +This example shows how to run the License Management tool on your +project's dependencies by using GitLab CI/CD. + +First, you need GitLab Runner with +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). + +Once you set up the Runner, add a new job to `.gitlab-ci.yml` that +generates the expected report: + +```yaml +license_management: + image: + name: "registry.gitlab.com/gitlab-org/security-products/license-management:$CI_SERVER_VERSION_MAJOR-$CI_SERVER_VERSION_MINOR-stable" + entrypoint: [""] + stage: test + allow_failure: true + script: + - /run.sh analyze . + artifacts: + reports: + license_management: gl-license-management-report.json +``` + +The above example will create a `license_management` job in your CI/CD pipeline +and scan your dependencies to find their licenses. The report will be saved as a +[License Management report artifact](../../ci/yaml/README.md#artifactsreportslicense_management-ultimate) +that you can later download and analyze. +Due to implementation limitations we always take the latest License Management artifact available. + +## Install custom project dependencies + +> Introduced in GitLab Ultimate 11.4. + +The `license_management` image already embeds many auto-detection scripts, languages, +and packages. Nevertheless, it's almost impossible to cover all cases, for all projects. +That's why sometimes it's necessary to install extra packages, or to have extra steps +in the project automated setup, like the download and installation of a certificate. +For that, a `SETUP_CMD` environment variable can be passed to the container, +with the required commands to run before license detection. + +If present, this variable will override the setup step necessary to install all the packages +of your application (ex: for a project with a `Gemfile`, the setup step will be `bundle install`). + +Example: + +```yaml +license_management: + image: + name: "registry.gitlab.com/gitlab-org/security-products/license-management:$CI_SERVER_VERSION_MAJOR-$CI_SERVER_VERSION_MINOR-stable" + entrypoint: [""] + stage: test + variables: + SETUP_CMD: ./my-custom-install-script.sh + allow_failure: true + script: + - /run.sh analyze . + artifacts: + reports: + license_management: gl-license-management-report.json +``` + +In this example, `my-custom-install-script.sh` is a shell script at the root of the project. + +TIP: **Tip:** +For [GitLab Ultimate][ee] users, this information will +be automatically extracted and shown right in the merge request widget. +[Learn more on License Management in merge requests](../../user/project/merge_requests/license_management.md). + +## Previous job definitions + +CAUTION: **Caution:** +Before GitLab 11.5, License Management job and artifact had to be named specifically +to automatically extract report data and show it in the merge request widget. +While these old job definitions are still maintained they have been deprecated +and may be removed in next major release, GitLab 12.0. +You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. + +For GitLab 11.4 and earlier, the job should look like: + +```yaml +license_management: + image: + name: "registry.gitlab.com/gitlab-org/security-products/license-management:$CI_SERVER_VERSION_MAJOR-$CI_SERVER_VERSION_MINOR-stable" + entrypoint: [""] + stage: test + allow_failure: true + script: + - /run.sh analyze . + artifacts: + paths: [gl-license-management-report.json] +``` + +[ee]: https://about.gitlab.com/pricing/ diff --git a/doc/ci/examples/sast.md b/doc/ci/examples/sast.md new file mode 100644 index 00000000000..2c9db74b9b9 --- /dev/null +++ b/doc/ci/examples/sast.md @@ -0,0 +1,95 @@ +# Static Application Security Testing with GitLab CI/CD **[ULTIMATE]** + +CAUTION: **Caution:** +The job definition shown below is supported on GitLab 11.5 and later versions. +It also requires the GitLab Runner 11.5 or later. +For earlier versions, use the [previous job definitions](#previous-job-definitions). + +This example shows how to run +[Static Application Security Testing (SAST)](https://en.wikipedia.org/wiki/Static_program_analysis) +on your project's source code by using GitLab CI/CD. + +First, you need GitLab Runner with +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). + +Once you set up the Runner, add a new job to `.gitlab-ci.yml` that +generates the expected report: + +```yaml +sast: + image: docker:stable + variables: + DOCKER_DRIVER: overlay2 + allow_failure: true + services: + - docker:stable-dind + script: + - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') + - docker run + --env SAST_CONFIDENCE_LEVEL="${SAST_CONFIDENCE_LEVEL:-3}" + --volume "$PWD:/code" + --volume /var/run/docker.sock:/var/run/docker.sock + "registry.gitlab.com/gitlab-org/security-products/sast:$SP_VERSION" /app/bin/run /code + artifacts: + reports: + sast: gl-sast-report.json +``` + +The above example will create a `sast` job in your CI/CD pipeline +and scan your dependencies for possible vulnerabilities. The report will be saved as a +[SAST report artifact](../../ci/yaml/README.md#artifactsreportssast-ultimate) +that you can later download and analyze. +Due to implementation limitations we always take the latest SAST artifact available. + +The results are sorted by the priority of the vulnerability: + +1. Critical +1. High +1. Medium +1. Low +1. Unknown +1. Everything else + +Behind the scenes, the [GitLab SAST Docker image](https://gitlab.com/gitlab-org/security-products/sast) +is used to detect the languages/frameworks and in turn runs the matching scan tools. + +TIP: **Tip:** +For [GitLab Ultimate][ee] users, this information will +be automatically extracted and shown right in the merge request widget. +[Learn more on SAST in merge requests](../../user/project/merge_requests/sast.md). + +## Supported languages and frameworks + +See [the full list of supported languages and frameworks](../../user/project/merge_requests/sast.md#supported-languages-and-frameworks). + +## Previous job definitions + +CAUTION: **Caution:** +Before GitLab 11.5, SAST job and artifact had to be named specifically +to automatically extract report data and show it in the merge request widget. +While these old job definitions are still maintained they have been deprecated +and may be removed in next major release, GitLab 12.0. +You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. + +For GitLab 11.4 and earlier, the job should look like: + +```yaml +sast: + image: docker:stable + variables: + DOCKER_DRIVER: overlay2 + allow_failure: true + services: + - docker:stable-dind + script: + - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') + - docker run + --env SAST_CONFIDENCE_LEVEL="${SAST_CONFIDENCE_LEVEL:-3}" + --volume "$PWD:/code" + --volume /var/run/docker.sock:/var/run/docker.sock + "registry.gitlab.com/gitlab-org/security-products/sast:$SP_VERSION" /app/bin/run /code + artifacts: + paths: [gl-sast-report.json] +``` + +[ee]: https://about.gitlab.com/pricing/ diff --git a/doc/ci/img/multi_pipeline_mini_graph.gif b/doc/ci/img/multi_pipeline_mini_graph.gif Binary files differnew file mode 100644 index 00000000000..de49ba5aa12 --- /dev/null +++ b/doc/ci/img/multi_pipeline_mini_graph.gif diff --git a/doc/ci/img/multi_project_pipeline_graph.png b/doc/ci/img/multi_project_pipeline_graph.png Binary files differnew file mode 100644 index 00000000000..723a455cb4a --- /dev/null +++ b/doc/ci/img/multi_project_pipeline_graph.png diff --git a/doc/ci/img/pipelines-goal.png b/doc/ci/img/pipelines-goal.png Binary files differindex f15716d0b8f..17bd77e951f 100644 --- a/doc/ci/img/pipelines-goal.png +++ b/doc/ci/img/pipelines-goal.png diff --git a/doc/ci/img/types-of-pipelines.png b/doc/ci/img/types-of-pipelines.png Binary files differindex 829a53d5d52..4a64f4f4cf7 100644 --- a/doc/ci/img/types-of-pipelines.png +++ b/doc/ci/img/types-of-pipelines.png diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index 2a4160f62b0..7109b2ec583 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -54,3 +54,8 @@ terminal will block the job from finishing for the duration configured in close the terminal window.  + +## Interactive Web Terminals for the Web IDE **[ULTIMATE ONLY]** + +Read the Web IDE docs to learn how to run [Interactive Terminals through the Web IDE](../../user/project/web_ide/index.md). + diff --git a/doc/ci/merge_request_pipelines/img/pipeline_detail.png b/doc/ci/merge_request_pipelines/img/pipeline_detail.png Binary files differindex 6094a0975fb..90e7c449a66 100644 --- a/doc/ci/merge_request_pipelines/img/pipeline_detail.png +++ b/doc/ci/merge_request_pipelines/img/pipeline_detail.png diff --git a/doc/ci/multi_project_pipeline_graphs.md b/doc/ci/multi_project_pipeline_graphs.md new file mode 100644 index 00000000000..af10e5b6126 --- /dev/null +++ b/doc/ci/multi_project_pipeline_graphs.md @@ -0,0 +1,5 @@ +--- +redirect_to: 'multi_project_pipelines.md' +--- + +This document was moved to [another location](multi_project_pipelines.md). diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md new file mode 100644 index 00000000000..1a0d3367dd8 --- /dev/null +++ b/doc/ci/multi_project_pipelines.md @@ -0,0 +1,152 @@ +# Multi-project pipelines **[PREMIUM]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/2121) in +[GitLab Premium 9.3](https://about.gitlab.com/2017/06/22/gitlab-9-3-released/#multi-project-pipeline-graphs). + +When you set up [GitLab CI/CD](README.md) across multiple projects, you can visualize +the entire pipeline, including all cross-project inter-dependencies. + +## Overview + +GitLab CI/CD is a powerful continuous integration tool that works not only per project, but also across projects. When you +configure GitLab CI for your project, you can visualize the stages +of your [jobs](pipelines.md#jobs) on a [pipeline graph](pipelines.md#pipeline-graphs). + + + +In the Merge Request Widget, multi-project pipeline mini-graphs are displayed, +and when hovering or tapping (on touchscreen devices) they will expand and be shown adjacent to each other. + + + +Multi-project pipelines are useful for larger products that require cross-project inter-dependencies, such as those +adopting a [microservices architecture](https://about.gitlab.com/2016/08/16/trends-in-version-control-land-microservices/). + +## Use cases + +Let's assume you deploy your web app from different projects in GitLab: + +- One for the free version, which has its own pipeline that builds and tests your app +- One for the paid version add-ons, which also pass through builds and tests +- One for the documentation, which also builds, tests, and deploys with an SSG + +With Multi-Project Pipelines, you can visualize the entire pipeline, including all stages of builds and tests for the three projects. + +## Triggering multi-project pipelines through API + +When you use the [`CI_JOB_TOKEN` to trigger pipelines](triggers/README.md#ci-job-token), GitLab +recognizes the source of the job token, and thus internally ties these pipelines +together, allowing you to visualize their relationships on pipeline graphs. + +These relationships are displayed in the pipeline graph by showing inbound and +outbound connections for upstream and downstream pipeline dependencies. + +## Creating multi-project pipelines from `.gitlab-ci.yml` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. + +### Triggering a downstream pipeline using a bridge job + +Before GitLab 11.8, it was necessary to implement a pipeline job that was +responsible for making the API request [to trigger a pipeline](triggers/README.md#creating-cross-project-pipeline-through-API) +in a different project. + +In GitLab 11.8, GitLab provides a new CI/CD configuration syntax to make this +task easier, and avoid needing GitLab Runner for triggering cross-project +pipelines. The following illustrates configuring a bridge job: + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + variables: + ENVIRONMENT: staging + stage: deploy + trigger: my/deployment +``` + +In the example above, as soon as `rspec` job succeeds in the `test` stage, +the `staging` bridge job is going to be started. The initial status of this +job will be `pending`. GitLab will create a downstream pipeline in the +`my/deployment` project and, as soon as the pipeline gets created, the +`staging` job will succeed. `my/deployment` is a full path to that project. + +The user that created the upstream pipeline needs to have access rights to the +downstream project (`my/deployment` in this case). If a downstream project can +not be found, or a user does not have access rights to create pipeline there, +the `staging` job is going to be marked as _failed_. + +Caution: **Caution:** +`staging` will succeed as soon as a downstream pipeline gets created. +GitLab does not support status attribution yet, however adding first-class +`trigger` configuration syntax is ground work for implementing +[status attribution](https://gitlab.com/gitlab-org/gitlab-ce/issues/39640). + +NOTE: **Note:** +Bridge jobs do not support every configuration entry that a user can use +in the case of regular jobs. Bridge jobs will not to be picked by a Runner, +thus there is no point in adding support for `script`, for example. If a user +tries to use unsupported configuration syntax, YAML validation will fail upon +pipeline creation. + +### Specifying a downstream pipeline branch + +It is possible to specify a branch name that a downstream pipeline will use: + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + stage: deploy + trigger: + project: my/deployment + branch: stable-11-2 +``` + +Use a `project` keyword to specify full path to a downstream project. Use +a `branch` keyword to specify a branch name. + +GitLab will use a commit that is currently on the HEAD of the branch when +creating a downstream pipeline. + +### Passing variables to a downstream pipeline + +Sometimes you might want to pass variables to a downstream pipeline. +You can do that using the `variables` keyword, just like you would when +defining a regular job. + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + variables: + ENVIRONMENT: staging + stage: deploy + trigger: my/deployment +``` + +The `ENVIRONMENT` variable will be passed to every job defined in a downstream +pipeline. It will be available as an environment variable when GitLab Runner picks a job. + +### Limitations + +Because bridge jobs are a little different to regular jobs, it is not +possible to use exactly the same configuration syntax here, as one would +normally do when defining a regular job that will be picked by a runner. + +Some features are not implemented yet. For example, support for environments. + +[Configuration keywords](yaml/README.md) available for bridge jobs are: + +- `trigger` (to define a downstream pipeline trigger) +- `stage` +- `allow_failure` +- `only` and `except` +- `when` +- `extends` diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 4f3106c6dc6..b1b9bd649a5 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -3,7 +3,7 @@ > Introduced in GitLab 8.8. NOTE: **Note:** -If you have a [mirrored repository where GitLab pulls from](https://docs.gitlab.com/ee/workflow/repository_mirroring.html#pulling-from-a-remote-repository-starter), +If you have a [mirrored repository where GitLab pulls from](../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), you may need to enable pipeline triggering in your project's **Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. @@ -231,6 +231,11 @@ by name. The order of severity is:  +### Multi-project pipelines graphs **[PREMIUM]** + +With [multi-project pipelines](multi_project_pipelines.md), +you can visualize cross-project pipelines. + ## How the pipeline duration is calculated Total running time for a given pipeline would exclude retries and pending @@ -297,9 +302,9 @@ runners will not use regular runners, they must be tagged accordingly. [manual]: yaml/README.md#whenmanual [env-manual]: environments.md#manually-deploying-to-environments [stages]: yaml/README.md#stages -[runners]: runners/README.html +[runners]: runners/README.md [pipelines settings]: ../user/project/pipelines/settings.md -[triggers]: triggers/README.md +[triggers]: triggers/README.md#ci-job-token [ce-5742]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5742 [ce-6242]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6242 [ce-7931]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7931 @@ -307,3 +312,5 @@ runners will not use regular runners, they must be tagged accordingly. [ce-17782]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17782 [ce-17814]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17814 [regexp]: https://gitlab.com/gitlab-org/gitlab-ce/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99 +[eep]: https://about.gitlab.com/pricing/ "GitLab Premium" +[ee-2121]: https://gitlab.com/gitlab-org/gitlab-ee/issues/2121 diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 9684cb6ed98..a1eb7043e1b 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -127,7 +127,7 @@ Now if you go to the **Pipelines** page you will see that the pipeline is pending. NOTE: **Note:** -If you have a [mirrored repository where GitLab pulls from](https://docs.gitlab.com/ee/workflow/repository_mirroring.html#pulling-from-a-remote-repository-starter), +If you have a [mirrored repository where GitLab pulls from](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), you may need to enable pipeline triggering in your project's **Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 398b017277f..6274eeee300 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -23,6 +23,63 @@ attackers can impersonate the user that exposed their trigger token publicly in their `.gitlab-ci.yml` file. Use [variables](../variables/README.md#variables) to protect trigger tokens. +### CI job token + +You can use the `CI_JOB_TOKEN` [variable][predef] (used to authenticate +with the [GitLab Container Registry][registry]) in the following cases. + +#### When used with multi-project pipelines **[PREMIUM]** + +> **Note**: +The use of `CI_JOB_TOKEN` for multi-project pipelines was [introduced][ee-2017] +in [GitLab Premium][ee] 9.3. + +This way of triggering can only be used when invoked inside `.gitlab-ci.yml`, +and it creates a dependent pipeline relation visible on the +[pipeline graph](../pipelines.md#multi-project-pipelines-graphs-premium). For example: + +```yaml +build_docs: + stage: deploy + script: + - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline + only: + - tags +``` + +Pipelines triggered that way also expose a special variable: +`CI_PIPELINE_SOURCE=pipeline`. + +Read more about the [pipelines trigger API][trigapi]. + +#### When a pipeline depends on the artifacts of another pipeline **[PREMIUM]** + +> The use of `CI_JOB_TOKEN` in the artifacts download API was [introduced][ee-2346] + in [GitLab Premium][ee] 9.5. + +With the introduction of dependencies between different projects, one of +them may need to access artifacts created by a previous one. This process +must be granted for authorized accesses, and it can be done using the +`CI_JOB_TOKEN` variable that identifies a specific job. For example: + +```yaml +build_submodule: + image: debian + stage: test + script: + - apt update && apt install -y unzip + - curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test&job_token=$CI_JOB_TOKEN" + - unzip artifacts.zip + only: + - tags +``` + +This allows you to use that for multi-project pipelines and download artifacts +from any project to which you have access as this follows the same principles +with the [permission model][permissions]. + +Read more about the [jobs API](../../api/jobs.md#download-the-artifacts-archive). + ## Adding a new trigger You can add a new trigger by going to your project's @@ -225,7 +282,10 @@ removed with one of the future versions of GitLab. You are advised to [take ownership](#taking-ownership-of-a-trigger) of any legacy triggers. [ee-2017]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2017 +[ee-2346]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2346 [ee]: https://about.gitlab.com/pricing/ [variables]: ../variables/README.md [predef]: ../variables/README.md#predefined-environment-variables [registry]: ../../user/project/container_registry.md +[permissions]: ../../user/permissions.md#job-permissions +[trigapi]: ../../api/pipeline_triggers.md diff --git a/doc/ci/triggers/img/trigger_variables.png b/doc/ci/triggers/img/trigger_variables.png Binary files differindex d273b1fe3a2..910622d74cc 100644 --- a/doc/ci/triggers/img/trigger_variables.png +++ b/doc/ci/triggers/img/trigger_variables.png diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 55ecc0dccd1..46523e13538 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -77,7 +77,7 @@ future GitLab releases.** | **CI_JOB_MANUAL** | 8.12 | all | The flag to indicate that job was manually started | | **CI_JOB_NAME** | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml` | | **CI_JOB_STAGE** | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | -| **CI_JOB_TOKEN** | 9.0 | 1.2 | Token used for authenticating with the [GitLab Container Registry][registry] and downloading [dependent repositories][dependent-repositories] | +| **CI_JOB_TOKEN** | 9.0 | 1.2 | Token used for authenticating with [GitLab Container Registry][registry], downloading [dependent repositories][dependent-repositories], authenticate with multi-project pipelines when [triggers][trigger-job-token] are involved, and for [downloading job artifacts][get-job-artifacts] | | **CI_JOB_URL** | 11.1 | 0.5 | Job details URL | | **CI_MERGE_REQUEST_ID** | 11.6 | all | The ID of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) | | **CI_MERGE_REQUEST_IID** | 11.6 | all | The IID of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md) | @@ -139,6 +139,7 @@ future GitLab releases.** | **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job | | **GITLAB_USER_LOGIN** | 10.0 | all | The login username of the user who started the job | | **GITLAB_USER_NAME** | 10.0 | all | The real name of the user who started the job | +| **GITLAB_FEATURES** | 10.6 | all | The comma separated list of licensed features available for your instance and plan | | **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a job | ## GitLab 9.0 renaming @@ -253,6 +254,25 @@ Protected variables can be added by going to your project's Once you set them, they will be available for all subsequent pipelines. +### Limiting environment scopes of variables **[PREMIUM]** + +> [Introduced][ee-2112] in [GitLab Premium](https://about.gitlab.com/pricing/) 9.4. + +You can limit the environment scope of a variable by +[defining which environments][envs] it can be available for. + +Wildcards can be used, and the default environment scope is `*` which means +any jobs will have this variable, not matter if an environment is defined or +not. + +For example, if the environment scope is `production`, then only the jobs +having the environment `production` defined would have this specific variable. +Wildcards (`*`) can be used along with the environment name, therefore if the +environment scope is `review/*` then any jobs with environment names starting +with `review/` would have that particular variable. + +To learn more about about scoping environments, see [Scoping environments with specs](../environments.md#scoping-environments-with-specs-premium). + ### Manually-specified variables > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/44059) in GitLab 10.8. @@ -632,14 +652,16 @@ Below you can find supported syntax reference: Pattern matching is case-sensitive by default. Use `i` flag modifier, like `/pattern/i` to make a pattern case-insensitive. +[ee-2112]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2112 [ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI variables" -[eep]: https://about.gitlab.com/pricing/ "Available only in GitLab Premium" [envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md [protected tags]: ../../user/project/protected_tags.md [shellexecutors]: https://docs.gitlab.com/runner/executors/ [triggered]: ../triggers/README.md [builds-policies]: ../yaml/README.md#only-and-except-complex +[trigger-job-token]: ../triggers/README.md#ci-job-token [gitlab-deploy-token]: ../../user/project/deploy_tokens/index.md#gitlab-deploy-token [registry]: ../../user/project/container_registry.md [dependent-repositories]: ../../user/project/new_ci_build_permissions_model.md#dependent-repositories +[get-job-artifacts]: ../../api/jobs.html#get-job-artifacts diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index ceca4af1bee..0fc9e7f67d5 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -112,3 +112,22 @@ They are: - Not supported: - For definitions where the ["Expansion place"](#gitlab-ciyml-file) is GitLab. - In the `only` and `except` [variables expressions](README.md#variables-expressions). + +## Variables with an environment scope + +Variables defined with an environment scope are supported. Given that +there is a variable `$STAGING_SECRET` defined in a scope of +`review/staging/*`, the following job that is using dynamic environments +is going to be created, based on the matching variable expression: + +```yaml +my-job: + stage: staging + environment: + name: review/$CI_JOB_STAGE/deploy + script: + - 'deploy staging' + only: + variables: + - $STAGING_SECRET == 'something' +``` diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index a44f4b62a0e..04a57011822 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -19,7 +19,7 @@ We have complete examples of configuring pipelines: - To see a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab-ce`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml). NOTE: **Note:** -If you have a [mirrored repository where GitLab pulls from](https://docs.gitlab.com/ee/workflow/repository_mirroring.html#pulling-from-a-remote-repository-starter), +If you have a [mirrored repository where GitLab pulls from](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), you may need to enable pipeline triggering in your project's **Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. @@ -56,7 +56,7 @@ independently from each other. Each instance of GitLab CI has an embedded debug tool called Lint, which validates the content of your `.gitlab-ci.yml` files. You can find the Lint under the page `ci/lint` of your -project namespace. For example, `http://gitlab.example.com/gitlab-org/project-123/-/ci/lint`. +project namespace. For example, `https://gitlab.example.com/gitlab-org/project-123/-/ci/lint`. ### Unavailable names for jobs @@ -1632,7 +1632,7 @@ test: from `trigger` definition is started by GitLab, a downstream pipeline gets created. -Learn more about [multi-project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipelines.html#creating-cross-project-pipelines-from-gitlab-ci-yml). +Learn more about [multi-project pipelines](../multi_project_pipelines.md#creating-cross-project-pipelines-from-gitlab-ci-yml). #### Simple `trigger` syntax |
