diff options
Diffstat (limited to 'doc/ci')
65 files changed, 1665 insertions, 726 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 7048ceaac41..4be13204227 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -13,6 +13,11 @@ through the [continuous methodologies](introduction/index.md#introduction-to-cic - Continuous Delivery (CD) - Continuous Deployment (CD) +NOTE: **Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more.** +Watch our +["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. + ## Overview Continuous Integration works by pushing small code chunks to your @@ -48,6 +53,9 @@ the following documents: - [GitLab CI/CD basic workflow](introduction/index.md#basic-cicd-workflow). - [Step-by-step guide for writing `.gitlab-ci.yml` for the first time](../user/project/pages/getting_started_part_four.md). +If you're coming over from Jenkins, you can also check out our handy [reference](jenkins/index.md) +for converting your pipelines. + You can also get started by using one of the [`.gitlab-ci.yml` templates](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates) available through the UI. You can use them by creating a new file, @@ -59,11 +67,11 @@ to your needs: For a broader overview, see the [CI/CD getting started](quick_start/README.md) guide. Once you're familiar with how GitLab CI/CD works, see the -[`. gitlab-ci.yml` full reference](yaml/README.md) +[`.gitlab-ci.yml` full reference](yaml/README.md) for all the attributes you can set and use. NOTE: **Note:** -GitLab CI/CD and [shared runners](runners/README.md#shared-specific-and-group-runners) are enabled in GitLab.com and available for all users, limited only to the [user's pipelines quota](../user/admin_area/settings/continuous_integration.md#extra-shared-runners-pipeline-minutes-quota-free-only). +GitLab CI/CD and [shared runners](runners/README.md#shared-specific-and-group-runners) are enabled in GitLab.com and available for all users, limited only to the [user's pipelines quota](../user/gitlab_com/index.md#shared-runners). ## Configuration @@ -123,7 +131,7 @@ Its feature set is listed on the table below according to DevOps stages. | **Secure** || | [Container Scanning](../user/application_security/container_scanning/index.md) **(ULTIMATE)** | Check your Docker containers for known vulnerabilities.| | [Dependency Scanning](../user/application_security/dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | -| [License Management](../user/application_security/license_management/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | +| [License Compliance](../user/application_security/license_management/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | | [Security Test reports](../user/project/merge_requests/index.md#security-reports-ultimate) **(ULTIMATE)** | Check for app vulnerabilities. | ## Examples @@ -155,6 +163,7 @@ for your CI/CD infrastructure: - [Why we chose GitLab CI for our CI/CD solution](https://about.gitlab.com/2016/10/17/gitlab-ci-oohlala/) - [Building our web-app on GitLab CI](https://about.gitlab.com/2016/07/22/building-our-web-app-on-gitlab-ci/) +- [5 Teams that made the switch to GitLab CI/CD](https://about.gitlab.com/2019/04/25/5-teams-that-made-the-switch-to-gitlab-ci-cd/) See also the [Why CI/CD?](https://docs.google.com/presentation/d/1OGgk2Tcxbpl7DJaIOzCX4Vqg3dlwfELC3u2jEeCBbDk) presentation. @@ -165,7 +174,7 @@ been necessary. These are: #### 12.0 -- [Use refspec to clone/fetch git +- [Use refspec to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/issues/4069). - [Old cache configuration](https://gitlab.com/gitlab-org/gitlab-runner/issues/4070). diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 9a5a3624c73..a59a0477b80 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -8,7 +8,7 @@ GitLab CI/CD provides a caching mechanism that can be used to save time when your jobs are running. Caching is about speeding the time a job is executed by reusing the same -content of a previous job. It can be particularly useful when your are +content of a previous job. It can be particularly useful when you are developing software that depends on other libraries which are fetched via the internet during build time. @@ -172,6 +172,29 @@ job: cache: {} ``` +### Inherit global config, but override specific settings per job + +You can override cache settings without overwriting the global cache by using +[anchors](../yaml/README.md#anchors). For example, if you want to override the +`policy` for one job: + +```yaml +cache: &global_cache + key: ${CI_COMMIT_REF_SLUG} + paths: + - node_modules/ + - public/ + - vendor/ + policy: pull-push + +job: + cache: + # inherit all global cache settings + <<: *global_cache + # override the policy + policy: pull +``` + For more fine tuning, read also about the [`cache: policy`](../yaml/README.md#cachepolicy). @@ -378,8 +401,8 @@ Here's what happens behind the scenes: 1. `script` is executed. 1. `after_script` is executed. 1. `cache` runs and the `vendor/` directory is zipped into `cache.zip`. - This file is then saved in the directory based on the - [Runner's setting](#where-the-caches-are-stored) and the `cache: key`. + This file is then saved in the directory based on the + [Runner's setting](#where-the-caches-are-stored) and the `cache: key`. 1. `job B` runs. 1. The cache is extracted (if found). 1. `before_script` is executed. @@ -520,7 +543,7 @@ via GitLab's UI: 1. Navigate to your project's **CI/CD > Pipelines** page. 1. Click on the **Clear Runner caches** button to clean up the cache. -  +  1. On the next push, your CI/CD job will use a new cache. diff --git a/doc/ci/chatops/README.md b/doc/ci/chatops/README.md index 241134783da..29d4f93f02e 100644 --- a/doc/ci/chatops/README.md +++ b/doc/ci/chatops/README.md @@ -4,14 +4,14 @@ type: index, concepts, howto # GitLab ChatOps -> **Notes:** -> -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4466) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. [Moved](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24780) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.9. -> -> - ChatOps is currently in alpha, with some important features missing like access control. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4466) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24780) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.9. GitLab ChatOps provides a method to interact with CI/CD jobs through chat services like Slack. Many organizations' discussion, collaboration, and troubleshooting is taking place in chat services these days, and having a method to run CI/CD jobs with output posted back to the channel can significantly augment a team's workflow. +NOTE: **Note:** +ChatOps is currently in alpha with some important features missing, like access control. + ## How it works GitLab ChatOps is built upon two existing features: diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index b3110b435db..54b21939116 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -14,9 +14,9 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and create the project. -  +  - GitLab will import the repository and enable [Pull Mirroring][pull-mirroring]. + GitLab will import the repository and enable [Pull Mirroring][pull-mirroring]. 1. In GitLab create a [Personal Access Token](../../user/profile/personal_access_tokens.md) @@ -26,127 +26,127 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify GitLab of new commits. - The web hook URL should be set to the GitLab API to trigger pull mirroring, - using the Personal Access Token we just generated for authentication. + The web hook URL should be set to the GitLab API to trigger pull mirroring, + using the Personal Access Token we just generated for authentication. - ```text - https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> - ``` + ```text + https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> + ``` - The web hook Trigger should be set to 'Repository Push'. + The web hook Trigger should be set to 'Repository Push'. -  +  - After saving, test the web hook by pushing a change to your Bitbucket - repository. + After saving, test the web hook by pushing a change to your Bitbucket + repository. 1. In Bitbucket, create an **App Password** from **Bitbucket Settings > App Passwords** to authenticate the build status script setting commit build statuses in Bitbucket. Repository write permissions are required. -  +  1. In GitLab, from **Settings > CI/CD > Environment variables**, add variables to allow communication with Bitbucket via the Bitbucket API: - `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. + `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. - `BITBUCKET_USERNAME`: the username of the Bitbucket account. + `BITBUCKET_USERNAME`: the username of the Bitbucket account. - `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ. + `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ. - `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ. + `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ. 1. In Bitbucket, add a script to push the pipeline status to Bitbucket. - > Note: changes made in GitLab will be overwritten by any changes made - > upstream in Bitbucket. - - Create a file `build_status` and insert the script below and run - `chmod +x build_status` in your terminal to make the script executable. - - ```bash - #!/usr/bin/env bash - - # Push GitLab CI/CD build status to Bitbucket Cloud - - if [ -z "$BITBUCKET_ACCESS_TOKEN" ]; then - echo "ERROR: BITBUCKET_ACCESS_TOKEN is not set" - exit 1 - fi - if [ -z "$BITBUCKET_USERNAME" ]; then - echo "ERROR: BITBUCKET_USERNAME is not set" - exit 1 - fi - if [ -z "$BITBUCKET_NAMESPACE" ]; then - echo "Setting BITBUCKET_NAMESPACE to $CI_PROJECT_NAMESPACE" - BITBUCKET_NAMESPACE=$CI_PROJECT_NAMESPACE - fi - if [ -z "$BITBUCKET_REPOSITORY" ]; then - echo "Setting BITBUCKET_REPOSITORY to $CI_PROJECT_NAME" - BITBUCKET_REPOSITORY=$CI_PROJECT_NAME - fi - - BITBUCKET_API_ROOT="https://api.bitbucket.org/2.0" - BITBUCKET_STATUS_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/commit/$CI_COMMIT_SHA/statuses/build" - BITBUCKET_KEY="ci/gitlab-ci/$CI_JOB_NAME" - - case "$BUILD_STATUS" in - running) - BITBUCKET_STATE="INPROGRESS" - BITBUCKET_DESCRIPTION="The build is running!" - ;; - passed) - BITBUCKET_STATE="SUCCESSFUL" - BITBUCKET_DESCRIPTION="The build passed!" - ;; - failed) - BITBUCKET_STATE="FAILED" - BITBUCKET_DESCRIPTION="The build failed." - ;; - esac - - echo "Pushing status to $BITBUCKET_STATUS_API..." - curl --request POST $BITBUCKET_STATUS_API \ - --user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \ - --header "Content-Type:application/json" \ - --silent \ - --data "{ \"state\": \"$BITBUCKET_STATE\", \"key\": \"$BITBUCKET_KEY\", \"description\": - \"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }" - ``` + > Note: changes made in GitLab will be overwritten by any changes made + > upstream in Bitbucket. + + Create a file `build_status` and insert the script below and run + `chmod +x build_status` in your terminal to make the script executable. + + ```bash + #!/usr/bin/env bash + + # Push GitLab CI/CD build status to Bitbucket Cloud + + if [ -z "$BITBUCKET_ACCESS_TOKEN" ]; then + echo "ERROR: BITBUCKET_ACCESS_TOKEN is not set" + exit 1 + fi + if [ -z "$BITBUCKET_USERNAME" ]; then + echo "ERROR: BITBUCKET_USERNAME is not set" + exit 1 + fi + if [ -z "$BITBUCKET_NAMESPACE" ]; then + echo "Setting BITBUCKET_NAMESPACE to $CI_PROJECT_NAMESPACE" + BITBUCKET_NAMESPACE=$CI_PROJECT_NAMESPACE + fi + if [ -z "$BITBUCKET_REPOSITORY" ]; then + echo "Setting BITBUCKET_REPOSITORY to $CI_PROJECT_NAME" + BITBUCKET_REPOSITORY=$CI_PROJECT_NAME + fi + + BITBUCKET_API_ROOT="https://api.bitbucket.org/2.0" + BITBUCKET_STATUS_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/commit/$CI_COMMIT_SHA/statuses/build" + BITBUCKET_KEY="ci/gitlab-ci/$CI_JOB_NAME" + + case "$BUILD_STATUS" in + running) + BITBUCKET_STATE="INPROGRESS" + BITBUCKET_DESCRIPTION="The build is running!" + ;; + passed) + BITBUCKET_STATE="SUCCESSFUL" + BITBUCKET_DESCRIPTION="The build passed!" + ;; + failed) + BITBUCKET_STATE="FAILED" + BITBUCKET_DESCRIPTION="The build failed." + ;; + esac + + echo "Pushing status to $BITBUCKET_STATUS_API..." + curl --request POST $BITBUCKET_STATUS_API \ + --user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \ + --header "Content-Type:application/json" \ + --silent \ + --data "{ \"state\": \"$BITBUCKET_STATE\", \"key\": \"$BITBUCKET_KEY\", \"description\": + \"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }" + ``` 1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push pipeline success and failures to Bitbucket. - ```yaml - stages: - - test - - ci_status - - unit-tests: - script: - - echo "Success. Add your tests!" - - success: - stage: ci_status - before_script: - - "" - after_script: - - "" - script: - - BUILD_STATUS=passed BUILD_KEY=push ./build_status - when: on_success - - failure: - stage: ci_status - before_script: - - "" - after_script: - - "" - script: - - BUILD_STATUS=failed BUILD_KEY=push ./build_status - when: on_failure - ``` + ```yaml + stages: + - test + - ci_status + + unit-tests: + script: + - echo "Success. Add your tests!" + + success: + stage: ci_status + before_script: + - "" + after_script: + - "" + script: + - BUILD_STATUS=passed BUILD_KEY=push ./build_status + when: on_success + + failure: + stage: ci_status + before_script: + - "" + after_script: + - "" + script: + - BUILD_STATUS=failed BUILD_KEY=push ./build_status + when: on_failure + ``` GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines configured in `.gitlab-ci.yml` and push the status to Bitbucket. diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 0bb3aa35ed0..f639e3dadee 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -19,12 +19,12 @@ administrator: 1. In GitLab create a **CI/CD for external repo** project and select **GitHub**. -  +  1. Once authenticated, you will be redirected to a list of your repositories to connect. Click **Connect** to select the repository. -  +  1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). @@ -55,14 +55,14 @@ repositories: Token**. This token with be used to access your repository and push commit statuses to GitHub. - The `repo` and `admin:repo_hook` should be enable to allow GitLab access to - your project, update commit statuses, and create a web hook to notify - GitLab of new commits. + The `repo` and `admin:repo_hook` should be enable to allow GitLab access to + your project, update commit statuses, and create a web hook to notify + GitLab of new commits. 1. In GitLab create a **CI/CD for external repo** project and select **GitHub**. -  +  1. Paste the token into the **Personal access token** field and click **List Repositories**. Click **Connect** to select the repository. @@ -86,21 +86,21 @@ your repository: Access Token.** GitLab will use this token to access your repository and push commit statuses. - Enter a **Token description** and update the scope to allow: + Enter a **Token description** and update the scope to allow: - `repo` so that GitLab can access your project and update commit statuses + `repo` so that GitLab can access your project and update commit statuses 1. In GitLab create a **CI/CD project** using the Git URL option and the HTTPS URL for your GitHub repository. If your project is private, use the personal access token you just created for authentication. - GitLab will automatically configure polling-based pull mirroring. + GitLab will automatically configure polling-based pull mirroring. 1. Still in GitLab, enable the [GitHub project integration](../../user/project/integrations/github.md) from **Settings > Integrations.** - Check the **Active** checkbox to enable the integration, paste your - personal access token and HTTPS repository URL into the form, and **Save.** + Check the **Active** checkbox to enable the integration, paste your + personal access token and HTTPS repository URL into the form, and **Save.** 1. Still in GitLab create a **Personal Access Token** with `API` scope to authenticate the GitHub web hook notifying GitLab of new commits. @@ -108,15 +108,15 @@ your repository: 1. In GitHub from **Settings > Webhooks** create a web hook to notify GitLab of new commits. - The web hook URL should be set to the GitLab API to - [trigger pull mirroring](../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter), - using the GitLab personal access token we just created. + The web hook URL should be set to the GitLab API to + [trigger pull mirroring](../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter), + using the GitLab personal access token we just created. - ``` - https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> - ``` + ``` + https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> + ``` -  +  1. In GitHub add a `.gitlab-ci.yml` to configure GitLab CI/CD. diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md new file mode 100644 index 00000000000..1c38c08b7cb --- /dev/null +++ b/doc/ci/directed_acyclic_graph/index.md @@ -0,0 +1,76 @@ +--- +type: reference +--- + +# Directed Acyclic Graph + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/47063) in GitLab 12.2 (enabled by `ci_dag_support` feature flag). + +A [directed acyclic graph](https://www.techopedia.com/definition/5739/directed-acyclic-graph-dag) can be +used in the context of a CI/CD pipeline to build relationships between jobs such that +execution is performed in the quickest possible manner, regardless how stages may +be set up. + +For example, you may have a specific tool or separate website that is built +as part of your main project. Using a DAG, you can specify the relationship between +these jobs and GitLab will then execute the jobs as soon as possible instead of waiting +for each stage to complete. + +Unlike other DAG solutions for CI/CD, GitLab does not require you to choose one or the +other. You can implement a hybrid combination of DAG and traditional +stage-based operation within a single pipeline. Configuration is kept very simple, +requiring a single keyword to enable the feature for any job. + +Consider a monorepo as follows: + +``` +./service_a +./service_b +./service_c +./service_d +``` + +It has a pipeline that looks like the following: + +| build | test | deploy | +| ----- | ---- | ------ | +| build_a | test_a | deploy_a | +| build_b | test_b | deploy_b | +| build_c | test_c | deploy_c | +| build_d | test_d | deploy_d | + +Using a DAG, you can relate the `_a` jobs to each other separately from the `_b` jobs, +and even if service `a` takes a very long time to build, service `b` will not +wait for it and will finish as quickly as it can. In this very same pipeline, `_c` and +`_d` can be left alone and will run together in staged sequence just like any normal +GitLab pipeline. + +## Use cases + +A DAG can help solve several different kinds of relationships between jobs within +a CI/CD pipeline. Most typically this would cover when jobs need to fan in or out, +and/or merge back together (diamond dependencies). This can happen when you're +handling multi-platform builds or complex webs of dependencies as in something like +an operating system build or a complex deployment graph of independently deployable +but related microservices. + +Additionally, a DAG can help with general speediness of pipelines and helping +to deliver fast feedback. By creating dependency relationships that don't unnecessarily +block each other, your pipelines will run as quickly as possible regardless of +pipeline stages, ensuring output (including errors) is available to developers +as quickly as possible. + +## Usage + +Relationships are defined between jobs using the [`needs:` keyword](../yaml/README.md#needs). + +Note that `needs:` also works with the [parallel](../yaml/README.md#parallel) keyword, +giving you powerful options for parallelization within your pipeline. + +## Limitations + +A directed acyclic graph is a complicated feature, and as of the initial MVC there +are certain use cases that you may need to work around. For more information: + +- [`needs` requirements and limitations](../yaml/README.md#requirements-and-limitations). +- Related epic [gitlab-org#1716](https://gitlab.com/groups/gitlab-org/-/epics/1716). diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index efdcaf5a6f5..2cbad5f101c 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -6,7 +6,6 @@ type: concepts, howto GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects. - One of the new trends in Continuous Integration/Deployment is to: 1. Create an application image. @@ -29,7 +28,16 @@ during jobs. ## Runner Configuration -There are three methods to enable the use of `docker build` and `docker run` during jobs; each with their own tradeoffs. +There are three methods to enable the use of `docker build` and `docker run` +during jobs; each with their own tradeoffs. + +An alternative to using `docker build` is to [use kaniko](using_kaniko.md). +This avoids having to execute Runner in privileged mode. + +TIP: **Tip:** +To see how Docker and Runner are configured for shared Runners on +GitLab.com, see [GitLab.com Shared +Runners](../../user/gitlab_com/index.md#shared-runners). ### Use shell executor @@ -40,42 +48,42 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. 1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor shell \ - --description "My Runner" - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor shell \ + --description "My Runner" + ``` 1. Install Docker Engine on server. - For more information how to install Docker Engine on different systems - checkout the [Supported installations](https://docs.docker.com/engine/installation/). + For more information how to install Docker Engine on different systems + checkout the [Supported installations](https://docs.docker.com/engine/installation/). 1. Add `gitlab-runner` user to `docker` group: - ```bash - sudo usermod -aG docker gitlab-runner - ``` + ```bash + sudo usermod -aG docker gitlab-runner + ``` 1. Verify that `gitlab-runner` has access to Docker: - ```bash - sudo -u gitlab-runner -H docker info - ``` + ```bash + sudo -u gitlab-runner -H docker info + ``` - You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: + You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: - ```yaml - before_script: - - docker info + ```yaml + before_script: + - docker info - build_image: - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + build_image: + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` 1. You can now use `docker` command (and **install** `docker-compose` if needed). @@ -83,101 +91,28 @@ NOTE: **Note:** By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). -### Use docker-in-docker executor +### Use docker-in-docker workflow with Docker executor The second approach is to use the special docker-in-docker (dind) [Docker image](https://hub.docker.com/_/docker/) with all tools installed (`docker`) and run the job script in context of that -image in privileged mode. - -NOTE: **Note:** `docker-compose` is not part of docker-in-docker (dind). In case you'd like to use `docker-compose` in your CI builds, please follow the [installation instructions for docker-compose](https://docs.docker.com/compose/install/) provided by docker. - -In order to do that, follow the steps: +image in privileged mode. -1. Install [GitLab Runner](https://docs.gitlab.com/runner/install). - -1. Register GitLab Runner from the command line to use `docker` and `privileged` - mode: - - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:stable" \ - --docker-privileged - ``` - - The above command will register a new Runner to use the special - `docker:stable` image which is provided by Docker. **Notice that it's using - the `privileged` mode to start the build and service containers.** If you - want to use [docker-in-docker] mode, you always have to use `privileged = true` - in your Docker containers. - - The above command will create a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:stable" - privileged = true - disable_cache = false - volumes = ["/cache"] - [runners.cache] - Insecure = false - ``` - -1. You can now use `docker` in the build script (note the inclusion of the - `docker:dind` service): - - ```yaml - image: docker:stable +NOTE: **Note:** +`docker-compose` is not part of docker-in-docker (dind). To use `docker-compose` in your +CI builds, follow the `docker-compose` +[installation instructions](https://docs.docker.com/compose/install/). - variables: - # When using dind service we need to instruct docker, to talk with the - # daemon started inside of the service. The daemon is available with - # a network connection instead of the default /var/run/docker.sock socket. - # - # The 'docker' hostname is the alias of the service container as described at - # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services - # - # Note that if you're using the Kubernetes executor, the variable should be set to - # tcp://localhost:2375/ because of how the Kubernetes executor connects services - # to the job container - # DOCKER_HOST: tcp://localhost:2375/ - # - # For non-Kubernetes executors, we use tcp://docker:2375/ - DOCKER_HOST: tcp://docker:2375/ - # When using dind, it's wise to use the overlayfs driver for - # improved performance. - DOCKER_DRIVER: overlay2 - - services: - - docker:dind - - before_script: - - docker info - - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` +DANGER: **Danger:** +By enabling `--docker-privileged`, you are effectively disabling all of +the security mechanisms of containers and exposing your host to privilege +escalation which can lead to container breakout. For more information, check +out the official Docker documentation on +[Runtime privilege and Linux capabilities][docker-cap]. Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges: -- By enabling `--docker-privileged`, you are effectively disabling all of - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. For more information, check - out the official Docker documentation on - [Runtime privilege and Linux capabilities][docker-cap]. - When using docker-in-docker, each job is in a clean environment without the past history. Concurrent jobs work fine because every build gets it's own instance of Docker engine so they won't conflict with each other. But this @@ -187,7 +122,7 @@ not without its own challenges: [Using the overlayfs driver](#using-the-overlayfs-driver). - Since the `docker:dind` container and the runner container don't share their root filesystem, the job's working directory can be used as a mount point for - children containers. For example, if you have files you want to share with a + child containers. For example, if you have files you want to share with a child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH` and use it as your mount point (for a more thorough explanation, check [issue #41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)): @@ -203,6 +138,177 @@ not without its own challenges: An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. +In the examples below, we are using Docker images tags to specify a +specific version, such as `docker:19.03.1`. If tags like `docker:stable` +are used, you have no control over what version is going to be used and this +can lead to unpredictable behavior, especially when new versions are +released. + +#### TLS enabled + +NOTE: **Note** +This requires GitLab Runner 11.11 or higher. + +The Docker daemon supports connection over TLS and it's done by default +for Docker 19.03.1 or higher. This is the **suggested** way to use the +docker-in-docker service and +[GitLab.com Shared Runners](../../user/gitlab_com/index.html#shared-runners) +support this. + +1. Install [GitLab Runner](https://docs.gitlab.com/runner/install). + +1. Register GitLab Runner from the command line to use `docker` and `privileged` + mode: + + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:19.03.1" \ + --docker-privileged \ + --docker-volumes "/certs/client" + ``` + + The above command will register a new Runner to use the special + `docker:19.03.1` image, which is provided by Docker. **Notice that it's + using the `privileged` mode to start the build and service + containers.** If you want to use [docker-in-docker] mode, you always + have to use `privileged = true` in your Docker containers. + + This will also mount `/certs/client` for the service and build + container, which is needed for the docker client to use the + certificates inside of that directory. For more information how + Docker with TLS works check <https://hub.docker.com/_/docker/#tls>. + + The above command will create a `config.toml` entry similar to this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.1" + privileged = true + disable_cache = false + volumes = ["/certs/client", "/cache"] + [runners.cache] + [runners.cache.s3] + [runners.cache.gcs] + ``` + +1. You can now use `docker` in the build script (note the inclusion of the + `docker:19.03.1-dind` service): + + ```yaml + image: docker:19.03.1 + + variables: + # When using dind service, we need to instruct docker, to talk with + # the daemon started inside of the service. The daemon is available + # with a network connection instead of the default + # /var/run/docker.sock socket. docker:19.03.1 does this automatically + # by setting the DOCKER_HOST in + # https://github.com/docker-library/docker/blob/d45051476babc297257df490d22cbd806f1b11e4/19.03.1/docker-entrypoint.sh#L23-L29 + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. + # + # Note that if you're using the Kubernetes executor, the variable + # should be set to tcp://localhost:2376/ because of how the + # Kubernetes executor connects services to the job container + # DOCKER_HOST: tcp://localhost:2376/ + # + # When using dind, it's wise to use the overlayfs driver for + # improved performance. + DOCKER_DRIVER: overlay2 + # Specify to Docker where to create the certificates, Docker will + # create them automatically on boot, and will create + # `/certs/client` that will be shared between the service and job + # container, thanks to volume mount from config.toml + DOCKER_TLS_CERTDIR: "/certs" + + services: + - docker:19.03.1-dind + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +#### TLS disabled + +Sometimes there are legitimate reasons why you might want to disable TLS. +For example, you have no control over the GitLab Runner configuration +that you are using. + +Assuming that the Runner `config.toml` is similar to: + +```toml +[[runners]] + url = "https://gitlab.com/" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.1" + privileged = true + disable_cache = false + volumes = ["/cache"] + [runners.cache] + [runners.cache.s3] + [runners.cache.gcs] +``` + +You can now use `docker` in the build script (note the inclusion of the +`docker:19.03.1-dind` service): + +```yaml +image: docker:19.03.1 + +variables: + # When using dind service we need to instruct docker, to talk with the + # daemon started inside of the service. The daemon is available with + # a network connection instead of the default /var/run/docker.sock socket. + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services + # + # Note that if you're using the Kubernetes executor, the variable should be set to + # tcp://localhost:2375/ because of how the Kubernetes executor connects services + # to the job container + # DOCKER_HOST: tcp://localhost:2375/ + # + # For non-Kubernetes executors, we use tcp://docker:2375/ + DOCKER_HOST: tcp://docker:2375/ + # When using dind, it's wise to use the overlayfs driver for + # improved performance. + DOCKER_DRIVER: overlay2 + # + # This will instruct Docker not to start over TLS. + DOCKER_TLS_CERTDIR: "" + +services: + - docker:19.03.1-dind + +before_script: + - docker info + +build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` + ### Use Docker socket binding The third approach is to bind-mount `/var/run/docker.sock` into the @@ -220,54 +326,54 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:stable" \ - --docker-volumes /var/run/docker.sock:/var/run/docker.sock - ``` - - The above command will register a new Runner to use the special - `docker:stable` image which is provided by Docker. **Notice that it's using - the Docker daemon of the Runner itself, and any containers spawned by docker - commands will be siblings of the Runner rather than children of the runner.** - This may have complications and limitations that are unsuitable for your workflow. - - The above command will create a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = REGISTRATION_TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:stable" - privileged = false - disable_cache = false - volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] - [runners.cache] - Insecure = false - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:stable" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock + ``` + + The above command will register a new Runner to use the special + `docker:stable` image which is provided by Docker. **Notice that it's using + the Docker daemon of the Runner itself, and any containers spawned by docker + commands will be siblings of the Runner rather than children of the runner.** + This may have complications and limitations that are unsuitable for your workflow. + + The above command will create a `config.toml` entry similar to this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = REGISTRATION_TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:stable" + privileged = false + disable_cache = false + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] + [runners.cache] + Insecure = false + ``` 1. You can now use `docker` in the build script (note that you don't need to include the `docker:dind` service as when using the Docker in Docker executor): - ```yaml - image: docker:stable + ```yaml + image: docker:stable - before_script: - - docker info + before_script: + - docker info - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` While the above method avoids using Docker in privileged mode, you should be aware of the following implications: @@ -283,9 +389,9 @@ aware of the following implications: work as expected since volume mounting is done in the context of the host machine, not the build container. For example: - ```sh - docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests - ``` + ```sh + docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests + ``` ## Making docker-in-docker builds faster with Docker layer caching @@ -356,23 +462,23 @@ which can be avoided if a different driver is used, for example `overlay2`. 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: - ```sh - sudo lsmod | grep overlay - ``` + ```sh + sudo lsmod | grep overlay + ``` - If you see no result, then it isn't loaded. To load it use: + If you see no result, then it isn't loaded. To load it use: - ```sh - sudo modprobe overlay - ``` + ```sh + sudo modprobe overlay + ``` - If everything went fine, you need to make sure module is loaded on reboot. - On Ubuntu systems, this is done by editing `/etc/modules`. Just add the - following line into it: + If everything went fine, you need to make sure module is loaded on reboot. + On Ubuntu systems, this is done by editing `/etc/modules`. Just add the + following line into it: - ```text - overlay - ``` + ```text + overlay + ``` ### Use driver per project @@ -440,9 +546,9 @@ For all projects, mostly suitable for public ones: your Docker images and has read/write access to the Registry. This is ephemeral, so it's only valid for one job. You can use the following example as-is: - ```sh - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - ``` + ```sh + docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + ``` For private and internal projects: @@ -455,9 +561,9 @@ For private and internal projects: Replace the `<username>` and `<access_token>` in the following example: - ```sh - docker login -u <username> -p <access_token> $CI_REGISTRY - ``` + ```sh + docker login -u <username> -p <access_token> $CI_REGISTRY + ``` - **Using the GitLab Deploy Token**: You can create and use a [special deploy token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token) @@ -465,9 +571,9 @@ For private and internal projects: Once created, you can use the special environment variables, and GitLab CI/CD will fill them in for you. You can use the following example as-is: - ```sh - docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY - ``` + ```sh + docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY + ``` ### Container Registry examples diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 2d7fb323d79..489791141ed 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -35,8 +35,8 @@ sudo gitlab-runner register \ --description "docker-ruby-2.1" \ --executor "docker" \ --docker-image ruby:2.1 \ - --docker-postgres latest \ - --docker-mysql latest + --docker-services postgres:latest \ + --docker-services mysql:latest ``` The registered runner will use the `ruby:2.1` Docker image and will run two @@ -193,13 +193,14 @@ You can simply define an image that will be used for all jobs and a list of services that you want to use during build time: ```yaml -image: ruby:2.2 +default: + image: ruby:2.2 -services: - - postgres:9.3 + services: + - postgres:9.3 -before_script: - - bundle install + before_script: + - bundle install test: script: @@ -209,8 +210,9 @@ test: It is also possible to define different images and services per job: ```yaml -before_script: - - bundle install +default: + before_script: + - bundle install test:2.1: image: ruby:2.1 @@ -231,16 +233,53 @@ Or you can pass some [extended configuration options](#extended-docker-configura for `image` and `services`: ```yaml +default: + image: + name: ruby:2.2 + entrypoint: ["/bin/bash"] + + services: + - name: my-postgres:9.4 + alias: db-postgres + entrypoint: ["/usr/local/bin/db-postgres"] + command: ["start"] + + before_script: + - bundle install + +test: + script: + - bundle exec rake spec +``` + +## Passing environment variables to services + +You can also pass custom environment [variables](../variables/README.md) +to fine tune your Docker `images` and `services` directly in the `.gitlab-ci.yml` file. +For more information, see [custom environment variables](../variables/README.md#gitlab-ciyml-defined-variables) + +```yaml +# The following variables will automatically be passed down to the Postgres container +# as well as the Ruby container and available within each. +variables: + HTTPS_PROXY: "https://10.1.1.1:8090" + HTTP_PROXY: "https://10.1.1.1:8090" + POSTGRES_DB: "my_custom_db" + POSTGRES_USER: "postgres" + POSTGRES_PASSWORD: "example" + PGDATA: "/var/lib/postgresql/data" + POSTGRES_INITDB_ARGS: "--encoding=UTF8 --data-checksums" + +services: +- name: postgres:9.4 + alias: db + entrypoint: ["docker-entrypoint.sh"] + command: ["postgres"] + image: name: ruby:2.2 entrypoint: ["/bin/bash"] -services: -- name: my-postgres:9.4 - alias: db-postgres - entrypoint: ["/usr/local/bin/db-postgres"] - command: ["start"] - before_script: - bundle install @@ -266,25 +305,25 @@ For example, the following two definitions are equal: 1. Using a string as an option to `image` and `services`: - ```yaml - image: "registry.example.com/my/image:latest" + ```yaml + image: "registry.example.com/my/image:latest" - services: - - postgresql:9.4 - - redis:latest - ``` + services: + - postgresql:9.4 + - redis:latest + ``` 1. Using a map as an option to `image` and `services`. The use of `image:name` is required: - ```yaml - image: - name: "registry.example.com/my/image:latest" + ```yaml + image: + name: "registry.example.com/my/image:latest" - services: - - name: postgresql:9.4 - - name: redis:latest - ``` + services: + - name: postgresql:9.4 + - name: redis:latest + ``` ### Available settings for `image` @@ -456,14 +495,6 @@ that runner. ## Define an image from a private Container Registry -> **Notes:** -> -> - This feature requires GitLab Runner **1.8** or higher -> - For GitLab Runner versions **>= 0.6, <1.8** there was a partial -> support for using private registries, which required manual configuration -> of credentials on runner's host. We recommend to upgrade your Runner to -> at least version **1.8** if you want to use private registries. - To access private container registries, the GitLab Runner process can use: - [Statically defined credentials](#using-statically-defined-credentials). That is, a username and password for a specific registry. @@ -486,7 +517,19 @@ it's provided as an environment variable. This is because GitLab Runnner uses ** `config.toml` configuration and doesn't interpolate **ANY** environment variables at runtime. +### Requirements and limitations + +- This feature requires GitLab Runner **1.8** or higher. +- For GitLab Runner versions **>= 0.6, <1.8** there was a partial + support for using private registries, which required manual configuration + of credentials on runner's host. We recommend to upgrade your Runner to + at least version **1.8** if you want to use private registries. +- Not available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html), + follow <https://gitlab.com/gitlab-org/gitlab-runner/issues/2673> for + details. + ### Using statically-defined credentials + There are two approaches that you can take in order to access a private registry. Both require setting the environment variable `DOCKER_AUTH_CONFIG` with appropriate authentication info. @@ -516,18 +559,18 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: - **First way -** Do a `docker login` on your local machine: - ```bash - docker login registry.example.com:5000 --username my_username --password my_password - ``` + ```bash + docker login registry.example.com:5000 --username my_username --password my_password + ``` - Then copy the content of `~/.docker/config.json`. + Then copy the content of `~/.docker/config.json`. - If you don't need access to the registry from your computer, you - can do a `docker logout`: + If you don't need access to the registry from your computer, you + can do a `docker logout`: - ```bash - docker logout registry.example.com:5000 - ``` + ```bash + docker logout registry.example.com:5000 + ``` - **Second way -** In some setups, it's possible that Docker client will use the available system keystore to store the result of `docker @@ -536,12 +579,12 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: `${username}:${password}` manually. Open a terminal and execute the following command: - ```bash - echo -n "my_username:my_password" | base64 + ```bash + echo -n "my_username:my_password" | base64 - # Example output to copy - bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= - ``` + # Example output to copy + bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= + ``` #### Configuring a job @@ -551,25 +594,25 @@ follow these steps: 1. Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: - ```json - { - "auths": { - "registry.example.com:5000": { - "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" - } - } - } - ``` + ```json + { + "auths": { + "registry.example.com:5000": { + "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" + } + } + } + ``` 1. You can now use any private image from `registry.example.com:5000` defined in `image` and/or `services` in your `.gitlab-ci.yml` file: - ```yaml - image: registry.example.com:5000/namespace/image:tag - ``` + ```yaml + image: registry.example.com:5000/namespace/image:tag + ``` - In the example above, GitLab Runner will look at `registry.example.com:5000` for the - image `namespace/image:tag`. + In the example above, GitLab Runner will look at `registry.example.com:5000` for the + image `namespace/image:tag`. You can add configuration for as many registries as you want, adding more registries to the `"auths"` hash as described above. @@ -598,10 +641,10 @@ To add `DOCKER_AUTH_CONFIG` to a Runner: 1. Modify the Runner's `config.toml` file as follows: - ```toml - [[runners]] - environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"] - ``` + ```toml + [[runners]] + environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"] + ``` 1. Restart the Runner service. @@ -623,16 +666,17 @@ To configure credentials store, follow these steps: Make sure helper program is available in GitLab Runner `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: + - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the - Docker configuration file as the value: + Docker configuration file as the value: - ```json - { - "credsStore": "osxkeychain" - } - ``` + ```json + { + "credsStore": "osxkeychain" + } + ``` - Or, if you are running self-hosted Runners, add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this config file @@ -654,17 +698,18 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th 1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: + - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the - Docker configuration file as the value: + Docker configuration file as the value: - ```json - { - "credHelpers": { - "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" - } - } - ``` + ```json + { + "credHelpers": { + "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" + } + } + ``` - Or, if you are running self-hosted Runners, add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. @@ -674,12 +719,12 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th 1. You can now use any private image from `aws_account_id.dkr.ecr.region.amazonaws.com` defined in `image` and/or `services` in your `.gitlab-ci.yml` file: - ```yaml - image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest - ``` + ```yaml + image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest + ``` - In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the - image `private/image:latest`. + In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the + image `private/image:latest`. You can add configuration for as many registries as you want, adding more registries to the `"credHelpers"` hash as described above. diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 50f1ac3d54a..925653f9fdf 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -11,7 +11,8 @@ Requires GitLab Runner 11.2 and above. container images from a Dockerfile, inside a container or Kubernetes cluster. kaniko solves two problems with using the -[docker-in-docker build](using_docker_build.md#use-docker-in-docker-executor) method: +[docker-in-docker +build](using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor) method: - Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) in order to function, which is a significant security concern. diff --git a/doc/ci/environments.md b/doc/ci/environments.md index f86ca8f74f2..f6c47a99712 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -591,15 +591,13 @@ exist, you should see something like: ### Monitoring environments -> **Notes:** -> -> - For the monitoring dashboard to appear, you need to: -> - Enable the [Prometheus integration](../user/project/integrations/prometheus.md). -> - Configure Prometheus to collect at least one [supported metric](../user/project/integrations/prometheus_library/index.md). -> - With GitLab 9.2, all deployments to an environment are shown directly on the monitoring dashboard. - If you have enabled [Prometheus for monitoring system and response metrics](../user/project/integrations/prometheus.md), -you can monitor the behavior of your app running in each environment. +you can monitor the behavior of your app running in each environment. For the monitoring +dashboard to appear, you need to Configure Prometheus to collect at least one +[supported metric](../user/project/integrations/prometheus_library/index.md). + +NOTE: **Note:** +Since GitLab 9.2, all deployments to an environment are shown directly on the monitoring dashboard. Once configured, GitLab will attempt to retrieve [supported performance metrics](../user/project/integrations/prometheus_library/index.md) for any environment that has had a successful deployment. If monitoring data was @@ -621,6 +619,10 @@ versions of the app, all without leaving GitLab. Add a [button to the Monitoring dashboard](../user/project/operations/linking_to_an_external_dashboard.md) linking directly to your existing external dashboards. +#### Embedding metrics in GitLab Flavored Markdown + +Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab Flavored Markdown](../user/project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) for more details. + ### Web terminals > Web terminals were added in GitLab 8.15 and are only available to project Maintainers and Owners. @@ -692,7 +694,7 @@ with `review/` would have that particular variable. Some GitLab features can behave differently for each environment. For example, you can -[create a secret variable to be injected only into a production environment](variables/README.md#limiting-environment-scopes-of-environment-variables-premium). **(PREMIUM)** +[create a secret variable to be injected only into a production environment](variables/README.md#limiting-environment-scopes-of-environment-variables). In most cases, these features use the _environment specs_ mechanism, which offers an efficient way to implement scoping within each environment group. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 5a302392c54..00995f881da 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -33,10 +33,11 @@ The following table lists examples with step-by-step tutorials that are containe | Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | | PHP with PHPunit, atoum | [Testing PHP projects](php.md). | | PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with Laravel, Ennvoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | | Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | | Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | | Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| Parallel testing Ruby & JS | [GitLab CI parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | ### Contributing examples diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index c9f700ed190..e5f307e570d 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -1,5 +1,5 @@ --- -redirect_from: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' +disqus_identifier: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' author: Fabio Busatto author_gitlab: bikebilly level: intermediate @@ -39,9 +39,10 @@ project: 1. Create a new project by selecting **Import project from ➔ Repo by URL** 1. Add the following URL: - ``` - https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git - ``` + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git + ``` + 1. Click **Create project** This application is nothing more than a basic class with a stub for a JUnit based test suite. @@ -63,15 +64,15 @@ The application is ready to use, but you need some additional steps to deploy it 1. Copy the snippet in the `pom.xml` file for your project, just after the `dependencies` section. The snippet should look like this: - ```xml - <distributionManagement> - <repository> - <id>central</id> - <name>83d43b5afeb5-releases</name> - <url>${env.MAVEN_REPO_URL}/libs-release-local</url> - </repository> - </distributionManagement> - ``` + ```xml + <distributionManagement> + <repository> + <id>central</id> + <name>83d43b5afeb5-releases</name> + <url>${env.MAVEN_REPO_URL}/libs-release-local</url> + </repository> + </distributionManagement> + ``` Another step you need to do before you can deploy the dependency to Artifactory is to configure the authentication data. It is a simple task, but Maven requires @@ -86,18 +87,18 @@ parameter in `.gitlab-ci.yml` to use the custom location instead of the default 1. Create a file called `settings.xml` in the `.m2` folder 1. Copy the following content into a `settings.xml` file: - ```xml - <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" - xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> - <servers> - <server> - <id>central</id> - <username>${env.MAVEN_REPO_USER}</username> - <password>${env.MAVEN_REPO_PASS}</password> - </server> - </servers> - </settings> - ``` + ```xml + <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" + xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> + <servers> + <server> + <id>central</id> + <username>${env.MAVEN_REPO_USER}</username> + <password>${env.MAVEN_REPO_PASS}</password> + </server> + </servers> + </settings> + ``` Username and password will be replaced by the correct values using variables. @@ -187,9 +188,10 @@ We'll use again a Maven app that can be cloned from our example project: 1. Create a new project by selecting **Import project from ➔ Repo by URL** 1. Add the following URL: - ``` - https://gitlab.com/gitlab-examples/maven/simple-maven-app.git - ``` + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-app.git + ``` + 1. Click **Create project** This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` @@ -204,13 +206,13 @@ Since Maven doesn't know how to resolve the dependency, you need to modify the c 1. Copy the snippet in the `dependencies` section of the `pom.xml` file. The snippet should look like this: - ```xml - <dependency> - <groupId>com.example.dep</groupId> - <artifactId>simple-maven-dep</artifactId> - <version>1.0</version> - </dependency> - ``` + ```xml + <dependency> + <groupId>com.example.dep</groupId> + <artifactId>simple-maven-dep</artifactId> + <version>1.0</version> + </dependency> + ``` ### Configure the Artifactory repository location diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index 8ecac4a5a4f..3266e5dc62e 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -15,7 +15,7 @@ your code by using GitLab CI/CD and [sitespeed.io](https://www.sitespeed.io) using Docker-in-Docker. First, you need GitLab Runner with -[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). +[docker-in-docker build](../docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). Once you set up the Runner, add a new job to `.gitlab-ci.yml` that generates the expected report: @@ -155,4 +155,4 @@ performance: paths: - performance.json - sitespeed-results/ -```
\ No newline at end of file +``` diff --git a/doc/ci/examples/code_quality.md b/doc/ci/examples/code_quality.md index 43f773dab7c..9c65de115b4 100644 --- a/doc/ci/examples/code_quality.md +++ b/doc/ci/examples/code_quality.md @@ -1,5 +1,5 @@ --- -redirect_from: 'https://docs.gitlab.com/ee/ci/examples/code_climate.html' +disqus_identifier: 'https://docs.gitlab.com/ee/ci/examples/code_climate.html' type: reference, howto --- @@ -14,7 +14,7 @@ This example shows how to run Code Quality on your code by using GitLab CI/CD and Docker. First, you need GitLab Runner with -[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). Once you set up the Runner, include the CodeQuality template in your CI config: @@ -34,6 +34,12 @@ For [GitLab Starter][ee] users, this information will be automatically extracted and shown right in the merge request widget. [Learn more on Code Quality in merge requests](../../user/project/merge_requests/code_quality.md). +CAUTION: **Caution:** +On self-managed instances, if a malicious actor compromises the Code Quality job +definition they will be able to execute privileged docker commands on the Runner +host. Having proper access control policies mitigates this attack vector by +allowing access only to trusted actors. + ## Previous job definitions CAUTION: **Caution:** diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md index 538843ab8dc..1d4c9221cf2 100644 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -79,10 +79,10 @@ image: java:8 stages: - build - deploy - + before_script: - chmod +x mvnw - + build: stage: build script: ./mvnw package diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 50e61cafeb9..44d3ec8046c 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -418,10 +418,14 @@ fully understand [IAM Best Practices in AWS](https://docs.aws.amazon.com/IAM/lat 1. Log into your AWS account and go to the [Security Credentials page](https://console.aws.amazon.com/iam/home#/security_credential) 1. Click the **Access Keys** section and **Create New Access Key**. Create the key and keep the id and secret around, you'll need them later -  + +  + 1. Go to your GitLab project, click **Settings > CI/CD** on the left sidebar 1. Expand the **Variables** section -  + +  + 1. Add a key named `AWS_KEY_ID` and copy the key id from Step 2 into the **Value** textbox 1. Add a key named `AWS_KEY_SECRET` and copy the key secret from Step 2 into the **Value** textbox diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/img/deployed_dependency_update.png b/doc/ci/examples/end_to_end_testing_webdriverio/img/deployed_dependency_update.png Binary files differindex c45d70d7f7a..9fe1739f37e 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/img/deployed_dependency_update.png +++ b/doc/ci/examples/end_to_end_testing_webdriverio/img/deployed_dependency_update.png diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 38fcf05f519..38d0d86ffa2 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -143,7 +143,7 @@ Which brings us to the exciting part: how do we run this in GitLab CI/CD? There need to do for this: 1. Set up [CI/CD jobs](../../yaml/README.md#introduction) that actually have a browser available. -2. Update our WebdriverIO configuration to use those browsers to visit the review apps. +1. Update our WebdriverIO configuration to use those browsers to visit the review apps. For the scope of this article, we've defined an additional [CI/CD stage](../../yaml/README.md#stages) `confidence-check` that is executed _after_ the stage that deploys the review app. It uses the `node:latest` [Docker diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index d7308a3a5ec..808e4285f2f 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -1,5 +1,5 @@ --- -redirect_from: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' +disqus_identifier: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' author: Mehran Rasulian author_gitlab: mehranrasulian level: intermediate @@ -452,6 +452,7 @@ To start using Container Registry on our machine, we first need to login to the ```bash docker login registry.gitlab.com ``` + Then we can build and push our image to GitLab: ```bash diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 0b9e9e93e55..1dd3049d53d 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -6,7 +6,7 @@ type: tutorial This guide covers basic building instructions for PHP projects. -Two testing scenarios are covered: using the Docker executor and +Two testing scenarios are covered: using the Docker executor and using the Shell executor. ## Test PHP projects using the Docker executor diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md index a5fed00972f..0e595e1a0be 100644 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md @@ -188,28 +188,27 @@ when running our Phoenix in our `localhost`. - Open `hello_gitlab_ci/config/test.exs` on your favorite code editor - Go to **Configure your database** session and edit the block to include `System.get_env`: - ```elixir - # Configure your database - config :hello_gitlab_ci, HelloGitlabCi.Repo, - adapter: Ecto.Adapters.Postgres, - username: System.get_env("POSTGRES_USER") || "postgres", - password: System.get_env("POSTGRES_PASSWORD") || "postgres", - database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test", - hostname: System.get_env("POSTGRES_HOST") || "localhost", - pool: Ecto.Adapters.SQL.Sandbox - ``` - - We'll need these system variables later on. + ```elixir + # Configure your database + config :hello_gitlab_ci, HelloGitlabCi.Repo, + adapter: Ecto.Adapters.Postgres, + username: System.get_env("POSTGRES_USER") || "postgres", + password: System.get_env("POSTGRES_PASSWORD") || "postgres", + database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test", + hostname: System.get_env("POSTGRES_HOST") || "localhost", + pool: Ecto.Adapters.SQL.Sandbox + ``` + + We'll need these system variables later on. - Create an empty file named `.gitkeep` into `hello_gitlab_ci/priv/repo/migrations` - As our project is still fresh, we don't have any data on our database, so, the `migrations` -directory will be empty. - Without `.gitkeep`, git will not upload this empty directory and we'll got an error when running our -test on GitLab. + As our project is still fresh, we don't have any data on our database, so, the `migrations` + directory will be empty. + Without `.gitkeep`, git will not upload this empty directory and we'll got an error when running our + test on GitLab. - > **Note:** - If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir. + > **Note:** If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir. Now, let's run a local test and see if everything we did didn't break anything. @@ -248,64 +247,64 @@ project. - The fastest and easiest way to do this, is to click on **Set up CI** on project's main page: -  +  - On next screen, we can select a template ready to go. Click on **Apply a GitLab CI/CD Yaml template** and select **Elixir**: -  +  - This template file tells GitLab CI/CD about what we wish to do every time a new commit is made. - However, we have to adapt it to run a Phoenix app. + This template file tells GitLab CI/CD about what we wish to do every time a new commit is made. + However, we have to adapt it to run a Phoenix app. - The first line tells GitLab what Docker image will be used. - Remember when we learn about Runners, the isolated virtual machine where GitLab CI/CD build and test - our application? This virtual machine must have all dependencies to run our application. This is - where a Docker image is needed. The correct image will provide the entire system for us. + Remember when we learn about Runners, the isolated virtual machine where GitLab CI/CD build and test + our application? This virtual machine must have all dependencies to run our application. This is + where a Docker image is needed. The correct image will provide the entire system for us. - As a suggestion, you can use [trenpixster's elixir image][docker-image], which already has all - dependencies for Phoenix installed, such as Elixir, Erlang, NodeJS and PostgreSQL: + As a suggestion, you can use [trenpixster's elixir image][docker-image], which already has all + dependencies for Phoenix installed, such as Elixir, Erlang, NodeJS and PostgreSQL: - ```yml - image: trenpixster/elixir:latest - ``` + ```yml + image: trenpixster/elixir:latest + ``` - At `services` session, we'll only use `postgres`, so we'll delete `mysql` and `redis` lines: - ```yml - services: - - postgres:latest - ``` + ```yml + services: + - postgres:latest + ``` - Now, we'll create a new entry called `variables`, before `before_script` session: - ```yml - variables: - POSTGRES_DB: hello_gitlab_ci_test - POSTGRES_HOST: postgres - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "postgres" - MIX_ENV: "test" - ``` + ```yml + variables: + POSTGRES_DB: hello_gitlab_ci_test + POSTGRES_HOST: postgres + POSTGRES_USER: postgres + POSTGRES_PASSWORD: "postgres" + MIX_ENV: "test" + ``` - Here, we are setting up the values for GitLab CI/CD authenticate into PostgreSQL, as we did on - `config/test.exs` earlier. + Here, we are setting up the values for GitLab CI/CD authenticate into PostgreSQL, as we did on + `config/test.exs` earlier. - In `before_script` session, we'll add some commands to prepare everything to the test: - ```yml - before_script: - - apt-get update && apt-get -y install postgresql-client - - mix local.hex --force - - mix deps.get --only test - - mix ecto.create - - mix ecto.migrate - ``` - - It's important to install `postgresql-client` to let GitLab CI/CD access PostgreSQL and create our - database with the login information provided earlier. More important is to respect the indentation, - to avoid syntax errors when running the build. + ```yml + before_script: + - apt-get update && apt-get -y install postgresql-client + - mix local.hex --force + - mix deps.get --only test + - mix ecto.create + - mix ecto.migrate + ``` + + It's important to install `postgresql-client` to let GitLab CI/CD access PostgreSQL and create our + database with the login information provided earlier. More important is to respect the indentation, + to avoid syntax errors when running the build. - And finally, we'll let `mix` session intact. diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 1354a26d6e2..cce33c7a6b4 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -69,12 +69,14 @@ correctly with your CI jobs: 1. Next, if you are using `gitlab-runner` v1.10+, you can set the `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` to tell the runner to fetch your submodules before the job: - ```yaml - variables: - GIT_SUBMODULE_STRATEGY: recursive - ``` - See the [`.gitlab-ci.yml` reference](yaml/README.md#git-submodule-strategy) - for more details about `GIT_SUBMODULE_STRATEGY`. + + ```yaml + variables: + GIT_SUBMODULE_STRATEGY: recursive + ``` + + See the [`.gitlab-ci.yml` reference](yaml/README.md#git-submodule-strategy) + for more details about `GIT_SUBMODULE_STRATEGY`. 1. If you are using an older version of `gitlab-runner`, then use `git submodule sync/update` in `before_script`: diff --git a/doc/ci/img/collapsible_log.png b/doc/ci/img/collapsible_log.png Binary files differindex 2785033b349..d2a570e246e 100644 --- a/doc/ci/img/collapsible_log.png +++ b/doc/ci/img/collapsible_log.png diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differindex 12090434bef..9e2b7e89577 100644 --- a/doc/ci/img/deployments_view.png +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environments_available.png b/doc/ci/img/environments_available.png Binary files differindex 48fc6effc2d..6c64e9398f7 100644 --- a/doc/ci/img/environments_available.png +++ b/doc/ci/img/environments_available.png diff --git a/doc/ci/img/environments_mr_review_app.png b/doc/ci/img/environments_mr_review_app.png Binary files differindex 6a7b7ce5679..86c20d8d3b6 100644 --- a/doc/ci/img/environments_mr_review_app.png +++ b/doc/ci/img/environments_mr_review_app.png diff --git a/doc/ci/img/manual_job_variables.png b/doc/ci/img/manual_job_variables.png Binary files differnew file mode 100644 index 00000000000..a5ed351fdcd --- /dev/null +++ b/doc/ci/img/manual_job_variables.png diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index 6c73cbbf8d7..58307660e51 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -8,7 +8,7 @@ type: reference Interactive web terminals give the user access to a terminal in GitLab for running one-off commands for their CI pipeline. Since this is giving the user -shell access to the environment where [GitLab Runner](https://docs.gitlab.com/runner/) +shell access to the environment where [GitLab Runner](https://docs.gitlab.com/runner/) is deployed, some [security precautions](../../administration/integration/terminal.md#security) were taken to protect the users. @@ -62,4 +62,3 @@ close the terminal window. ## Interactive Web Terminals for the Web IDE **(ULTIMATE ONLY)** Read the Web IDE docs to learn how to run [Interactive Terminals through the Web IDE](../../user/project/web_ide/index.md). - diff --git a/doc/ci/introduction/img/gitlab_workflow_example_11_9.png b/doc/ci/introduction/img/gitlab_workflow_example_11_9.png Binary files differindex 204e9c462e5..f3fb9444b55 100644 --- a/doc/ci/introduction/img/gitlab_workflow_example_11_9.png +++ b/doc/ci/introduction/img/gitlab_workflow_example_11_9.png diff --git a/doc/ci/introduction/img/gitlab_workflow_example_extended_11_11.png b/doc/ci/introduction/img/gitlab_workflow_example_extended_11_11.png Binary files differindex 5089a1088c5..a0874f66eaa 100644 --- a/doc/ci/introduction/img/gitlab_workflow_example_extended_11_11.png +++ b/doc/ci/introduction/img/gitlab_workflow_example_extended_11_11.png diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index fc89f0fc94f..366aca3442e 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -9,6 +9,11 @@ In this document we'll present an overview of the concepts of Continuous Integra Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD. +NOTE: **Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more.** +Watch our +["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. + ## Introduction to CI/CD methodologies The continuous methodologies of software development are based on @@ -146,14 +151,14 @@ so, GitLab CI/CD: - Runs automated scripts (sequential or parallel) to: - Build and test your app. - Preview the changes per merge request with Review Apps, as you - would see in your `localhost`. + would see in your `localhost`. Once you're happy with your implementation: - Get your code reviewed and approved. - Merge the feature branch into the default branch. - GitLab CI/CD deploys your changes automatically to a production environment. -- And finally, you and your team can easily roll it back if something goes wrong. +- And finally, you and your team can easily roll it back if something goes wrong.  @@ -176,24 +181,24 @@ you'll see some of the features available in GitLab according to each stage (Verify, Package, Release). 1. **Verify**: - - Automatically build and test your application with Continuous Integration. - - Analyze your source code quality with [GitLab Code Quality](../../user/project/merge_requests/code_quality.md). **(STARTER)** - - Determine the performance impact of code changes with [Browser Performance Testing](../../user/project/merge_requests/browser_performance_testing.md). **(PREMIUM)** - - Perform a series of tests, such as [Container Scanning](../../user/application_security/container_scanning/index.md) **(ULTIMATE)**, [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) **(ULTIMATE)**, and [JUnit tests](../junit_test_reports.md). - - Deploy your changes with [Review Apps](../review_apps/index.md) to preview the app changes on every branch. + - Automatically build and test your application with Continuous Integration. + - Analyze your source code quality with [GitLab Code Quality](../../user/project/merge_requests/code_quality.md). **(STARTER)** + - Determine the performance impact of code changes with [Browser Performance Testing](../../user/project/merge_requests/browser_performance_testing.md). **(PREMIUM)** + - Perform a series of tests, such as [Container Scanning](../../user/application_security/container_scanning/index.md) **(ULTIMATE)**, [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) **(ULTIMATE)**, and [JUnit tests](../junit_test_reports.md). + - Deploy your changes with [Review Apps](../review_apps/index.md) to preview the app changes on every branch. 1. **Package**: - - Store Docker images with [Container Registry](../../user/project/container_registry.md). - - Store NPM packages with [NPM Registry](../../user/project/packages/npm_registry.md). **(PREMIUM)** - - Store Maven artifacts with [Maven Repository](../../user/project/packages/maven_repository.md). **(PREMIUM)** + - Store Docker images with [Container Registry](../../user/project/container_registry.md). + - Store NPM packages with [NPM Registry](../../user/project/packages/npm_registry.md). **(PREMIUM)** + - Store Maven artifacts with [Maven Repository](../../user/project/packages/maven_repository.md). **(PREMIUM)** 1. **Release**: - - Continuous Deployment, automatically deploying your app to production. - - Continuous Delivery, manually click to deploy your app to production. - - Deploy static websites with [GitLab Pages](../../user/project/pages/index.md). - - Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature with [Canary Deployments](../../user/project/canary_deployments.md). **(PREMIUM)** - - Deploy your features behind [Feature Flags](../../user/project/operations/feature_flags.md). **(PREMIUM)** - - Add release notes to any Git tag with [GitLab Releases](../../user/project/releases/index.md). - - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). **(PREMIUM)** - - Deploy your application to a production environment in a Kubernetes cluster with [Auto Deploy](../../topics/autodevops/index.md#auto-deploy). + - Continuous Deployment, automatically deploying your app to production. + - Continuous Delivery, manually click to deploy your app to production. + - Deploy static websites with [GitLab Pages](../../user/project/pages/index.md). + - Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature with [Canary Deployments](../../user/project/canary_deployments.md). **(PREMIUM)** + - Deploy your features behind [Feature Flags](../../user/project/operations/feature_flags.md). **(PREMIUM)** + - Add release notes to any Git tag with [GitLab Releases](../../user/project/releases/index.md). + - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). **(PREMIUM)** + - Deploy your application to a production environment in a Kubernetes cluster with [Auto Deploy](../../topics/autodevops/index.md#auto-deploy). With GitLab CI/CD you can also: diff --git a/doc/ci/jenkins/index.md b/doc/ci/jenkins/index.md new file mode 100644 index 00000000000..f8a3fab88e3 --- /dev/null +++ b/doc/ci/jenkins/index.md @@ -0,0 +1,232 @@ +--- +comments: false +type: index, howto +--- + +# Migrating from Jenkins + +A lot of GitLab users have successfully migrated to GitLab CI/CD from Jenkins. To make this +easier if you're just getting started, we've collected several resources here that you might find useful +before diving in. + +First of all, our [Quick Start Guide](../quick_start/README.md) contains a good overview of how GitLab CI/CD works. +You may also be interested in [Auto DevOps](../../topics/autodevops/index.md) which can potentially be used to build, test, +and deploy your applications with little to no configuration needed at all. + +Otherwise, read on for important information that will help you get the ball rolling. Welcome +to GitLab! + +## Important differences + +There are some high level differences between the products worth mentioning: + +- With GitLab you don't need a root `pipeline` keyword to wrap everything. +- All jobs within a single stage always run in parallel, and all stages run in sequence. We are planning + to allow certain jobs to break this sequencing as needed with our [directed acyclic graph](https://gitlab.com/gitlab-org/gitlab-ce/issues/47063) + feature. +- The `.gitlab-ci.yml` file is checked in to the root of your repository, much like a Jenkinsfile, but + is in the YAML format (see [complete reference](../yaml/README.md)) instead of a Groovy DSL. It's most + analagous to the declarative Jenkinsfile format. +- GitLab comes with a [container registry](../../user/project/container_registry.md), and we recommend using + container images to set up your build environment. + +## Groovy vs. YAML + +Jenkins Pipelines are based on [Groovy](https://groovy-lang.org/), so the pipeline specification is written as code. +GitLab works a bit differently, we use the more highly structured [YAML](https://yaml.org/) format, which +places scripting elements inside of `script:` blocks separate from the pipeline specification itself. + +This is a strength of GitLab, in that it helps keep the learning curve much simpler to get up and running +and avoids some of the problem of unconstrained complexity which can make your Jenkinsfiles hard to understand +and manage. + +That said, we do of course still value DRY (don't repeat yourself) principles and want to ensure that +behaviors of your jobs can be codified once and applied as needed. You can use the `extends:` syntax to +[templatize your jobs](../yaml/README.md#extends), and `include:` can be used to [bring in entire sets of behaviors](../yaml/README.md#include) +to pipelines in different projects. + +```yaml +.in-docker: + tags: + - docker + image: alpine + +rspec: + extends: + - .in-docker + script: + - rake rspec +``` + +## Artifact publishing + +Artifacts may work a bit differently than you've used them with Jenkins. In GitLab, any job can define +a set of artifacts to be saved by using the `artifacts:` keyword. This can be configured to point to a file +or set of files that can then be persisted from job to job. Read more on our detailed [artifacts documentation](../../user/project/pipelines/job_artifacts.html) + +```yaml +pdf: + script: xelatex mycv.tex + artifacts: + paths: + - ./mycv.pdf + - ./output/ + expire_in: 1 week +``` + +Additionally, we have package management features like a built-in container, NPM, and Maven registry that you +can leverage. You can see the complete list of packaging features (which includes links to documentation) +in the [Packaging section of our documentation](../../README.md#package). + +## Integrated features + +Where you may have used plugins to get things like code quality, unit tests, security scanning, and so on working in Jenkins, +GitLab takes advantage of our connected ecosystem to automatically pull these kinds of results into +your Merge Requests, pipeline details pages, and other locations. You may find that you actually don't +need to configure anything to have these appear. + +If they aren't working as expected, or if you'd like to see what's available, our [CI feature index](../README.md#feature-set) has the full list +of bundled features and links to the documentation for each. + +## Converting Declarative Jenkinsfiles + +Declarative Jenkinsfiles contain "Sections" and "Directives" which are used to control the behavior of your +pipelines. There are equivalents for all of these in GitLab, which we've documented below. + +This section is based on the [Jenkinsfile syntax documentation](https://jenkins.io/doc/book/pipeline/syntax/) +and is meant to be a mapping of concepts there to concepts in GitLab. + +### Sections + +#### `agent` + +The agent section is used to define how a pipeline will be executed. For GitLab, we use the [GitLab Runner](../runners/README.md) +to provide this capability. You can configure your own runners in Kubernetes or on any host, or take advantage +of our shared runner fleet (note that the shared runner fleet is only available for GitLab.com users.) The link above will bring you to the documenation which will describe how to get +up and running quickly. We also support using [tags](../runners/README.md#using-tags) to direct different jobs +to different Runners (execution agents). + +The `agent` section also allows you to define which Docker images should be used for execution, for which we use +the [`image`](../yaml/README.md#image) keyword. The `image` can be set on a single job or at the top level, in which +case it will apply to all jobs in the pipeline. + +```yaml +my_job: + image: alpine + ... +``` + +#### `post` + +The `post` section defines the actions that should be performed at the end of the pipeline. GitLab also supports +this through the use of stages. You can define your stages as follows, and any jobs assigned to the `before_pipeline` +or `after_pipeline` stages will run as expected. You can call these stages anything you like. + +```yaml +stages: + - before_pipeline + - build + - test + - deploy + - after_pipeline +``` + +Setting a step to be performed before and after any job can be done via the [`before_script` and `after_script` keywords](../yaml/README.md#before_script-and-after_script). + +```yaml +default: + before_script: + - echo "I run before any jobs starts in the entire pipeline, and can be responsible for setting up the environment." +``` + +#### `stages` + +GitLab CI also lets you define stages, but is a little bit more free-form to configure. The GitLab [`stages` keyword](../yaml/README.md#stages) +is a top level setting that enumerates the list of stages, but you are not required to nest individual jobs underneath +the `stages` section. Any job defined in the `.gitlab-ci.yml` can be made a part of any stage through use of the +[`stage:` keyword](../yaml/README.md#stage). + +Note that, unless otherwise specified, every pipeline is instantiated with a `build`, `test`, and `deploy` stage +which are run in that order. Jobs that have no `stage` defined are placed by default in the `test` stage. +Of course, each job that refers to a stage must refer to a stage that exists in the pipeline configuration. + +```yaml +stages: + - build + - test + - deploy + +my_job: + stage: build + ... +``` + +#### `steps` + +The `steps` section is equivalent to the [`script` section](../yaml/README.md#script) of an individual job. This is +a simple YAML array with each line representing an individual command to be run. + +```yaml +my_job: + script: + - echo "hello! the current time is:" + - time + ... +``` + +### Directives + +#### `environment` + +In GitLab, we use the [`variables` keyword](../yaml/README.md#variables) to define different variables at runtime. +These can also be set up through the GitLab UI, under CI/CD settings. See also our [general documentation on variables](../variables/README.md), +including the section on [protected variables](../variables/README.md#protected-environment-variables) which can be used +to limit access to certain variables to certain environments or runners. + +```yaml +variables: + POSTGRES_USER: user + POSTGRES_PASSWORD: testing_password +``` + +#### `options` + +Here, options for different things exist associated with the object in question itself. For example, options related +to jobs are defined in relation to the job itself. If you're looking for a certain option, you should be able to find +where it's located by searching our [complete configuration reference](../yaml/README.md) page. + +#### `parameters` + +GitLab does not require you to define which variables you want to be available when starting a manual job. A user +can provide any variables they like. + +#### `triggers` / `cron` + +Because GitLab is integrated tightly with git, SCM polling options for triggers are not needed. We support an easy to use +[syntax for scheduling pipelines](../../user/project/pipelines/schedules.md). + +#### `tools` + +GitLab does not support a separate `tools` directive. Our best-practice reccomendation is to use pre-built +container images, which can be cached, and can be built to already contain the tools you need for your pipelines. Pipelines can +be set up to automatically build these images as needed and deploy them to the [container registry](../../user/project/container_registry.md). + +If you're not using container images with Docker/Kubernetes, for example on Mac or FreeBSD, then the `shell` executor does require you to +set up your environment either in advance or as part of the jobs. You could create a `before_script` +action that handles this for you. + +#### `input` + +Similar to the `parameters` keyword, this is not needed because a manual job can always be provided runtime +variable entry. + +#### `when` + +GitLab does support a [`when` keyword](../yaml/README.md#when) which is used to indicate when a job should be +run in case of (or despite) failure, but most of the logic for controlling pipelines can be found in +our very powerful [`only/except` rules system](../yaml/README.md#onlyexcept-basic) (see also our [advanced syntax](../yaml/README.md#onlyexcept-basic)) + +```yaml +my_job: + only: branches +```
\ No newline at end of file diff --git a/doc/ci/merge_request_pipelines/img/merge_request.png b/doc/ci/merge_request_pipelines/img/merge_request.png Binary files differindex d03fdc6a885..bb64e17cc91 100644 --- a/doc/ci/merge_request_pipelines/img/merge_request.png +++ b/doc/ci/merge_request_pipelines/img/merge_request.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline.png Binary files differindex 58d5581f628..6d4b66824e1 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline_config.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline_config.png Binary files differindex 0a84e61d284..3ee9d8ec93b 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline_config.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merge_request_pipeline_config.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index a13857bee25..126e12e460f 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -6,7 +6,6 @@ last_update: 2019-07-03 # Pipelines for Merged Results **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/7380) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.10. -> This feature is disabled by default until we resolve issues with [contention handling](https://gitlab.com/gitlab-org/gitlab-ee/issues/11222), but [can be enabled manually](#enabling-pipelines-for-merged-results). It's possible for your source and target branches to diverge, which can result in the scenario that source branch's pipeline was green, the target's pipeline was green, @@ -25,8 +24,10 @@ new ref and a pipeline for this ref validates the result prior to merging. There are some cases where creating a combined ref is not possible or not wanted. For example, a source branch that has conflicts with the target branch -or a merge request that is still in WIP status. In this case, the merge request pipeline falls back to a "detached" state -and runs on the source branch ref as if it was a regular pipeline. +or a merge request that is still in WIP status. In this case, +GitLab doesn't create a merge commit and the pipeline runs on source branch, instead, +which is a default behavior of [Pipelines for merge requests](../index.md) + i.e. `detached` label is shown to the pipelines. The detached state serves to warn you that you are working in a situation subjected to merge problems, and helps to highlight that you should @@ -34,15 +35,14 @@ get out of WIP status or resolve merge conflicts as soon as possible. ## Requirements and limitations -Pipelines for merged results require: +Pipelines for merged results require a [GitLab Runner][runner] 11.9 or newer. -- [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or newer. -- [Gitaly](https://gitlab.com/gitlab-org/gitaly) 1.21.0 or newer. +[runner]: https://gitlab.com/gitlab-org/gitlab-runner In addition, pipelines for merged results have the following limitations: - Forking/cross-repo workflows are not currently supported. To follow progress, - see [#9713](https://gitlab.com/gitlab-org/gitlab-ee/issues/9713). + see [#11934](https://gitlab.com/gitlab-org/gitlab-ee/issues/11934). - This feature is not available for [fast forward merges](../../../user/project/merge_requests/fast_forward_merge.md) yet. To follow progress, see [#58226](https://gitlab.com/gitlab-org/gitlab-ce/issues/58226). @@ -61,18 +61,32 @@ CAUTION: **Warning:** Make sure your `gitlab-ci.yml` file is [configured properly for pipelines for merge requests](../index.md#configuring-pipelines-for-merge-requests), otherwise pipelines for merged results won't run and your merge requests will be stuck in an unresolved state. -## Merge Trains **(PREMIUM)** +## Troubleshooting -Read the [documentation on Merge Trains](merge_trains/index.md). +### Pipelines for merged results not created even with new change pushed to merge request -<!-- ## Troubleshooting +Can be caused by some disabled feature flags. Please make sure that +the following feature flags are enabled on your GitLab instance: -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +- `:ci_use_merge_request_ref` +- `:merge_ref_auto_sync` -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +To check these feature flag values, please ask administrator to execute the following commands: + +```shell +> sudo gitlab-rails console # Login to Rails console of GitLab instance. +> Feature.enabled?(:ci_use_merge_request_ref) # Check if it's enabled or not. +> Feature.enable(:ci_use_merge_request_ref) # Enable the feature flag. +``` + +## Using Merge Trains **(PREMIUM)** + +By enabling [Pipelines for merged results](#pipelines-for-merged-results-premium), +GitLab will [automatically display](merge_trains/index.md#how-to-add-a-merge-request-to-a-merge-train) +a **Start/Add Merge Train button** as the most recommended merge strategy. + +Generally, this is a safer option than merging merge requests immediately as your +merge request will be evaluated with an expected post-merge result before the actual +merge happens. + +For more information, read the [documentation on Merge Trains](merge_trains/index.md). diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_cancel_v12_0.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_cancel_v12_0.png Binary files differindex 1561fdcc7a5..d7720ac1143 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_cancel_v12_0.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_cancel_v12_0.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_config_v12_0.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_config_v12_0.png Binary files differindex fb0af43556e..9da959ad440 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_config_v12_0.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_config_v12_0.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_failure.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_failure.png Binary files differnew file mode 100644 index 00000000000..8a795fff432 --- /dev/null +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_failure.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge.png Binary files differnew file mode 100644 index 00000000000..03bc61129ba --- /dev/null +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_position_v12_0.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_position_v12_0.png Binary files differnew file mode 100644 index 00000000000..ec4b157d428 --- /dev/null +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_position_v12_0.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_v12_0.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_v12_0.png Binary files differindex f20108157d2..a4d0c8cf0e6 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_v12_0.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_v12_0.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_when_pipeline_succeeds_v12_0.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_when_pipeline_succeeds_v12_0.png Binary files differindex 62c2f2f5ff5..45762b8e85e 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_when_pipeline_succeeds_v12_0.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_start_when_pipeline_succeeds_v12_0.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index 57358434c02..80a1c264bc4 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -6,7 +6,6 @@ last_update: 2019-07-03 # Merge Trains **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9186) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.0. -> This feature is disabled by default, but [can be enabled manually](#enabling-merge-trains). [Pipelines for merged results](../index.md#pipelines-for-merged-results-premium) introduces running a build on the result of the merged code prior to merging, as a way to keep master green. @@ -33,35 +32,40 @@ Merge trains have the following requirements and limitations: - This feature requires that [pipelines for merged results](../index.md#pipelines-for-merged-results-premium) are **configured properly**. -- Each merge train can generate a merge ref and run a pipeline **one at a time**. - We plan to make the pipelines for merged results - [run in parallel](https://gitlab.com/gitlab-org/gitlab-ee/issues/11222) in a - future release. +- Each merge train can run a maximum of **four** pipelines in parallel. + If more than four merge requests are added to the merge train, the merge requests + will be queued until a slot in the merge train is free. There is no limit to the + number of merge requests that can be queued. +- This feature does not support [squash and merge](../../../../user/project/merge_requests/squash_and_merge.md). -## Enabling Merge Trains - -To enable merge trains at the project level: - -1. Visit your project's **Settings > General** and expand **Merge requests**. -1. Check **Allow merge trains**. -1. Click **Save changes** button. - - +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +Watch this video for a demonstration on [how parallel execution +of Merge Trains can prevent commits from breaking the default +branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ). ## How to add a merge request to a merge train To add a merge request to a merge train: -1. Click "Start/Add merge train" button in a merge request +1. Visit a merge request. +1. Click the **Start/Add to merge train** button.  ## How to remove a merge request from a merge train -1. Click "Remove from merge train" button in the merge request widget. +1. Visit a merge request. +1. Click the **Remove from merge train** button.  +## How to view a merge request's current position on the merge train + +After a merge request has been added to the merge train, the merge request's +current position will be displayed under the pipeline widget: + + + ## Start/Add to merge train when pipeline succeeds You can add a merge request to a merge train only when the latest pipeline in the @@ -70,19 +74,71 @@ the merge request to a train because the current change of the merge request may be broken thus it could affect the following merge requests. In this case, you can schedule to add the merge request to a merge train **when the latest -pipeline succeeds**. You can see the following button instead of the regular "Start/Add merge train" +pipeline succeeds** (This pipeline is [Pipelines for merged results](../index.md), not Pipelines for merge train). +You can see the following button instead of the regular **Start/Add to merge train** button while the latest pipeline is running.  -<!-- ## Troubleshooting +## Immediately merge a merge request with a merge train + +In case, you have a high-priority merge request (e.g. critical patch) to be merged urgently, +you can use **Merge Immediately** option for bypassing the merge train. +This is the fastest option to get the change merged into the target branch. + + + +However, every time you merge a merge request immediately, it could affect the +existing merge train to be reconstructed, specifically, it regenerates expected +merge commits and pipelines. This means, merging immediately essentially wastes +CI resources. + +## Troubleshooting + +### Merge request dropped from the merge train immediately + +If a merge request is not mergeable (for example, it's WIP, there is a merge +conflict, etc), your merge request will be dropped from the merge train automatically. + +In these cases, the reason for dropping the merge request is in the **system notes**. + +To check the reason: + +1. Open the merge request that was dropped from the merge train. +1. Open the **Discussion** tab. +1. Find a system note that includes either: + - The text **... removed this merge request from the merge train because ...** + - **... aborted this merge request from the merge train because ...** + The reason is given in the text after the **because ...** phrase. + + + +### Merge When Pipeline Succeeds cannot be chosen + +[Merge When Pipeline Succeeds](../../../../user/project/merge_requests/merge_when_pipeline_succeeds.md) +is unavailable when +[Pipelines for Merged Results is enabled](../index.md#enabling-pipelines-for-merged-results). + +Follow [this issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/12267) to +track progress on this issue. + +### Merge Train disturbs your workflow + +First of all, please check if [merge immediately](#immediately-merge-a-merge-request-with-a-merge-train) +is available as a workaround in your workflow. This is the most recommended +workaround you'd be able to take immediately. If it's not available or acceptable, +please read through this section. + +Merge train is enabled by default when you enable [Pipelines for merged results](../index.md), +however, you can forcibly disable this feature by disabling the feature flag `:merge_trains_enabled`. +After you disabled this feature, all the existing merge trains will be aborted and +you will no longer see the **Start/Add Merge Train** button in merge requests. -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +To check if the feature flag is enabled on your GitLab instance, +please ask administrator to execute the following commands: -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +```shell +> sudo gitlab-rails console # Login to Rails console of GitLab instance. +> Feature.enabled?(:merge_trains_enabled) # Check if it's enabled or not. +> Feature.disable(:merge_trains_enabled) # Disable the feature flag. +```
\ No newline at end of file diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 463b9194c58..61f260cb70d 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -171,6 +171,26 @@ In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the upstream pipeline will be passed to the `downstream-job` job, and will be available within the context of all downstream builds. +NOTE: **Tip:** +Upstream pipelines take precedence over downstream ones. If there are two +variables with the same name defined in both upstream and downstream projects, +the ones defined in the upstream project will take precedence. + +### Mirroring status from upstream pipeline + +You can mirror the pipeline status from an upstream pipeline to a bridge job by +using the `needs:pipeline` keyword. The latest pipeline status from master is +replicated to the bridge job. + +Example: + +```yaml +upstream_bridge: + stage: test + needs: + pipeline: other/project +``` + ### Limitations Because bridge jobs are a little different to regular jobs, it is not @@ -185,5 +205,5 @@ Some features are not implemented yet. For example, support for environments. - `stage` - `allow_failure` - `only` and `except` -- `when` +- `when` (only with `on_success`, `on_failure`, and `always` values) - `extends` diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 06a81c3d0c7..eaa6efc526d 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -6,6 +6,11 @@ type: reference > Introduced in GitLab 8.8. +NOTE: **Tip:** +Watch our +["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +webcast to see a comprehensive demo of GitLab CI/CD pipeline. + ## Introduction Pipelines are the top-level component of continuous integration, delivery, and deployment. @@ -318,6 +323,20 @@ stage has a job with a manual action.  +### Specifying variables when running manual jobs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/30485) in GitLab 12.2. + +When running manual jobs you can supply additional job specific variables. + +You can do this from the job page of the manual job you want to run with +additional variables. + +This is useful when you want to alter the execution of a job by using +environment variables. + + + ### Delay a job in a pipeline graph > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21767) in GitLab 11.4. @@ -345,7 +364,7 @@ GitLab provides API endpoints to: - [Triggering pipelines through the API](triggers/README.md). - [Pipeline triggers API](../api/pipeline_triggers.md). -###Â Start multiple manual actions in a stage +### Start multiple manual actions in a stage > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27188) in GitLab 11.11. @@ -358,6 +377,10 @@ This functionality is only available: - For users with at least Developer access. - If the the stage contains [manual actions](#manual-actions-from-pipeline-graphs). +## Most Recent Pipeline + +There's a link to the latest pipeline for the last commit of a given branch at `/project/pipelines/[branch]/latest`. Also, `/project/pipelines/latest` will redirect you to the latest pipeline for the last commit on the project's default branch. + ## Security on protected branches A strict security model is enforced when pipelines are executed on diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 0480b83d183..5626428bfc3 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -13,6 +13,10 @@ NOTE: **Note:** Please keep in mind that only project Maintainers and Admin users have the permissions to access a project's settings. +NOTE: **Note:** +Coming over to GitLab from Jenkins? Check out our [reference](../jenkins/index.md) +for converting your pre-existing pipelines over to our format. + GitLab offers a [continuous integration][ci] service. If you [add a `.gitlab-ci.yml` file][yaml] to the root directory of your repository, and configure your GitLab project to use a [Runner], then each commit or diff --git a/doc/ci/quick_start/img/build_log.png b/doc/ci/quick_start/img/build_log.png Binary files differindex 2bf0992c50e..16698629edc 100644 --- a/doc/ci/quick_start/img/build_log.png +++ b/doc/ci/quick_start/img/build_log.png diff --git a/doc/ci/quick_start/img/builds_status.png b/doc/ci/quick_start/img/builds_status.png Binary files differindex 58978e23978..b4aeeb988d2 100644 --- a/doc/ci/quick_start/img/builds_status.png +++ b/doc/ci/quick_start/img/builds_status.png diff --git a/doc/ci/quick_start/img/pipelines_status.png b/doc/ci/quick_start/img/pipelines_status.png Binary files differindex 06d1559f5d2..39a77a26b25 100644 --- a/doc/ci/quick_start/img/pipelines_status.png +++ b/doc/ci/quick_start/img/pipelines_status.png diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png Binary files differindex cd83c1a7e4c..ac09e1d0137 100644 --- a/doc/ci/quick_start/img/runners_activated.png +++ b/doc/ci/quick_start/img/runners_activated.png diff --git a/doc/ci/review_apps/img/review_button.png b/doc/ci/review_apps/img/review_button.png Binary files differindex 0b231c50858..4f1cf4d7cfd 100644 --- a/doc/ci/review_apps/img/review_button.png +++ b/doc/ci/review_apps/img/review_button.png diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 9b89988bf42..8ab7982fd65 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -147,11 +147,11 @@ Once you have the route mapping set up, it will take effect in the following loc - In the diff for a merge request, comparison, or commit. -  +  - In the blob file view. -  +  ## Visual Reviews **(STARTER)** diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 03a219e03ca..269bd5c3428 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -62,7 +62,7 @@ You can only register a shared Runner if you are an admin of the GitLab instance 1. Grab the shared-Runner token on the `admin/runners` page -  +  1. [Register the Runner][register] @@ -88,7 +88,7 @@ visit the project you want to make the Runner work for in GitLab: ## Registering a group Runner -Creating a group Runner requires Maintainer permissions for the group. To create a +Creating a group Runner requires Owner permissions for the group. To create a group Runner visit the group you want to make the Runner work for in GitLab: 1. Go to **Settings > CI/CD** to obtain the token @@ -124,9 +124,9 @@ To lock/unlock a Runner: ## Assigning a Runner to another project -If you are Maintainer on a project where a specific Runner is assigned to, and the +If you are an Owner on a project where a specific Runner is assigned to, and the Runner is not [locked only to that project](#locking-a-specific-runner-from-being-enabled-for-other-projects), -you can enable the Runner also on any other project where you have Maintainer permissions. +you can enable the Runner also on any other project where you have Owner permissions. To enable/disable a Runner in your project: @@ -250,7 +250,7 @@ When you [register a Runner][register], its default behavior is to **only pick** [tagged jobs](../yaml/README.md#tags). NOTE: **Note:** -Maintainer [permissions](../../user/permissions.md) are required to change the +Owner [permissions](../../user/permissions.md) are required to change the Runner settings. To make a Runner pick untagged jobs: @@ -319,21 +319,21 @@ How this feature will work: 1. You set the _maximum job timeout_ for a Runner to 24 hours 1. You set the _CI/CD Timeout_ for a project to **2 hours** 1. You start a job -1. The job, if running longer, will be timeouted after **2 hours** +1. The job, if running longer, will be timed out after **2 hours** **Example 2 - Runner timeout not configured** 1. You remove the _maximum job timeout_ configuration from a Runner 1. You set the _CI/CD Timeout_ for a project to **2 hours** 1. You start a job -1. The job, if running longer, will be timeouted after **2 hours** +1. The job, if running longer, will be timed out after **2 hours** **Example 3 - Runner timeout smaller than project timeout** 1. You set the _maximum job timeout_ for a Runner to **30 minutes** 1. You set the _CI/CD Timeout_ for a project to 2 hours 1. You start a job -1. The job, if running longer, will be timeouted after **30 minutes** +1. The job, if running longer, will be timed out after **30 minutes** ### Be careful with sensitive information @@ -373,12 +373,12 @@ attacker. To reset the token: -1. Go to **Settings > CI/CD** for a specified Project -1. Expand the **General pipelines settings** section -1. Find the **Runner token** form field and click the **Reveal value** button -1. Delete the value and save the form +1. Go to **Settings > CI/CD** for a specified Project. +1. Expand the **General pipelines settings** section. +1. Find the **Runner token** form field and click the **Reveal value** button. +1. Delete the value and save the form. 1. After the page is refreshed, expand the **Runners settings** section - and check the registration token - it should be changed + and check the registration token - it should be changed. From now on the old token is not valid anymore and will not allow to register a new Runner to the project. If you are using any tools to provision and diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 697452cee83..9ea113969c8 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -25,6 +25,12 @@ variables: MYSQL_ROOT_PASSWORD: "<your_mysql_password>" ``` +NOTE: **Note:** +The `MYSQL_DATABASE` and `MYSQL_ROOT_PASSWORD` variables can't be set in the GitLab UI. +To set them, assign them to a variable [in the UI](../variables/README.md#via-the-ui), +and then assign that variable to the +`MYSQL_DATABASE` and `MYSQL_ROOT_PASSWORD` variables in your `.gitlab-ci.yml`. + And then configure your application to use the database, for example: ```yaml diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 960346ac11a..142f4f262e5 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -25,6 +25,13 @@ variables: POSTGRES_PASSWORD: "" ``` +NOTE: **Note:** +The `POSTGRES_DB`, `POSTGRES_USER`, and `POSTGRES_PASSWORD` variables can't be set in +the GitLab UI. To set them, assign them to a variable +[in the UI](../variables/README.md#via-the-ui), and then assign that +variable to the `POSTGRES_DB`, `POSTGRES_USER`, and `POSTGRES_PASSWORD` variables in +your `.gitlab-ci.yml`. + And then configure your application to use the database, for example: ```yaml @@ -70,7 +77,7 @@ template1=# CREATE USER runner WITH PASSWORD '$password' CREATEDB; ``` *__Note:__ Notice that we created the user with the privilege to be able to -create databases (`CREATEDB`). In the following steps we will create a database +create databases (`CREATEDB`). In the following steps we will create a database explicitly for that user but having that privilege can be useful if in your testing framework you have tools that drop and create databases.* diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 69591ed605c..b6aebd3bd78 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -57,44 +57,44 @@ to access it. This is where an SSH key pair comes in handy. 1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following example, a Debian based image is assumed. Edit to your needs: - ```yaml - before_script: - ## - ## Install ssh-agent if not already installed, it is required by Docker. - ## (change apt-get to yum if you use an RPM-based image) - ## - - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - - ## - ## Run ssh-agent (inside the build environment) - ## - - eval $(ssh-agent -s) - - ## - ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store - ## We're using tr to fix line endings which makes ed25519 keys work - ## without extra base64 encoding. - ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556 - ## - - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null - - ## - ## Create the SSH directory and give it the right permissions - ## - - mkdir -p ~/.ssh - - chmod 700 ~/.ssh - - ## - ## Optionally, if you will be using any Git commands, set the user name and - ## and email. - ## - #- git config --global user.email "user@example.com" - #- git config --global user.name "User name" - ``` - - NOTE: **Note:** - The [`before_script`](../yaml/README.md#before_script-and-after_script) can be set globally - or per-job. + ```yaml + before_script: + ## + ## Install ssh-agent if not already installed, it is required by Docker. + ## (change apt-get to yum if you use an RPM-based image) + ## + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + + ## + ## Run ssh-agent (inside the build environment) + ## + - eval $(ssh-agent -s) + + ## + ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store + ## We're using tr to fix line endings which makes ed25519 keys work + ## without extra base64 encoding. + ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556 + ## + - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - + + ## + ## Create the SSH directory and give it the right permissions + ## + - mkdir -p ~/.ssh + - chmod 700 ~/.ssh + + ## + ## Optionally, if you will be using any Git commands, set the user name and + ## and email. + ## + #- git config --global user.email "user@example.com" + #- git config --global user.name "User name" + ``` + + NOTE: **Note:** + The [`before_script`](../yaml/README.md#before_script-and-after_script) can be set globally + or per-job. 1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys). @@ -118,9 +118,9 @@ on, and use that key for all projects that are run on this machine. 1. Then from the terminal login as the `gitlab-runner` user: - ``` - sudo su - gitlab-runner - ``` + ``` + sudo su - gitlab-runner + ``` 1. Generate the SSH key pair as described in the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair). diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index d1f9aa03b6b..2a382f18038 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -97,17 +97,6 @@ overview of the time the triggers were last used.  -## Taking ownership of a trigger - -> **Note**: -GitLab 9.0 introduced a trigger ownership to solve permission problems. - -Each created trigger when run will impersonate their associated user including -their access to projects and their project permissions. - -You can take ownership of existing triggers by clicking *Take ownership*. -From now on the trigger will be run as you. - ## Revoking a trigger You can revoke a trigger any time by going at your project's @@ -282,8 +271,7 @@ Old triggers, created before GitLab 9.0 will be marked as legacy. Triggers with the legacy label do not have an associated user and only have access to the current project. They are considered deprecated and will be -removed with one of the future versions of GitLab. You are advised to -[take ownership](#taking-ownership-of-a-trigger) of any legacy triggers. +removed with one of the future versions of GitLab. [ee-2017]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2017 [ee-2346]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2346 diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 42dd4f08ed8..5a15b907da0 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -94,7 +94,10 @@ This means that the value of the variable will be hidden in job logs, though it must match certain requirements to do so: - The value must be in a single line. -- The value must only consist of characters from the Base64 alphabet, defined in [RFC4648](https://tools.ietf.org/html/rfc4648). +- The value must only consist of characters from the Base64 alphabet (RFC4648). + + [In GitLab 12.2](https://gitlab.com/gitlab-org/gitlab-ce/issues/63043) + and newer, `@` and `:` are also valid values. - The value must be at least 8 characters long. - The value must not use variables. @@ -273,6 +276,7 @@ export CI_RUNNER_ID="10" export CI_RUNNER_DESCRIPTION="my runner" export CI_RUNNER_TAGS="docker, linux" export CI_SERVER="yes" +export CI_SERVER_HOST="example.com" export CI_SERVER_NAME="GitLab" export CI_SERVER_REVISION="70606bf" export CI_SERVER_VERSION="8.9.0" @@ -369,8 +373,11 @@ variables take precedence over those defined in `.gitlab-ci.yml`. ## Unsupported variables There are cases where some variables cannot be used in the context of a -`.gitlab-ci.yml` definition (for example under `script`). Read more -about which variables are [not supported](where_variables_can_be_used.md). +`.gitlab-ci.yml` definition (for example under `script`). Read more about which variables are [not supported](where_variables_can_be_used.md). + +## Where variables can be used + +Click [here](where_variables_can_be_used.md) for a section that describes where and how the different types of variables can be used. ## Advanced use @@ -389,7 +396,7 @@ Protected variables can be added by going to your project's Once you set them, they will be available for all subsequent pipelines. -### Limiting environment scopes of environment variables **(PREMIUM)** +### Limiting environment scopes of environment variables You can limit the environment scope of a variable by [defining which environments][envs] it can be available for. @@ -480,81 +487,86 @@ Below you can find supported syntax reference: 1. Equality matching using a string - > Example: `$VARIABLE == "some value"` + Examples: - > Example: `$VARIABLE != "some value"` (introduced in GitLab 11.11) + - `$VARIABLE == "some value"` + - `$VARIABLE != "some value"` (introduced in GitLab 11.11) - You can use equality operator `==` or `!=` to compare a variable content to a - string. We support both, double quotes and single quotes to define a string - value, so both `$VARIABLE == "some value"` and `$VARIABLE == 'some value'` - are supported. `"some value" == $VARIABLE` is correct too. + You can use equality operator `==` or `!=` to compare a variable content to a + string. We support both, double quotes and single quotes to define a string + value, so both `$VARIABLE == "some value"` and `$VARIABLE == 'some value'` + are supported. `"some value" == $VARIABLE` is correct too. 1. Checking for an undefined value - > Example: `$VARIABLE == null` + Examples: - > Example: `$VARIABLE != null` (introduced in GitLab 11.11) + - `$VARIABLE == null` + - `$VARIABLE != null` (introduced in GitLab 11.11) - It sometimes happens that you want to check whether a variable is defined - or not. To do that, you can compare a variable to `null` keyword, like - `$VARIABLE == null`. This expression is going to evaluate to truth if - variable is not defined when `==` is used, or to falsey if `!=` is used. + It sometimes happens that you want to check whether a variable is defined + or not. To do that, you can compare a variable to `null` keyword, like + `$VARIABLE == null`. This expression is going to evaluate to truth if + variable is not defined when `==` is used, or to falsey if `!=` is used. 1. Checking for an empty variable - > Example: `$VARIABLE == ""` + Examples: - > Example: `$VARIABLE != ""` (introduced in GitLab 11.11) + - `$VARIABLE == ""` + - `$VARIABLE != ""` (introduced in GitLab 11.11) - If you want to check whether a variable is defined, but is empty, you can - simply compare it against an empty string, like `$VAR == ''` or non-empty - string `$VARIABLE != ""`. + If you want to check whether a variable is defined, but is empty, you can + simply compare it against an empty string, like `$VAR == ''` or non-empty + string `$VARIABLE != ""`. 1. Comparing two variables - > Example: `$VARIABLE_1 == $VARIABLE_2` + Examples: - > Example: `$VARIABLE_1 != $VARIABLE_2` (introduced in GitLab 11.11) + - `$VARIABLE_1 == $VARIABLE_2` + - `$VARIABLE_1 != $VARIABLE_2` (introduced in GitLab 11.11) - It is possible to compare two variables. This is going to compare values - of these variables. + It is possible to compare two variables. This is going to compare values + of these variables. 1. Variable presence check - > Example: `$STAGING` + Example: `$STAGING` - If you only want to create a job when there is some variable present, - which means that it is defined and non-empty, you can simply use - variable name as an expression, like `$STAGING`. If `$STAGING` variable - is defined, and is non empty, expression will evaluate to truth. - `$STAGING` value needs to a string, with length higher than zero. - Variable that contains only whitespace characters is not an empty variable. + If you only want to create a job when there is some variable present, + which means that it is defined and non-empty, you can simply use + variable name as an expression, like `$STAGING`. If `$STAGING` variable + is defined, and is non empty, expression will evaluate to truth. + `$STAGING` value needs to a string, with length higher than zero. + Variable that contains only whitespace characters is not an empty variable. 1. Pattern matching (introduced in GitLab 11.0) - > Example: `$VARIABLE =~ /^content.*/` + Examples: - > Example: `$VARIABLE_1 !~ /^content.*/` (introduced in GitLab 11.11) + - `$VARIABLE =~ /^content.*/` + - `$VARIABLE_1 !~ /^content.*/` (introduced in GitLab 11.11) - It is possible perform pattern matching against a variable and regular - expression. Expression like this evaluates to truth if matches are found - when using `=~`. It evaluates to truth if matches are not found when `!~` is used. + It is possible perform pattern matching against a variable and regular + expression. Expression like this evaluates to truth if matches are found + when using `=~`. It evaluates to truth if matches are not found when `!~` is used. - Pattern matching is case-sensitive by default. Use `i` flag modifier, like - `/pattern/i` to make a pattern case-insensitive. + Pattern matching is case-sensitive by default. Use `i` flag modifier, like + `/pattern/i` to make a pattern case-insensitive. 1. Conjunction / Disjunction ([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27925) in GitLab 12.0) - > Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"` - - > Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 =~ /thing$/ && $VARIABLE3` + Examples: - > Example: `$VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/ && $VARIABLE3` + - `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"` + - `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 =~ /thing$/ && $VARIABLE3` + - `$VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/ && $VARIABLE3` - It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise - supported syntax may be used in a conjunctive or disjunctive statement. - Precedence of operators follows standard Ruby 2.5 operation - [precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html). + It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise + supported syntax may be used in a conjunctive or disjunctive statement. + Precedence of operators follows standard Ruby 2.5 operation + [precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html). ## Debug tracing @@ -644,6 +656,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ CI_PROJECT_DIR=/builds/gitlab-examples/ci-debug-trace ++ export CI_SERVER=yes ++ CI_SERVER=yes +++ export 'CI_SERVER_HOST=example.com' +++ CI_SERVER_HOST='example.com' ++ export 'CI_SERVER_NAME=GitLab CI' ++ CI_SERVER_NAME='GitLab CI' ++ export CI_SERVER_VERSION= @@ -678,6 +692,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ CI_JOB_NAME=debug_trace ++ export CI_JOB_STAGE=test ++ CI_JOB_STAGE=test +++ export CI_SERVER_HOST=example.com +++ CI_SERVER_HOST=example.com ++ export CI_SERVER_NAME=GitLab ++ CI_SERVER_NAME=GitLab ++ export CI_SERVER_VERSION=8.14.3-ee diff --git a/doc/ci/variables/img/custom_variables_output.png b/doc/ci/variables/img/custom_variables_output.png Binary files differindex 29f5c63b3d9..797e9ec07b9 100644 --- a/doc/ci/variables/img/custom_variables_output.png +++ b/doc/ci/variables/img/custom_variables_output.png diff --git a/doc/ci/variables/img/new_custom_variables_example.png b/doc/ci/variables/img/new_custom_variables_example.png Binary files differindex efe104efe4c..bb60e6bab21 100644 --- a/doc/ci/variables/img/new_custom_variables_example.png +++ b/doc/ci/variables/img/new_custom_variables_example.png diff --git a/doc/ci/variables/img/override_variable_manual_pipeline.png b/doc/ci/variables/img/override_variable_manual_pipeline.png Binary files differindex 3c8c59720cf..c77c5cb7764 100644 --- a/doc/ci/variables/img/override_variable_manual_pipeline.png +++ b/doc/ci/variables/img/override_variable_manual_pipeline.png diff --git a/doc/ci/variables/img/variable_types_usage_example.png b/doc/ci/variables/img/variable_types_usage_example.png Binary files differindex 0e8bde891fe..c2ae32fd048 100644 --- a/doc/ci/variables/img/variable_types_usage_example.png +++ b/doc/ci/variables/img/variable_types_usage_example.png diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index e911e97d3c8..409f7d62538 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -30,7 +30,7 @@ future GitLab releases.** | `CI_BUILDS_DIR` | all | 11.10 | Top-level directory where builds are executed. | | `CI_CONCURRENT_ID` | all | 11.10 | Unique ID of build execution within a single executor. | | `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | Unique ID of build execution within a single executor and project. | -| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch before a push request. Only populated when there is a merge request associated with the pipeline. | +| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch before a merge request. Only populated when there is a merge request associated with the pipeline. | | `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. | | `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | | `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built | @@ -101,6 +101,7 @@ future GitLab releases.** | `CI_RUNNER_TAGS` | 8.10 | 0.5 | The defined runner tags | | `CI_RUNNER_VERSION` | all | 10.6 | GitLab Runner version that is executing the current job | | `CI_SERVER` | all | all | Mark that job is executed in CI environment | +| `CI_SERVER_HOST` | 12.1 | all | Host component of the GitLab instance URL, without protocol and port (like gitlab.example.com) | | `CI_SERVER_NAME` | all | all | The name of CI server that is used to coordinate jobs | | `CI_SERVER_REVISION` | all | all | GitLab revision that is used to schedule jobs | | `CI_SERVER_VERSION` | all | all | GitLab version that is used to schedule jobs | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 474db05de06..1368764bcf8 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -100,6 +100,7 @@ The following table lists available parameters for jobs: | [`stage`](#stage) | Defines a job stage (default: `test`). | | [`only`](#onlyexcept-basic) | Limit when jobs are created. Also available: [`only:refs`, `only:kubernetes`, `only:variables`, and `only:changes`](#onlyexcept-advanced). | | [`except`](#onlyexcept-basic) | Limit when jobs are not created. Also available: [`except:refs`, `except:kubernetes`, `except:variables`, and `except:changes`](#onlyexcept-advanced). | +| [`rules`](#rules) | List of coniditions to evaluate and determine selected attributes of a build and whether or not it is created. May not be used alongside `only`/`except`. | [`tags`](#tags) | List of tags which are used to select Runner. | | [`allow_failure`](#allow_failure) | Allow job to fail. Failed job doesn't contribute to commit status. | | [`when`](#when) | When to run job. Also available: `when:manual` and `when:delayed`. | @@ -119,6 +120,35 @@ The following table lists available parameters for jobs: NOTE: **Note:** Parameters `types` and `type` are [deprecated](#deprecated-parameters). +## Setting default parameters + +Some parameters can be set globally as the default for all jobs using the +`default:` keyword. Default parameters can then be overridden by job-specific +configuration. + +The following job parameters can be defined inside a `default:` block: + +- [`image`](#image) +- [`services`](#services) +- [`before_script`](#before_script-and-after_script) +- [`after_script`](#before_script-and-after_script) +- [`cache`](#cache) + +In the following example, the `ruby:2.5` image is set as the default for all +jobs except the `rspec 2.6` job, which uses the `ruby:2.6` image: + +```yaml +default: + image: ruby:2.5 + +rspec: + script: bundle exec rspec + +rspec 2.6: + image: ruby:2.6 + script: bundle exec rspec +``` + ## Parameter details The following are detailed explanations for parameters used to configure CI/CD pipelines. @@ -239,8 +269,9 @@ It's possible to overwrite the globally defined `before_script` and `after_scrip if you set it per-job: ```yaml -before_script: - - global before script +default: + before_script: + - global before script job: before_script: @@ -356,7 +387,7 @@ In addition, `only` and `except` allow the use of special keywords: | `triggers` | For pipelines created using a trigger token. | | `web` | For pipelines created using **Run pipeline** button in GitLab UI (under your project's **Pipelines**). | | `merge_requests` | When a merge request is created or updated (See [pipelines for merge requests](../merge_request_pipelines/index.md)). | -| `chats` | For jobs created using a [GitLab ChatOps](../chatops/README.md) command. | +| `chat` | For jobs created using a [GitLab ChatOps](../chatops/README.md) command. | In the example below, `job` will run only for refs that start with `issue-`, whereas all branches will be skipped: @@ -475,7 +506,7 @@ Feature.enable(:allow_unsafe_ruby_regexp) ### `only`/`except` (advanced) CAUTION: **Warning:** -This an _alpha_ feature, and it is subject to change at any time without +This is an _alpha_ feature, and it is subject to change at any time without prior notice! GitLab supports both simple and complex strategies, so it's possible to use an @@ -488,10 +519,24 @@ Four keys are available: - `changes` - `kubernetes` -If you use multiple keys under `only` or `except`, they act as an AND. The logic is: +If you use multiple keys under `only` or `except`, the keys will be evaluated as a +single conjoined expression. That is: + +- `only:` means "include this job if all of the conditions match". +- `except:` means "exclude this job if any of the conditions match". + +With `only`, individual keys are logically joined by an AND: > (any of refs) AND (any of variables) AND (any of changes) AND (if kubernetes is active) +`except` is implemented as a negation of this complete expression: + +> NOT((any of refs) AND (any of variables) AND (any of changes) AND (if kubernetes is active)) + +This, more intuitively, means the keys join by an OR. A functionally equivalent expression: + +> (any of refs) OR (any of variables) OR (any of changes) OR (if kubernetes is active) + #### `only:refs`/`except:refs` > `refs` policy introduced in GitLab 10.0. @@ -646,6 +691,125 @@ In the scenario above, if a merge request is created or updated that changes either files in `service-one` directory or the `Dockerfile`, GitLab creates and triggers the `docker build service one` job. +### `rules` + +Using `rules` allows for a list of individual rule objects to be evaluated +*in order*, until one matches and dynamically provides attributes to the job. + +Available rule clauses include: + +- `if` (similar to [`only:variables`](#onlyvariablesexceptvariables)). +- `changes` (same as [`only:changes`](#onlychangesexceptchanges)). + +For example, using `if`: + +```yaml +job: + script: "echo Hello, Rules!" + rules: + - if: '$CI_MERGE_REQUEST_TARGET_BRANCH == "master"' # This rule will be evaluated + when: always + - if: '$VAR =~ /pattern/' # This rule will only be evaluated if the first does not match + when: manual + - when: on_success # A Rule entry with no conditional clauses evaluates to true. If neither of the first two Rules match, this one will and set job:when to "on_success" +``` + +If the first rule does not match, further rules will be evaluated sequentially +until a match is found. The above configuration will specify that `job` should +be built and run for every pipeline on merge requests targeting `master`, +regardless of the status of other builds. + +#### `rules:if` + +`rules:if` differs slightly from `only:variables` by accepting only a single +expression string, rather than an array of them. Any set of expressions to be +evaluated should be conjoined into a single expression using `&&` or `||`. For example: + +```yaml +job: + script: "echo Hello, Rules!" + rules: + - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH == "master"' # This rule will be evaluated + when: always + - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH =~ /^feature/' # This rule will only be evaluated if the target branch is not "master" + when: manual + - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH' # If neither of the first two match but the simple presence does, we set to "on_success" by default +``` + +If none of the provided rules match, the job will be set to `when:never`, and +not included in the pipeline. If `rules:when` is not included in the configuration +at all, the behavior defaults to `job:when`, which continues to default to +`on_success`. + +#### `rules:changes` + +`changes` works exactly the same way as [`only`/`except`](#onlychangesexceptchanges), +accepting an array of paths. The following configuration configures a job to be +run manually if `Dockerfile` has changed OR `$VAR == "string value"`. Otherwise +it is set to `when:on_success` by the last rule, where 0 clauses evaluate as +vacuously true. + +```yaml +docker build: + script: docker build -t my-image:$CI_COMMIT_REF_SLUG . + rules: + - changes: # Will include the job and set to when:manual if any of the follow paths match a modified file. + - Dockerfile + when: manual + - if: '$VAR == "string value"' + when: manual # Will include the job and set to when:manual if the expression evaluates to true, after the `changes:` rule fails to match. + - when: on_success # If neither of the first rules match, set to on_success + +``` + +#### Complex Rule Clauses + +To conjoin `if` and `changes` clauses with an AND, use them in the same rule. +Here we run the job manually if `Dockerfile` or any file in `docker/scripts/` +has changed AND `$VAR == "string value"`. Otherwise, the job will not be +included in the pipeline. + +```yaml +docker build: + script: docker build -t my-image:$CI_COMMIT_REF_SLUG . + rules: + - if: '$VAR == "string value"' + changes: # Will include the job and set to when:manual if any of the follow paths match a modified file. + - Dockerfile + - docker/scripts/* + when: manual + # - when: never would be redundant here, this is implied any time rules are listed. +``` + +The only clauses currently available are `if` and `changes`. Keywords such as +`branches` or `refs` that are currently available for `only`/`except` are not +yet available in `rules` as they are being individually considered for their +usage and behavior in the newer context. + +#### Permitted attributes + +The only job attributes currently set by `rules` are `when` and `start_in`, if +`when` is set to `delayed`. A job will be included in a pipeline if `when` is +evaluated to any value except `never`. + +Delayed jobs require a `start_in` value, so rule objects do as well. For example: + +```yaml +docker build: + script: docker build -t my-image:$CI_COMMIT_REF_SLUG . + rules: + - changes: # Will include the job and delay 3 hours when the Dockerfile has changed + - Dockerfile + when: delayed + start_in: '3 hours' + - when: on_success # Otherwise include the job and set to run normally + +``` + +Additional Job configuration may be added to rules in the future, if something +useful isn't available, please open an issue on +[Gitlab CE](https://www.gitlab.com/gitlab-org/gitlab-ce/issues). + ### `tags` `tags` is used to select specific Runners from the list of all Runners that are @@ -974,7 +1138,7 @@ review_app: stop_review_app: stage: deploy variables: - GIT_STRATEGY: none + GIT_STRATEGY: none script: make delete-app when: manual environment: @@ -1538,7 +1702,7 @@ dashboards. The `license_management` report collects [Licenses](../../user/project/merge_requests/license_management.md) as artifacts. -The collected License Management report will be uploaded to GitLab as an artifact and will +The collected License Compliance report will be uploaded to GitLab as an artifact and will be automatically shown in merge requests, pipeline view and provide data for security dashboards. @@ -1635,6 +1799,84 @@ You can ask your administrator to [flip this switch](../../administration/job_artifacts.md#validation-for-dependencies) and bring back the old behavior. +### `needs` + +> Introduced in GitLab 12.2. + +The `needs:` keyword enables executing jobs out-of-order, allowing you to implement +a [directed acyclic graph](../directed_acyclic_graph/index.md) in your `.gitlab-ci.yml`. + +This lets you run some jobs without waiting for other ones, disregarding stage ordering +so you can have multiple stages running concurrently. + +Let's consider the following example: + +```yaml +linux:build: + stage: build + +mac:build: + stage: build + +linux:rspec: + stage: test + needs: ["linux:build"] + +linux:rubocop: + stage: test + needs: ["linux:build"] + +mac:rspec: + stage: test + needs: ["mac:build"] + +mac:rubocop: + stage: test + needs: ["mac:build"] + +production: + stage: deploy +``` + +This example creates three paths of execution: + +- Linux path: the `linux:rspec` and `linux:rubocop` jobs will be run as soon + as the `linux:build` job finishes without waiting for `mac:build` to finish. + +- macOS path: the `mac:rspec` and `mac:rubocop` jobs will be run as soon + as the `mac:build` job finishes, without waiting for `linux:build` to finish. + +- The `production` job will be executed as soon as all previous jobs + finish; in this case: `linux:build`, `linux:rspec`, `linux:rubocop`, + `mac:build`, `mac:rspec`, `mac:rubocop`. + +#### Requirements and limitations + +1. If `needs:` is set to point to a job that is not instantiated + because of `only/except` rules or otherwise does not exist, the + job will fail. +1. Note that on day one of the launch, we are temporarily limiting the + maximum number of jobs that a single job can need in the `needs:` array. Track + our [infrastructure issue](https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7541) + for details on the current limit. +1. If you use `dependencies:` with `needs:`, it's important that you + do not mark a job as having a dependency on something that won't + have been run at the time it needs it. It's better to use both + keywords in this case so that GitLab handles the ordering appropriately. +1. It is impossible for now to have `needs: []` (empty needs), + the job always needs to depend on something, unless this is the job + in the first stage (see [gitlab-ce#65504](https://gitlab.com/gitlab-org/gitlab-ce/issues/65504)). +1. If `needs:` refers to a job that is marked as `parallel:`. + the current job will depend on all parallel jobs created. +1. `needs:` is similar to `dependencies:` in that it needs to use jobs from + prior stages, meaning it is impossible to create circular + dependencies or depend on jobs in the current stage (see [gitlab-ce#65505](https://gitlab.com/gitlab-org/gitlab-ce/issues/65505)). +1. Related to the above, stages must be explicitly defined for all jobs + that have the keyword `needs:` or are referred to by one. +1. For self-managed users, the feature must be turned on using the `ci_dag_support` + feature flag. The `ci_dag_limit_needs` option, if set, will limit the number of + jobs that a single job can need to `50`. If unset, the limit is `5`. + ### `coverage` > [Introduced][ce-7447] in GitLab 8.17. @@ -1741,14 +1983,41 @@ sequentially from `job_name 1/N` to `job_name N/N`. For every job, `CI_NODE_INDEX` and `CI_NODE_TOTAL` [environment variables](../variables/README.md#predefined-environment-variables) are set. -A simple example: +Marking a job to be run in parallel requires only a simple addition to your configuration file: + +```diff + test: + script: rspec ++ parallel: 5 +``` +TIP: **Tip:** +Parallelize tests suites across parallel jobs. +Different languages have different tools to facilitate this. + +A simple example using [Sempahore Test Boosters](https://github.com/renderedtext/test-boosters) and RSpec to run some Ruby tests: + +```ruby +# Gemfile +source 'https://rubygems.org' + +gem 'rspec' +gem 'semaphore_test_boosters' +``` ```yaml test: - script: rspec - parallel: 5 + parallel: 3 + script: + - bundle + - bundle exec rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL ``` +CAUTION: **Caution:** +Please be aware that semaphore_test_boosters reports usages statistics to the author. + +You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec +job split into three separate jobs. + ### `trigger` **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. @@ -2221,10 +2490,10 @@ spinach: script: rake spinach ``` -It's also possible to use multiple parents for `extends`. -The algorithm used for merge is "closest scope wins", so keys -from the last member will always shadow anything defined on other levels. -For example: +In GitLab 12.0 and later, it's also possible to use multiple parents for +`extends`. The algorithm used for merge is "closest scope wins", so +keys from the last member will always shadow anything defined on other +levels. For example: ```yaml .only-important: @@ -2411,20 +2680,20 @@ There are three possible values: `none`, `normal`, and `recursive`: - `normal` means that only the top-level submodules will be included. It is equivalent to: - ``` - git submodule sync - git submodule update --init - ``` + ``` + git submodule sync + git submodule update --init + ``` - `recursive` means that all submodules (including submodules of submodules) will be included. This feature needs Git v1.8.1 and later. When using a GitLab Runner with an executor not based on Docker, make sure the Git version meets that requirement. It is equivalent to: - ``` - git submodule sync --recursive - git submodule update --init --recursive - ``` + ``` + git submodule sync --recursive + git submodule update --init --recursive + ``` Note that for this feature to work correctly, the submodules must be configured (in `.gitmodules`) with either: @@ -2550,18 +2819,39 @@ You can set it globally or per-job in the [`variables`](#variables) section. The following parameters are deprecated. -### `types` +### Globally-defined `types` CAUTION: **Deprecated:** `types` is deprecated, and could be removed in a future release. Use [`stages`](#stages) instead. -### `type` +### Job-defined `type` CAUTION: **Deprecated:** `type` is deprecated, and could be removed in one of the future releases. Use [`stage`](#stage) instead. +### Globally-defined `image`, `services`, `cache`, `before_script`, `after_script` + +Defining `image`, `services`, `cache`, `before_script`, and +`after_script` globally is deprecated. Support could be removed +from a future release. + +Use [`default:`](#setting-default-parameters) instead. For example: + +```yaml +default: + image: ruby:2.5 + services: + - docker:dind + cache: + paths: [vendor/] + before_script: + - bundle install --path vendor/ + after_script: + - rm -rf tmp/ +``` + ## Custom build directories > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1267) in Gitlab Runner 11.10 @@ -2644,7 +2934,7 @@ variables: The value of `GIT_CLONE_PATH` is expanded once into `$CI_BUILDS_DIR/go/src/namespace/project`, and results in failure -because `$CI_BUILDS_DIR` is not expanded. +because `$CI_BUILDS_DIR` is not expanded. ## Special YAML features @@ -2829,7 +3119,8 @@ Alternatively, one can pass the `ci.skip` [Git push option][push-option] if using Git 2.10 or newer: ```sh -git push -o ci.skip +git push --push-option=ci.skip # using git 2.10+ +git push -o ci.skip # using git 2.18+ ``` <!-- ## Troubleshooting |
