diff options
Diffstat (limited to 'doc/ci/yaml/README.md')
| -rw-r--r-- | doc/ci/yaml/README.md | 743 |
1 files changed, 558 insertions, 185 deletions
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index accf6340398..e38628b288b 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -10,6 +10,11 @@ of your repository and contains definitions of how your project should be built. If you want a quick introduction to GitLab CI, follow our [quick start guide](../quick_start/README.md). +NOTE: **Note:** +If you have a [mirrored repository where GitLab pulls from](https://docs.gitlab.com/ee/workflow/repository_mirroring.html#pulling-from-a-remote-repository), +you may need to enable pipeline triggering in your project's +**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. + ## Jobs The YAML file defines a set of jobs with constraints stating when they should @@ -51,6 +56,7 @@ A job is defined by a list of parameters that define the job behavior. | Keyword | Required | Description | |---------------|----------|-------------| | script | yes | Defines a shell script which is executed by Runner | +| extends | no | Defines a configuration entry that this job is going to inherit from | | image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | | services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | | stage | no | Defines a job stage (default: `test`) | @@ -70,6 +76,81 @@ A job is defined by a list of parameters that define the job behavior. | coverage | no | Define code coverage settings for a given job | | retry | no | Define how many times a job can be auto-retried in case of a failure | +### `extends` + +> Introduced in GitLab 11.3. + +`extends` defines an entry name that a job that uses `extends` is going to +inherit from. + +It is an alternative to using [YAML anchors](#anchors) and is a little +more flexible and readable: + +```yaml +.tests: + script: rake test + stage: test + only: + refs: + - branches + +rspec: + extends: .tests + script: rake rspec + only: + variables: + - $RSPEC +``` + +In the example above, the `rspec` job is going to inherit from the `.tests` +template job. GitLab will perform a reverse deep merge, which means that it will +merge the `rspec` contents into `.tests` recursively, and this is going to result in +the following `rspec` job: + +```yaml +rspec: + script: rake rspec + stage: test + only: + refs: + - branches + variables: + - $RSPEC +``` + +`.tests` in this example is a [hidden key](#hidden-keys-jobs), but it's +possible to inherit from regular jobs as well. + +`extends` supports multi-level inheritance, however it is not recommended to +use more than three levels. The maximum nesting level that is supported is 10. +The following example has two levels of inheritance: + +```yaml +.tests: + only: + - pushes + +.rspec: + extends: .tests + script: rake rspec + +rspec 1: + variables: + RSPEC_SUITE: '1' + extends: .rspec + +rspec 2: + variables: + RSPEC_SUITE: '2' + extends: .rspec + +spinach: + extends: .tests + script: rake spinach +``` + +`extends` works across configuration files combined with [`include`](#include). + ### `pages` `pages` is a special job that is used to upload static content to GitLab that @@ -83,18 +164,18 @@ The example below simply moves all files from the root of the project to the `public/` directory. The `.public` workaround is so `cp` doesn't also copy `public/` to itself in an infinite loop: -``` +```yaml pages: stage: deploy script: - - mkdir .public - - cp -r * .public - - mv .public public + - mkdir .public + - cp -r * .public + - mv .public public artifacts: paths: - - public + - public only: - - master + - master ``` Read more on [GitLab Pages user documentation](../../user/project/pages/index.md). @@ -107,7 +188,7 @@ used for time of the job. The configuration of this feature is covered in ## `before_script` and `after_script` -> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 +> Introduced in GitLab 8.7 and requires GitLab Runner v1.2 `before_script` is used to define the command that should be run before all jobs, including deploy jobs, but after the restoration of [artifacts](#artifacts). @@ -126,15 +207,15 @@ if you set it per-job: ```yaml before_script: -- global before script + - global before script job: before_script: - - execute this instead of global before script + - execute this instead of global before script script: - - my command + - my command after_script: - - execute this after my script + - execute this after my script ``` ## `stages` @@ -303,7 +384,9 @@ except master. ## `only` and `except` (complex) -> Introduced in GitLab 10.0 +> `refs` and `kubernetes` policies introduced in GitLab 10.0 +> +> `variables` policy introduced in 10.7 CAUTION: **Warning:** This an _alpha_ feature, and it it subject to change at any time without @@ -315,9 +398,14 @@ policy configuration. GitLab now supports both, simple and complex strategies, so it is possible to use an array and a hash configuration scheme. -Two keys are now available: `refs` and `kubernetes`. Refs strategy equals to -simplified only/except configuration, whereas kubernetes strategy accepts only -`active` keyword. +Three keys are now available: `refs`, `kubernetes` and `variables`. +Refs strategy equals to simplified only/except configuration, whereas +kubernetes strategy accepts only `active` keyword. + +`variables` keyword is used to define variables expressions. In other words +you can use predefined variables / project / group or +environment-scoped variables to define an expression GitLab is going to +evaluate in order to decide whether a job should be created or not. See the example below. Job is going to be created only when pipeline has been scheduled or runs for a `master` branch, and only if kubernetes service is @@ -332,6 +420,31 @@ job: kubernetes: active ``` +Examples of using variables expressions: + +```yaml +deploy: + script: cap staging deploy + only: + refs: + - branches + variables: + - $RELEASE == "staging" + - $STAGING +``` + +Another use case is exluding jobs depending on a commit message _(added in 11.0)_: + +```yaml +end-to-end: + script: rake test:end-to-end + except: + variables: + - $CI_COMMIT_MESSAGE =~ /skip-end-to-end-tests/ +``` + +Learn more about variables expressions on [a separate page][variables-expressions]. + ## `tags` `tags` is used to select specific Runners from the list of all Runners that are @@ -353,10 +466,33 @@ job: The specification above, will make sure that `job` is built by a Runner that has both `ruby` AND `postgres` tags defined. +Tags are also a great way to run different jobs on different platforms, for +example, given an OS X Runner with tag `osx` and Windows Runner with tag +`windows`, the following jobs run on respective platforms: + +```yaml +windows job: + stage: + - build + tags: + - windows + script: + - echo Hello, %USERNAME%! + +osx job: + stage: + - build + tags: + - osx + script: + - echo "Hello, $USER!" +``` + ## `allow_failure` `allow_failure` is used when you want to allow a job to fail without impacting the rest of the CI suite. Failed jobs don't contribute to the commit status. +The default value is `false`. When enabled and the job fails, the pipeline will be successful/green for all intents and purposes, but a "CI build passed with warnings" message will be @@ -372,18 +508,18 @@ fails, it will not stop the next stage from running, since it's marked with job1: stage: test script: - - execute_script_that_will_fail + - execute_script_that_will_fail allow_failure: true job2: stage: test script: - - execute_script_that_will_succeed + - execute_script_that_will_succeed job3: stage: deploy script: - - deploy_to_staging + - deploy_to_staging ``` ## `when` @@ -405,38 +541,38 @@ For example: ```yaml stages: -- build -- cleanup_build -- test -- deploy -- cleanup + - build + - cleanup_build + - test + - deploy + - cleanup build_job: stage: build script: - - make build + - make build cleanup_build_job: stage: cleanup_build script: - - cleanup build when failed + - cleanup build when failed when: on_failure test_job: stage: test script: - - make test + - make test deploy_job: stage: deploy script: - - make deploy + - make deploy when: manual cleanup_job: stage: cleanup script: - - cleanup after jobs + - cleanup after jobs when: always ``` @@ -450,9 +586,10 @@ The above script will: ### `when:manual` > **Notes:** -- Introduced in GitLab 8.10. -- Blocking manual actions were introduced in GitLab 9.0. -- Protected actions were introduced in GitLab 9.2. +> +> - Introduced in GitLab 8.10. +> - Blocking manual actions were introduced in GitLab 9.0. +> - Protected actions were introduced in GitLab 9.2. Manual actions are a special type of job that are not executed automatically, they need to be explicitly started by a user. An example usage of manual actions @@ -483,11 +620,11 @@ have ability to merge to this branch. ## `environment` +> **Notes:** > -**Notes:** -- Introduced in GitLab 8.9. -- You can read more about environments and find more examples in the - [documentation about environments][environment]. +> - Introduced in GitLab 8.9. +> - You can read more about environments and find more examples in the +> [documentation about environments][environment]. `environment` is used to define that a job deploys to a specific environment. If `environment` is specified and no environment under that name exists, a new @@ -508,15 +645,15 @@ deployment to the `production` environment. ### `environment:name` +> **Notes:** > -**Notes:** -- Introduced in GitLab 8.11. -- Before GitLab 8.11, the name of an environment could be defined as a string like - `environment: production`. The recommended way now is to define it under the - `name` keyword. -- The `name` parameter can use any of the defined CI variables, - including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). - You however cannot use variables defined under `script`. +> - Introduced in GitLab 8.11. +> - Before GitLab 8.11, the name of an environment could be defined as a string like +> `environment: production`. The recommended way now is to define it under the +> `name` keyword. +> - The `name` parameter can use any of the defined CI variables, +> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). +> You however cannot use variables defined under `script`. The `environment` name can contain: @@ -547,14 +684,14 @@ deploy to production: ### `environment:url` +> **Notes:** > -**Notes:** -- Introduced in GitLab 8.11. -- Before GitLab 8.11, the URL could be added only in GitLab's UI. The - recommended way now is to define it in `.gitlab-ci.yml`. -- The `url` parameter can use any of the defined CI variables, - including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). - You however cannot use variables defined under `script`. +> - Introduced in GitLab 8.11. +> - Before GitLab 8.11, the URL could be added only in GitLab's UI. The +> recommended way now is to define it in `.gitlab-ci.yml`. +> - The `url` parameter can use any of the defined CI variables, +> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). +> You however cannot use variables defined under `script`. This is an optional value that when set, it exposes buttons in various places in GitLab which when clicked take you to the defined URL. @@ -574,12 +711,12 @@ deploy to production: ### `environment:on_stop` +> **Notes:** > -**Notes:** -- [Introduced][ce-6669] in GitLab 8.13. -- Starting with GitLab 8.14, when you have an environment that has a stop action - defined, GitLab will automatically trigger a stop action when the associated - branch is deleted. +> - [Introduced][ce-6669] in GitLab 8.13. +> - Starting with GitLab 8.14, when you have an environment that has a stop action +> defined, GitLab will automatically trigger a stop action when the associated +> branch is deleted. Closing (stoping) environments can be achieved with the `on_stop` keyword defined under `environment`. It declares a different job that runs in order to close @@ -630,13 +767,13 @@ The `stop_review_app` job is **required** to have the following keywords defined ### Dynamic environments +> **Notes:** > -**Notes:** -- [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. -- The `$CI_ENVIRONMENT_SLUG` was [introduced][ce-7983] in GitLab 8.15. -- The `name` and `url` parameters can use any of the defined CI variables, - including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). - You however cannot use variables defined under `script`. +> - [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. +> - The `$CI_ENVIRONMENT_SLUG` was [introduced][ce-7983] in GitLab 8.15. +> - The `name` and `url` parameters can use any of the defined CI variables, +> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). +> You however cannot use variables defined under `script`. For example: @@ -666,13 +803,17 @@ as Review Apps. You can see a simple example using Review Apps at ## `cache` +> **Notes:** > -**Notes:** -- Introduced in GitLab Runner v0.7.0. -- `cache` can be set globally and per-job. -- From GitLab 9.0, caching is enabled and shared between pipelines and jobs - by default. -- From GitLab 9.2, caches are restored before [artifacts](#artifacts). +> - Introduced in GitLab Runner v0.7.0. +> - `cache` can be set globally and per-job. +> - From GitLab 9.0, caching is enabled and shared between pipelines and jobs +> by default. +> - From GitLab 9.2, caches are restored before [artifacts](#artifacts). + +TIP: **Learn more:** +Read how caching works and find out some good practices in the +[caching dependencies documentation](../caching/index.md). `cache` is used to specify a list of files and directories which should be cached between jobs. You can only use paths that are within the project @@ -681,35 +822,20 @@ workspace. If `cache` is defined outside the scope of jobs, it means it is set globally and all jobs will use that definition. -Cache all files in `binaries` and `.config`: +### `cache:paths` -```yaml -rspec: - script: test - cache: - paths: - - binaries/ - - .config -``` +Use the `paths` directive to choose which files or directories will be cached. +Wildcards can be used as well. -Cache all Git untracked files: +Cache all files in `binaries` that end in `.apk` and the `.config` file: ```yaml rspec: script: test cache: - untracked: true -``` - -Cache all Git untracked files and files in `binaries`: - -```yaml -rspec: - script: test - cache: - untracked: true paths: - - binaries/ + - binaries/*.apk + - .config ``` Locally defined cache overrides globally defined options. The following `rspec` @@ -718,95 +844,82 @@ job will cache only `binaries/`: ```yaml cache: paths: - - my/files + - my/files rspec: script: test cache: key: rspec paths: - - binaries/ + - binaries/ ``` Note that since cache is shared between jobs, if you're using different paths for different jobs, you should also set a different **cache:key** otherwise cache content can be overwritten. -NOTE: **Note:** -The cache is provided on a best-effort basis, so don't expect that the cache -will be always present. - ### `cache:key` > Introduced in GitLab Runner v1.0.0. -The `key` directive allows you to define the affinity of caching -between jobs, allowing to have a single cache for all jobs, -cache per-job, cache per-branch or any other way that fits your needs. +Since the cache is shared between jobs, if you're using different +paths for different jobs, you should also set a different `cache:key` +otherwise cache content can be overwritten. -This way, you can fine tune caching, allowing you to cache data between -different jobs or even different branches. +The `key` directive allows you to define the affinity of caching between jobs, +allowing to have a single cache for all jobs, cache per-job, cache per-branch +or any other way that fits your workflow. This way, you can fine tune caching, +allowing you to cache data between different jobs or even different branches. The `cache:key` variable can use any of the -[predefined variables](../variables/README.md), and the default key, if not set, -is set as `$CI_JOB_NAME-$CI_COMMIT_REF_NAME` which translates as "per-job and -per-branch". It is the default across the project, therefore everything is -shared between pipelines and jobs running on the same branch by default. +[predefined variables](../variables/README.md), and the default key, if not +set, is just literal `default` which means everything is shared between each +pipelines and jobs by default, starting from GitLab 9.0. NOTE: **Note:** The `cache:key` variable cannot contain the `/` character, or the equivalent URI-encoded `%2F`; a value made only of dots (`.`, `%2E`) is also forbidden. -**Example configurations** - -To enable per-job caching: - -```yaml -cache: - key: "$CI_JOB_NAME" - untracked: true -``` - -To enable per-branch caching: +For example, to enable per-branch caching: ```yaml cache: key: "$CI_COMMIT_REF_SLUG" - untracked: true + paths: + - binaries/ ``` -To enable per-job and per-branch caching: +If you use **Windows Batch** to run your shell scripts you need to replace +`$` with `%`: ```yaml cache: - key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" - untracked: true + key: "%CI_COMMIT_REF_SLUG%" + paths: + - binaries/ ``` -To enable per-branch and per-stage caching: - -```yaml -cache: - key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" - untracked: true -``` +### `cache:untracked` -If you use **Windows Batch** to run your shell scripts you need to replace -`$` with `%`: +Set `untracked: true` to cache all files that are untracked in your Git +repository: ```yaml -cache: - key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%" - untracked: true +rspec: + script: test + cache: + untracked: true ``` -If you use **Windows PowerShell** to run your shell scripts you need to replace -`$` with `$env:`: +Cache all Git untracked files and files in `binaries`: ```yaml -cache: - key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG" - untracked: true +rspec: + script: test + cache: + untracked: true + paths: + - binaries/ ``` ### `cache:policy` @@ -858,42 +971,34 @@ skip the download step. ## `artifacts` +> **Notes:** > -**Notes:** -- Introduced in GitLab Runner v0.7.0 for non-Windows platforms. -- Windows support was added in GitLab Runner v.1.0.0. -- From GitLab 9.2, caches are restored before artifacts. -- Currently not all executors are supported. -- Job artifacts are only collected for successful jobs by default. +> - Introduced in GitLab Runner v0.7.0 for non-Windows platforms. +> - Windows support was added in GitLab Runner v.1.0.0. +> - From GitLab 9.2, caches are restored before artifacts. +> - Not all executors are [supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). +> - Job artifacts are only collected for successful jobs by default. `artifacts` is used to specify a list of files and directories which should be -attached to the job after success. You can only use paths that are within the -project workspace. To pass artifacts between different jobs, see [dependencies](#dependencies). -Below are some examples. +attached to the job after success. -Send all files in `binaries` and `.config`: +The artifacts will be sent to GitLab after the job finishes successfully and will +be available for download in the GitLab UI. -```yaml -artifacts: - paths: - - binaries/ - - .config -``` +[Read more about artifacts.](../../user/project/pipelines/job_artifacts.md) -Send all Git untracked files: +### `artifacts:paths` -```yaml -artifacts: - untracked: true -``` +You can only use paths that are within the project workspace. To pass artifacts +between different jobs, see [dependencies](#dependencies). -Send all Git untracked files and files in `binaries`: +Send all files in `binaries` and `.config`: ```yaml artifacts: - untracked: true paths: - - binaries/ + - binaries/ + - .config ``` To disable artifact passing, define the job with empty [dependencies](#dependencies): @@ -922,16 +1027,11 @@ release-job: - mvn package -U artifacts: paths: - - target/*.war + - target/*.war only: - tags ``` -The artifacts will be sent to GitLab after the job finishes successfully and will -be available for download in the GitLab UI. - -[Read more about artifacts.](../../user/project/pipelines/job_artifacts.md) - ### `artifacts:name` > Introduced in GitLab 8.6 and GitLab Runner v1.1.0. @@ -942,32 +1042,41 @@ useful when you'd like to download the archive from GitLab. The `artifacts:name` variable can make use of any of the [predefined variables](../variables/README.md). The default name is `artifacts`, which becomes `artifacts.zip` when downloaded. +NOTE: **Note:** +If your branch-name contains forward slashes +(e.g. `feature/my-feature`) it is advised to use `$CI_COMMIT_REF_SLUG` +instead of `$CI_COMMIT_REF_NAME` for proper naming of the artifact. + To create an archive with a name of the current job: ```yaml job: artifacts: name: "$CI_JOB_NAME" + paths: + - binaries/ ``` To create an archive with a name of the current branch or tag including only -the files that are untracked by Git: +the binaries directory: ```yaml job: - artifacts: - name: "$CI_COMMIT_REF_NAME" - untracked: true + artifacts: + name: "$CI_COMMIT_REF_NAME" + paths: + - binaries/ ``` To create an archive with a name of the current job and the current branch or -tag including only the files that are untracked by Git: +tag including only the binaries directory: ```yaml job: artifacts: name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME" - untracked: true + paths: + - binaries/ ``` To create an archive with a name of the current [stage](#stages) and branch name: @@ -976,7 +1085,8 @@ To create an archive with a name of the current [stage](#stages) and branch name job: artifacts: name: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" - untracked: true + paths: + - binaries/ ``` --- @@ -988,7 +1098,8 @@ If you use **Windows Batch** to run your shell scripts you need to replace job: artifacts: name: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%" - untracked: true + paths: + - binaries/ ``` If you use **Windows PowerShell** to run your shell scripts you need to replace @@ -998,7 +1109,33 @@ If you use **Windows PowerShell** to run your shell scripts you need to replace job: artifacts: name: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME" - untracked: true + paths: + - binaries/ +``` + +### `artifacts:untracked` + +`artifacts:untracked` is used to add all Git untracked files as artifacts (along +to the paths defined in `artifacts:paths`). + +NOTE: **Note:** +To exclude the folders/files which should not be a part of `untracked` just +add them to `.gitignore`. + +Send all Git untracked files: + +```yaml +artifacts: + untracked: true +``` + +Send all Git untracked files and files in `binaries`: + +```yaml +artifacts: + untracked: true + paths: + - binaries/ ``` ### `artifacts:when` @@ -1038,8 +1175,10 @@ keep artifacts forever. After their expiry, artifacts are deleted hourly by default (via a cron job), and are not accessible anymore. -The value of `expire_in` is an elapsed time. Examples of parsable values: +The value of `expire_in` is an elapsed time in seconds, unless a unit is +provided. Examples of parsable values: +- '42' - '3 mins 4 sec' - '2 hrs 20 min' - '2h20min' @@ -1055,6 +1194,55 @@ job: expire_in: 1 week ``` +### `artifacts:reports` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20390) in +GitLab 11.2. Requires GitLab Runner 11.2 and above. + +The `reports` keyword is used for collecting test reports from jobs and +exposing them in GitLab's UI (merge requests, pipeline views). Read how to use +this with [JUnit reports](#artifacts-reports-junit). + +NOTE: **Note:** +The test reports are collected regardless of the job results (success or failure). +You can use [`artifacts:expire_in`](#artifacts-expire_in) to set up an expiration +date for their artifacts. + +#### `artifacts:reports:junit` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20390) in +GitLab 11.2. Requires GitLab Runner 11.2 and above. + +The `junit` report collects [JUnit XML files](https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.1.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html) +as artifacts. Although JUnit was originally developed in Java, there are many +[third party ports](https://en.wikipedia.org/wiki/JUnit#Ports) for other +languages like Javascript, Python, Ruby, etc. + +Below is an example of collecting a JUnit XML file from Ruby's RSpec test tool: + +```yaml +rspec: + stage: test + script: + - bundle install + - rspec --format RspecJunitFormatter --out rspec.xml + artifacts: + reports: + junit: rspec.xml +``` + +The collected JUnit reports will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests. + +For more examples, see [JUnit test reports](../junit_test_reports.md). + +NOTE: **Note:** +In case the JUnit tool you use exports to multiple XML files, you can specify +multiple test report paths within a single job and they will be automatically +concatenated into a single file. Use a filename pattern (`junit: rspec-*.xml`), +an array of filenames (`junit: [rspec-1.xml, rspec-2.xml, rspec-3.xml]`), or a +combination thereof (`junit: [rspec.xml, test-results/TEST-*.xml]`). + ## `dependencies` > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. @@ -1088,26 +1276,26 @@ build:osx: script: make build:osx artifacts: paths: - - binaries/ + - binaries/ build:linux: stage: build script: make build:linux artifacts: paths: - - binaries/ + - binaries/ test:osx: stage: test script: make test:osx dependencies: - - build:osx + - build:osx test:linux: stage: test script: make test:linux dependencies: - - build:linux + - build:linux deploy: stage: deploy @@ -1150,7 +1338,7 @@ job1: ## `retry` -> [Introduced][ce-3442] in GitLab 9.5. +> [Introduced][ce-12909] in GitLab 9.5. `retry` allows you to configure how many times a job is going to be retried in case of a failure. @@ -1170,6 +1358,187 @@ test: retry: 2 ``` +## `include` + +> Introduced in [GitLab Edition Premium][ee] 10.5. +> Available for Starter, Premium and Ultimate [versions][gitlab-versions] since 10.6. +> Behaviour expanded in GitLab 10.8 to allow more flexible overriding. +> Available for Libre since [11.4](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21603) + +Using the `include` keyword, you can allow the inclusion of external YAML files. + +In the following example, the content of `.before-script-template.yml` will be +automatically fetched and evaluated along with the content of `.gitlab-ci.yml`: + +```yaml +# Content of https://gitlab.com/awesome-project/raw/master/.before-script-template.yml + +before_script: + - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs + - gem install bundler --no-ri --no-rdoc + - bundle install --jobs $(nproc) "${FLAGS[@]}" +``` + +```yaml +# Content of .gitlab-ci.yml + +include: 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml' + +rspec: + script: + - bundle exec rspec +``` + +You can define it either as a single string, or, in case you want to include +more than one files, an array of different values . The following examples +are both valid cases: + +```yaml +# Single string + +include: '/templates/.after-script-template.yml' +``` + +```yaml +# Array + +include: + - 'https://gitlab.com/awesome-project/raw/master/.before-script-template.yml' + - '/templates/.after-script-template.yml' +``` + +--- + +`include` supports two types of files: + +- **local** to the same repository, referenced by using full paths in the same + repository, with `/` being the root directory. For example: + + ```yaml + # Within the repository + include: '/templates/.gitlab-ci-template.yml' + ``` + + NOTE: **Note:** + You can only use files that are currently tracked by Git on the same branch + your configuration file is. In other words, when using a **local file**, make + sure that both `.gitlab-ci.yml` and the local file are on the same branch. + + NOTE: **Note:** + We don't support the inclusion of local files through Git submodules paths. + +- **remote** in a different location, accessed using HTTP/HTTPS, referenced + using the full URL. For example: + + ```yaml + include: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml' + ``` + + NOTE: **Note:** + The remote file must be publicly accessible through a simple GET request, as we don't support authentication schemas in the remote URL. + +--- + + +Since GitLab 10.8 we are now recursively merging the files defined in `include` +with those in `.gitlab-ci.yml`. Files defined by `include` are always +evaluated first and recursively merged with the content of `.gitlab-ci.yml`, no +matter the position of the `include` keyword. You can take advantage of +recursive merging to customize and override details in included CI +configurations with local definitions. + +The following example shows specific YAML-defined variables and details of the +`production` job from an include file being customized in `.gitlab-ci.yml`. + +```yaml +# Content of https://company.com/autodevops-template.yml + +variables: + POSTGRES_USER: user + POSTGRES_PASSWORD: testing_password + POSTGRES_DB: $CI_ENVIRONMENT_SLUG + +production: + stage: production + script: + - install_dependencies + - deploy + environment: + name: production + url: https://$CI_PROJECT_PATH_SLUG.$AUTO_DEVOPS_DOMAIN + only: + - master +``` + +```yaml +# Content of .gitlab-ci.yml + +include: 'https://company.com/autodevops-template.yml' + +image: alpine:latest + +variables: + POSTGRES_USER: root + POSTGRES_PASSWORD: secure_password + +stages: + - build + - test + - production + +production: + environment: + url: https://domain.com +``` + +In this case, the variables `POSTGRES_USER` and `POSTGRES_PASSWORD` along +with the environment url of the `production` job defined in +`autodevops-template.yml` have been overridden by new values defined in +`.gitlab-ci.yml`. + +NOTE: **Note:** +Recursive includes are not supported meaning your external files +should not use the `include` keyword, as it will be ignored. + +Recursive merging lets you extend and override dictionary mappings, but +you cannot add or modify items to an included array. For example, to add +an additional item to the production job script, you must repeat the +existing script items. + +```yaml +# Content of https://company.com/autodevops-template.yml + +production: + stage: production + script: + - install_dependencies + - deploy +``` + +```yaml +# Content of .gitlab-ci.yml + +include: 'https://company.com/autodevops-template.yml' + +stages: + - production + +production: + script: + - install_depedencies + - deploy + - notify_owner +``` + +In this case, if `install_dependencies` and `deploy` were not repeated in +`.gitlab-ci.yml`, they would not be part of the script for the `production` +job in the combined CI configuration. + +NOTE: **Note:** +We currently do not support using YAML aliases across different YAML files +sourced by `include`. You must only refer to aliases in the same file. Instead +of using YAML anchors you can use [`extends` keyword](#extends). + ## `variables` > Introduced in GitLab Runner v0.5.0. @@ -1207,7 +1576,7 @@ Runner itself](../variables/README.md#predefined-variables-environment-variables One example would be `CI_COMMIT_REF_NAME` which has the value of the branch or tag name for which project is built. Apart from the variables you can set in `.gitlab-ci.yml`, there are also the so called -[secret variables](../variables/README.md#secret-variables) +[Variables](../variables/README.md#variables) which can be set in GitLab's UI. [Learn more about variables and their priority.][variables] @@ -1274,7 +1643,9 @@ There are three possible values: `none`, `normal`, and `recursive`: ``` - `recursive` means that all submodules (including submodules of submodules) - will be included. It is equivalent to: + will be included. This feature needs Git v1.8.1 and later. When using a + GitLab Runner with an executor not based on Docker, make sure the Git version + meets that requirement. It is equivalent to: ``` git submodule sync --recursive @@ -1544,8 +1915,9 @@ capitalization, the commit will be created but the pipeline will be skipped. ## Validate the .gitlab-ci.yml -Each instance of GitLab CI has an embedded debug tool called Lint. -You can find the link under `/ci/lint` of your gitlab instance. +Each instance of GitLab CI has an embedded debug tool called Lint, which validates the +content of your `.gitlab-ci.yml` files. You can find the Lint under the page `ci/lint` of your +project namespace (e.g, `http://gitlab-example.com/gitlab-org/project-123/-/ci/lint`) ## Using reserved keywords @@ -1565,5 +1937,6 @@ CI with various languages. [variables]: ../variables/README.md [ce-7983]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7983 [ce-7447]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7447 -[ce-3442]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3442 +[ce-12909]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12909 [schedules]: ../../user/project/pipelines/schedules.md +[variables-expressions]: ../variables/README.md#variables-expressions |