diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/ci/img/parent_pipeline_graph_expanded_v12_6.png | bin | 0 -> 298902 bytes | |||
| -rw-r--r-- | doc/ci/parent_child_pipelines.md | 86 | ||||
| -rw-r--r-- | doc/ci/pipelines.md | 7 | ||||
| -rw-r--r-- | doc/ci/yaml/README.md | 31 |
4 files changed, 121 insertions, 3 deletions
diff --git a/doc/ci/img/parent_pipeline_graph_expanded_v12_6.png b/doc/ci/img/parent_pipeline_graph_expanded_v12_6.png Binary files differnew file mode 100644 index 00000000000..5c493109a54 --- /dev/null +++ b/doc/ci/img/parent_pipeline_graph_expanded_v12_6.png diff --git a/doc/ci/parent_child_pipelines.md b/doc/ci/parent_child_pipelines.md new file mode 100644 index 00000000000..269cbd75a9a --- /dev/null +++ b/doc/ci/parent_child_pipelines.md @@ -0,0 +1,86 @@ +--- +type: reference +--- + +# Parent-child pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/16094) in GitLab Starter 12.7. + +As pipelines grow more complex, a few related problems start to emerge: + +- The staged structure, where all steps in a stage must be completed before the first + job in next stage begins, causes arbitrary waits, slowing things down. +- Configuration for the single global pipeline becomes very long and complicated, + making it hard to manage. +- Imports with [`include`](yaml/README.md#include) increase the complexity of the configuration, and create the potential + for namespace collisions where jobs are unintentionally duplicated. +- Pipeline UX can become unwieldy with so many jobs and stages to work with. + +Additionally, sometimes the behavior of a pipeline needs to be more dynamic. The ability +to choose to start sub-pipelines (or not) is a powerful ability, especially if the +YAML is dynamically generated. + + + +Similarly to [multi-project pipelines](multi_project_pipelines.md), a pipeline can trigger a +set of concurrently running child pipelines, but within the same project: + +- Child pipelines still execute each of their jobs according to a stage sequence, but + would be free to continue forward through their stages without waiting for unrelated + jobs in the parent pipeline to finish. +- The configuration is split up into smaller child pipeline configurations, which are + easier to understand. This reduces the cognitive load to understand the overall configuration. +- Imports are done at the child pipeline level, reducing the likelihood of collisions. +- Each pipeline has only the steps relevant steps, making it easier to understand what's going on. + +Child pipelines work well with other GitLab CI features: + +- Use [`only: changes`](yaml/README.md#onlychangesexceptchanges) to trigger pipelines only when + certain files change. This is useful for monorepos, for example. +- Since the parent pipeline in `.gitlab-ci.yml` and the child pipeline run as normal + pipelines, they can have their own behaviors and sequencing in relation to triggers. + +All of this will work with [`include:`](yaml/README.md#include) feature so you can compose +the child pipeline configuration. + +## Examples + +The simplest case is [triggering a child pipeline](yaml/README.md#trigger-premium) using a +local YAML file to define the pipeline configuration. In this case, the parent pipeline will +trigger the child pipeline, and continue without waiting: + +```yaml +microservice_a: + trigger: + include: path/to/microservice_a.yml +``` + +You can include multiple files when composing a child pipeline: + +```yaml +microservice_a: + trigger: + include: + - local: path/to/microservice_a.yml + - template: SAST.gitlab-ci.yml +``` + +NOTE: **Note:** +The max number of entries that are accepted for `trigger:include:` is three. + +Similar to [multi-project pipelines](multi_project_pipelines.md#mirroring-status-from-triggered-pipeline), +we can set the parent pipeline to depend on the status of the child pipeline upon completion: + +```yaml +microservice_a: + trigger: + include: + - local: path/to/microservice_a.yml + - template: SAST.gitlab-ci.yml + strategy: depend +``` + +## Limitations + +A parent pipeline can trigger many child pipelines, but a child pipeline cannot trigger +further child pipelines. See the [related issue](https://gitlab.com/gitlab-org/gitlab/issues/29651) for discussion on possible future improvements. diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 4d942ea3d54..71c4c9ca0ec 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -246,6 +246,13 @@ Pipelines for different projects can be combined and visualized together. For more information, see [Multi-project pipelines](multi_project_pipelines.md). +## Parent-child pipelines + +Complex pipelines can be broken down into one parent pipeline that can trigger +multiple child sub-pipelines, which all run in the same project and with the same SHA. + +For more information, see [Parent-Child pipelines](parent_child_pipelines.md). + ## Working with pipelines In general, pipelines are executed automatically and require no intervention once created. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index e388fdde58a..aafbe4c9189 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -2600,14 +2600,17 @@ job split into three separate jobs. from `trigger` definition is started by GitLab, a downstream pipeline gets created. -Learn more about [multi-project pipelines](../multi_project_pipelines.md#creating-multi-project-pipelines-from-gitlab-ciyml). +This keyword allows the creation of two different types of downstream pipelines: + +- [Multi-project pipelines](../multi_project_pipelines.md#creating-multi-project-pipelines-from-gitlab-ciyml) +- [Child pipelines](../parent_child_pipelines.md) NOTE: **Note:** Using a `trigger` with `when:manual` together results in the error `jobs:#{job-name} when should be on_success, on_failure or always`, because `when:manual` prevents triggers being used. -#### Simple `trigger` syntax +#### Simple `trigger` syntax for multi-project pipelines The simplest way to configure a downstream trigger is to use `trigger` keyword with a full path to a downstream project: @@ -2622,7 +2625,7 @@ staging: trigger: my/deployment ``` -#### Complex `trigger` syntax +#### Complex `trigger` syntax for multi-project pipelines It is possible to configure a branch name that GitLab will use to create a downstream pipeline with: @@ -2657,6 +2660,28 @@ upstream_bridge: pipeline: other/project ``` +#### `trigger` syntax for child pipeline + +To create a [child pipeline](../parent_child_pipelines.md), specify the path to the +YAML file containing the CI config of the child pipeline: + +```yaml +trigger_job: + trigger: + include: path/to/child-pipeline.yml +``` + +Similar to [multi-project pipelines](../multi_project_pipelines.md#mirroring-status-from-triggered-pipeline), +it is possible to mirror the status from a triggered pipeline: + +```yaml +trigger_job: + trigger: + include: + - local: path/to/child-pipeline.yml + strategy: depend +``` + ### `interruptible` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/23464) in GitLab 12.3. |
