From 959a0b9834cc4c757eec74fabed7792840532b49 Mon Sep 17 00:00:00 2001
From: "James E. Blair" <jim@acmegating.com>
Date: Sat, 30 Apr 2022 14:40:15 -0700
Subject: Deduplicate jobs in dependency cycles

This adds support for deduplicating jobs within dependency cycles.

By default, this will happen automatically if we can determine that the
results of two builds would be expected to be identical.  This uses a
heuristic which should almost always be correct; the behavior can be
overidden otherwise.

Change-Id: I890407df822035d52ead3516942fd95e3633094b
---
 doc/source/config/job.rst                | 38 ++++++++++++++++++++++++++++++++
 doc/source/developer/model-changelog.rst |  7 ++++++
 2 files changed, 45 insertions(+)

(limited to 'doc')

diff --git a/doc/source/config/job.rst b/doc/source/config/job.rst
index da30bc56f..8a8bf4782 100644
--- a/doc/source/config/job.rst
+++ b/doc/source/config/job.rst
@@ -906,6 +906,44 @@ Here is an example of two job definitions:
       self-testing without requiring that the file matchers include
       the Zuul configuration file defining the job.
 
+   .. attr:: deduplicate
+      :default: auto
+
+      In the case of a dependency cycle where multiple changes within
+      the cycle run the same job, this setting indicates whether Zuul
+      should attempt to deduplicate the job.  If it is deduplicated,
+      then the job will only run for one queue item within the cycle
+      and other items which run the same job will use the results of
+      that build.
+
+      This setting determins whether Zuul will consider deduplication.
+      If it is set to ``false``, Zuul will never attempt to
+      deduplicate the job.  If it is set to ``auto`` (the default),
+      then Zuul will compare the job with other jobs of other queue
+      items in the dependency cycle, and if they are equivalent and
+      meet certain project criteria, it will deduplicate them.
+
+      The project criteria that Zuul considers under the ``auto``
+      setting are either:
+
+      * The job must specify :attr:`job.required-projects`.
+      * Or the queue items must be for the same project.
+
+      This is because of the following heuristic: if a job specifies
+      :attr:`job.required-projects`, it is most likely to be one which
+      operates in the same way regardless of which project the change
+      under test belongs to, therefore the result of the same job
+      running on two queue items in the same dependency cycle should
+      be the same.  If a job does not specify
+      :attr:`job.required-projects` and runs with two different
+      projects under test, the outcome is likely different for those
+      two items.
+
+      If this is not true for a job (e.g., the job ignores the project
+      under test and interacts only with external resources)
+      :attr:`job.deduplicate` may be set to ``true`` to ignore the
+      heuristic and deduplicate anyway.
+
    .. attr:: workspace-scheme
       :default: golang
 
diff --git a/doc/source/developer/model-changelog.rst b/doc/source/developer/model-changelog.rst
index efdf50e3d..0d4cb5077 100644
--- a/doc/source/developer/model-changelog.rst
+++ b/doc/source/developer/model-changelog.rst
@@ -79,3 +79,10 @@ Version 7
               Playbook secret references are now either an integer
               index into the job secret list, or a dict with a blob
               store key.  This affects schedulers and executors.
+
+Version 8
+---------
+
+:Prior Zuul version: 6.0.0
+:Description: Deduplicates jobs in dependency cycles.  Affects
+              schedulers only.
-- 
cgit v1.2.1