diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/source/client.rst | 252 | ||||
| -rw-r--r-- | doc/source/concepts.rst | 8 | ||||
| -rw-r--r-- | doc/source/config/job.rst | 118 | ||||
| -rw-r--r-- | doc/source/config/project.rst | 4 | ||||
| -rw-r--r-- | doc/source/config/semaphore.rst | 4 | ||||
| -rw-r--r-- | doc/source/developer/model-changelog.rst | 7 | ||||
| -rw-r--r-- | doc/source/developer/zookeeper.rst | 7 | ||||
| -rw-r--r-- | doc/source/drivers/gerrit.rst | 6 | ||||
| -rw-r--r-- | doc/source/drivers/github.rst | 31 | ||||
| -rw-r--r-- | doc/source/gating.rst | 22 | ||||
| -rw-r--r-- | doc/source/howtos/openid-with-keycloak.rst | 6 | ||||
| -rw-r--r-- | doc/source/tenants.rst | 111 |
12 files changed, 425 insertions, 151 deletions
diff --git a/doc/source/client.rst b/doc/source/client.rst index 0177c7347..cde18bfff 100644 --- a/doc/source/client.rst +++ b/doc/source/client.rst @@ -1,11 +1,12 @@ -:title: Zuul Client +:title: Zuul Admin Client -Zuul Client -=========== +Zuul Admin Client +================= Zuul includes a simple command line client that may be used to affect Zuul's -behavior while running. It must be run on a host with access to Zuul's web -server. +behavior while running. + +.. note:: For operations related to normal workflow like enqueue, dequeue, autohold and promote, the `zuul-client` CLI should be used instead. Configuration ------------- @@ -13,76 +14,177 @@ Configuration The client uses the same zuul.conf file as the server, and will look for it in the same locations if not specified on the command line. -The ``webclient`` section is required. - -It is also possible to run the client without a configuration file, by using the -``--zuul-url`` option to specify the base URL of the Zuul web server. - -.. note:: Not all commands are available through the REST API. - Usage ----- The general options that apply to all subcommands are: -.. program-output:: zuul --help +.. program-output:: zuul-admin --help The following subcommands are supported: +tenant-conf-check +^^^^^^^^^^^^^^^^^ + +.. program-output:: zuul-admin tenant-conf-check --help + +Example:: + + zuul-admin tenant-conf-check + +This command validates the tenant configuration schema. It exits '-1' in +case of errors detected. + +create-auth-token +^^^^^^^^^^^^^^^^^ + +.. note:: This command is only available if an authenticator is configured in + ``zuul.conf``. Furthermore the authenticator's configuration must + include a signing secret. + +.. program-output:: zuul-admin create-auth-token --help + +Example:: + + zuul-admin create-auth-token --auth-config zuul-operator --user alice --tenant tenantA --expires-in 1800 + +The return value is the value of the ``Authorization`` header the user must set +when querying a protected endpoint on Zuul's REST API. + +Example:: + + bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwOi8vbWFuYWdlc2Yuc2ZyZG90ZXN0aW5zdGFuY2Uub3JnIiwienV1bC50ZW5hbnRzIjp7ImxvY2FsIjoiKiJ9LCJleHAiOjE1Mzc0MTcxOTguMzc3NTQ0fQ.DLbKx1J84wV4Vm7sv3zw9Bw9-WuIka7WkPQxGDAHz7s + +export-keys +^^^^^^^^^^^ + +.. program-output:: zuul-admin export-keys --help + +Example:: + + zuul-admin export-keys /var/backup/zuul-keys.json + +import-keys +^^^^^^^^^^^ + +.. program-output:: zuul-admin import-keys --help + +Example:: + + zuul-admin import-keys /var/backup/zuul-keys.json + +copy-keys +^^^^^^^^^ + +.. program-output:: zuul-admin copy-keys --help + +Example:: + + zuul-admin copy-keys gerrit old_project gerrit new_project + +delete-keys +^^^^^^^^^^^ + +.. program-output:: zuul-admin delete-keys --help + +Example:: + + zuul-admin delete-keys gerrit old_project + +delete-state +^^^^^^^^^^^^ + +.. program-output:: zuul-admin delete-state --help + +Example:: + + zuul-admin delete-state + +delete-pipeline-state +^^^^^^^^^^^^^^^^^^^^^ + +.. program-output:: zuul-admin delete-pipeline-state --help + +Example:: + + zuul-admin delete-pipeline-state tenant pipeline + +prune-database +^^^^^^^^^^^^^^ + +.. program-output:: zuul-admin prune-database --help + +Example:: + + zuul-admin prune-database --older-than 180d + +Deprecated commands +------------------- + +The following commands are deprecated in the zuul-admin CLI, and thus may not be entirely supported in Zuul's current version. +They will be removed in a future release of Zuul. They can still be performed via the `zuul-client` CLI. +Please refer to `zuul-client's documentation <https://zuul-ci.org/docs/zuul-client/>`__ +for more details. + +In order to run these commands, the ``webclient`` section is required in the configuration file. + +It is also possible to run the client without a configuration file, by using the +``--zuul-url`` option to specify the base URL of the Zuul web server. + Autohold ^^^^^^^^ -.. program-output:: zuul autohold --help +.. program-output:: zuul-admin autohold --help Example:: - zuul autohold --tenant openstack --project example_project --job example_job --reason "reason text" --count 1 + zuul-admin autohold --tenant openstack --project example_project --job example_job --reason "reason text" --count 1 Autohold Delete ^^^^^^^^^^^^^^^ -.. program-output:: zuul autohold-delete --help +.. program-output:: zuul-admin autohold-delete --help Example:: - zuul autohold-delete --id 0000000123 + zuul-admin autohold-delete --id 0000000123 Autohold Info ^^^^^^^^^^^^^ -.. program-output:: zuul autohold-info --help +.. program-output:: zuul-admin autohold-info --help Example:: - zuul autohold-info --id 0000000123 + zuul-admin autohold-info --id 0000000123 Autohold List ^^^^^^^^^^^^^ -.. program-output:: zuul autohold-list --help +.. program-output:: zuul-admin autohold-list --help Example:: - zuul autohold-list --tenant openstack + zuul-admin autohold-list --tenant openstack Dequeue ^^^^^^^ -.. program-output:: zuul dequeue --help +.. program-output:: zuul-admin dequeue --help Examples:: - zuul dequeue --tenant openstack --pipeline check --project example_project --change 5,1 - zuul dequeue --tenant openstack --pipeline periodic --project example_project --ref refs/heads/master + zuul-admin dequeue --tenant openstack --pipeline check --project example_project --change 5,1 + zuul-admin dequeue --tenant openstack --pipeline periodic --project example_project --ref refs/heads/master Enqueue ^^^^^^^ -.. program-output:: zuul enqueue --help +.. program-output:: zuul-admin enqueue --help Example:: - zuul enqueue --tenant openstack --trigger gerrit --pipeline check --project example_project --change 12345,1 + zuul-admin enqueue --tenant openstack --trigger gerrit --pipeline check --project example_project --change 12345,1 Note that the format of change id is <number>,<patchset>. Enqueue-ref ^^^^^^^^^^^ -.. program-output:: zuul enqueue-ref --help +.. program-output:: zuul-admin enqueue-ref --help This command is provided to manually simulate a trigger from an external source. It can be useful for testing or replaying a trigger @@ -106,7 +208,7 @@ the jobs, pass the failed tag as the ``ref`` argument and set ``newrev`` to the change associated with the tag in the project repository (i.e. what you see from ``git show X.Y.Z``):: - zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline release --project openstack/example_project --ref refs/tags/X.Y.Z --newrev abc123.. + zuul-admin enqueue-ref --tenant openstack --trigger gerrit --pipeline release --project openstack/example_project --ref refs/tags/X.Y.Z --newrev abc123.. The command can also be used asynchronosly trigger a job in a ``periodic`` pipeline that would usually be run at a specific time by @@ -114,7 +216,7 @@ the ``timer`` driver. For example, the following command would trigger the ``periodic`` jobs against the current ``master`` branch top-of-tree for a project:: - zuul enqueue-ref --tenant openstack --trigger timer --pipeline periodic --project openstack/example_project --ref refs/heads/master + zuul-admin enqueue-ref --tenant openstack --trigger timer --pipeline periodic --project openstack/example_project --ref refs/heads/master Another common pipeline is a ``post`` queue listening for ``gerrit`` merge results. Triggering here is slightly more complicated as you @@ -128,7 +230,7 @@ current ``HEAD`` and the prior change, then enqueue the event:: NEW_REF=$(git rev-parse HEAD) OLD_REF=$(git rev-parse HEAD~1) - zuul enqueue-ref --tenant openstack --trigger gerrit --pipeline post --project openstack/example_project --ref refs/heads/master --newrev $NEW_REF --oldrev $OLD_REF + zuul-admin enqueue-ref --tenant openstack --trigger gerrit --pipeline post --project openstack/example_project --ref refs/heads/master --newrev $NEW_REF --oldrev $OLD_REF Note that zero values for ``oldrev`` and ``newrev`` can indicate branch creation and deletion; the source code is the best reference @@ -138,11 +240,11 @@ for these more advanced operations. Promote ^^^^^^^ -.. program-output:: zuul promote --help +.. program-output:: zuul-admin promote --help Example:: - zuul promote --tenant openstack --pipeline gate --changes 12345,1 13336,3 + zuul-admin promote --tenant openstack --pipeline gate --changes 12345,1 13336,3 Note that the format of changes id is <number>,<patchset>. @@ -164,89 +266,3 @@ If items in independent pipelines are promoted, no jobs will be restarted, but their change queues within the pipeline will be re-ordered so that they will be processed first and their node request priorities will increase. - -tenant-conf-check -^^^^^^^^^^^^^^^^^ - -.. program-output:: zuul tenant-conf-check --help - -Example:: - - zuul tenant-conf-check - -This command validates the tenant configuration schema. It exits '-1' in -case of errors detected. - -create-auth-token -^^^^^^^^^^^^^^^^^ - -.. note:: This command is only available if an authenticator is configured in - ``zuul.conf``. Furthermore the authenticator's configuration must - include a signing secret. - -.. program-output:: zuul create-auth-token --help - -Example:: - - zuul create-auth-token --auth-config zuul-operator --user alice --tenant tenantA --expires-in 1800 - -The return value is the value of the ``Authorization`` header the user must set -when querying a protected endpoint on Zuul's REST API. - -Example:: - - bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwOi8vbWFuYWdlc2Yuc2ZyZG90ZXN0aW5zdGFuY2Uub3JnIiwienV1bC50ZW5hbnRzIjp7ImxvY2FsIjoiKiJ9LCJleHAiOjE1Mzc0MTcxOTguMzc3NTQ0fQ.DLbKx1J84wV4Vm7sv3zw9Bw9-WuIka7WkPQxGDAHz7s - -export-keys -^^^^^^^^^^^ - -.. program-output:: zuul export-keys --help - -Example:: - - zuul export-keys /var/backup/zuul-keys.json - -import-keys -^^^^^^^^^^^ - -.. program-output:: zuul import-keys --help - -Example:: - - zuul import-keys /var/backup/zuul-keys.json - -copy-keys -^^^^^^^^^ - -.. program-output:: zuul copy-keys --help - -Example:: - - zuul copy-keys gerrit old_project gerrit new_project - -delete-keys -^^^^^^^^^^^ - -.. program-output:: zuul delete-keys --help - -Example:: - - zuul delete-keys gerrit old_project - -delete-state -^^^^^^^^^^^^ - -.. program-output:: zuul delete-state --help - -Example:: - - zuul delete-state - -delete-pipeline-state -^^^^^^^^^^^^^^^^^^^^^ - -.. program-output:: zuul delete-pipeline-state --help - -Example:: - - zuul delete-pipeline-state tenant pipeline diff --git a/doc/source/concepts.rst b/doc/source/concepts.rst index 7bf47e581..92384dbf6 100644 --- a/doc/source/concepts.rst +++ b/doc/source/concepts.rst @@ -53,8 +53,12 @@ substantially different than the git repo state of the change itself since the repo may have merged other changes since the change was originally authored). Items in a pipeline may depend on other items, and if they do, all of their dependent changes will be included in the -git repo state that Zuul prepares. For more detail on this process, -see :ref:`project_gating` and :ref:`dependencies`. +git repo state that Zuul prepares. Jobs may also specify that they +require additional git repos, and if so, the repo state (as of the +time when the item was enqueued in the pipeline) for those repos will +also be included. For more detail on this process, see +:ref:`project_gating`, :ref:`dependencies`, and +:ref:`global_repo_state`. The configuration for nearly everything described above is held in files inside of the git repos upon which Zuul operates. Zuul's diff --git a/doc/source/config/job.rst b/doc/source/config/job.rst index da30bc56f..87806727f 100644 --- a/doc/source/config/job.rst +++ b/doc/source/config/job.rst @@ -18,7 +18,10 @@ starting with very basic jobs which describe characteristics that all jobs on the system should have, progressing through stages of specialization before arriving at a particular job. A job may inherit from any other job in any project (however, if the other job is marked -as :attr:`job.final`, jobs may not inherit from it). +as :attr:`job.final`, jobs may not inherit from it). Generally, +attributes on child jobs will override (or completely replace) +attributes on the parent, however some attributes are combined. See +the documentation for individual attributes for these exceptions. A job with no parent is called a *base job* and may only be defined in a :term:`config-project`. Every other job must have a parent, and so @@ -58,6 +61,7 @@ be: * child playbook * child post-run playbook * parent post-run playbook +* parent cleanup-run playbook Further inheritance would nest even deeper. @@ -146,6 +150,11 @@ Here is an example of two job definitions: To indicate a job is not intended to be run directly, but instead must be inherited from, set this attribute to ``true``. + Once this is set to ``true`` in a job it cannot be reset to + ``false`` within the same job by other variants; however jobs + which inherit from it can (and by default do) reset it to + ``false``. + .. warning:: It is possible to circumvent the use of `abstract` in an @@ -162,6 +171,11 @@ Here is an example of two job definitions: *must* also be ``abstract``; a configuration error will be raised if not. + Once this is set to ``true`` in a job it cannot be reset to + ``false`` within the same job by other variants; however jobs + which inherit from it can (and by default do) reset it to + ``false``. + For example, you may define a base abstract job `foo` and create two abstract jobs that inherit from `foo` called `foo-production` and `foo-development`. If it would be an error @@ -205,14 +219,14 @@ Here is an example of two job definitions: .. attr:: semaphores - The name of a :ref:`semaphore` (or list of them) which should be - acquired and released when the job begins and ends. If the - semaphore is at maximum capacity, then Zuul will wait until it - can be acquired before starting the job. The format is either a - string, a dictionary, or a list of either of those in the case - of multiple semaphores. If it's a string it references a - semaphore using the default value for - :attr:`job.semaphores.resources-first`. + The name of a :ref:`semaphore` (or list of them) or + :ref:`global_semaphore` which should be acquired and released + when the job begins and ends. If the semaphore is at maximum + capacity, then Zuul will wait until it can be acquired before + starting the job. The format is either a string, a dictionary, + or a list of either of those in the case of multiple + semaphores. If it's a string it references a semaphore using the + default value for :attr:`job.semaphores.resources-first`. If multiple semaphores are requested, the job will not start until all have been acquired, and Zuul will wait until all are @@ -258,6 +272,10 @@ Here is an example of two job definitions: by this job which may be used by other jobs for other changes using the :attr:`job.requires` attribute. + When inheriting jobs or applying variants, the list of + `provides` is extended (`provides` specified in a job definition + are added to any supplied by their parents). + .. attr:: requires A list of free-form strings which identify resources which may @@ -277,6 +295,10 @@ Here is an example of two job definitions: returned by the `provides` jobs will be made available to the `requires` job. + When inheriting jobs or applying variants, the list of + `requires` is extended (`requires` specified in a job definition + are added to any supplied by their parents). + For example, a job which produces a builder container image in one project that is then consumed by a container image build job in another project might look like this: @@ -488,6 +510,10 @@ Here is an example of two job definitions: even when the job is canceled. Cleanup results are not taken into account. + When a job inherits from a parent, the child's cleanup-run + playbooks are run before the parent's. See :ref:`job` for more + information. + .. attr:: run The name of a playbook or list of playbooks for this job. If it @@ -539,15 +565,18 @@ Here is an example of two job definitions: appear on the job -- roles earlier in the list will take precedence over those which follow. - In the case of job inheritance or variance, the roles used for - each of the playbooks run by the job will be only those which - were defined along with that playbook. If a child job inherits - from a parent which defines a pre and post playbook, then the - pre and post playbooks it inherits from the parent job will run - only with the roles that were defined on the parent. If the - child adds its own pre and post playbooks, then any roles added - by the child will be available to the child's playbooks. This - is so that a job which inherits from a parent does not + This attribute is not overridden on inheritance or variance; + instead roles are added with each new job or variant. In the + case of job inheritance or variance, the roles used for each of + the playbooks run by the job will be only those which were + cumulatively defined up to that point in the inheritance + hierarchy where that playbook was added. If a child job + inherits from a parent which defines a pre and post playbook, + then the pre and post playbooks it inherits from the parent job + will run only with the roles that were defined on the parent. + If the child adds its own pre and post playbooks, then any roles + added by the child will be available to the child's playbooks. + This is so that a job which inherits from a parent does not inadvertently alter the behavior of the parent's playbooks by the addition of conflicting roles. Roles added by a child will appear before those it inherits from its parent. @@ -625,9 +654,15 @@ Here is an example of two job definitions: A list of other projects which are used by this job. Any Zuul projects specified here will also be checked out by Zuul into the working directory for the job. Speculative merging and - cross-repo dependencies will be honored. + cross-repo dependencies will be honored. If there is not a + change for the project ahead in the pipeline, its repo state as + of the time the item was enqueued will be frozen and used for + all jobs for a given change (see :ref:`global_repo_state`). - This attribute is a union of all applicable parents and variants. + This attribute is not overridden by inheritance; instead it is + the union of all applicable parents and variants (i.e., jobs can + expand but not reduce the set of required projects when they + inherit). The format for this attribute is either a list of strings or dictionaries. Strings are interpreted as project names, @@ -777,6 +812,11 @@ Here is an example of two job definitions: any project's pipeline. Apply caution when doing so as other projects may be able to expose the source project's secrets. + This attribute is not overridden by inheritance; instead it is + the intersection of all applicable parents and variants (i.e., + jobs can reduce but not expand the set of allowed projects when + they inherit). + .. warning:: It is possible to circumvent the use of `allowed-projects` in @@ -906,6 +946,44 @@ Here is an example of two job definitions: self-testing without requiring that the file matchers include the Zuul configuration file defining the job. + .. attr:: deduplicate + :default: auto + + In the case of a dependency cycle where multiple changes within + the cycle run the same job, this setting indicates whether Zuul + should attempt to deduplicate the job. If it is deduplicated, + then the job will only run for one queue item within the cycle + and other items which run the same job will use the results of + that build. + + This setting determins whether Zuul will consider deduplication. + If it is set to ``false``, Zuul will never attempt to + deduplicate the job. If it is set to ``auto`` (the default), + then Zuul will compare the job with other jobs of other queue + items in the dependency cycle, and if they are equivalent and + meet certain project criteria, it will deduplicate them. + + The project criteria that Zuul considers under the ``auto`` + setting are either: + + * The job must specify :attr:`job.required-projects`. + * Or the queue items must be for the same project. + + This is because of the following heuristic: if a job specifies + :attr:`job.required-projects`, it is most likely to be one which + operates in the same way regardless of which project the change + under test belongs to, therefore the result of the same job + running on two queue items in the same dependency cycle should + be the same. If a job does not specify + :attr:`job.required-projects` and runs with two different + projects under test, the outcome is likely different for those + two items. + + If this is not true for a job (e.g., the job ignores the project + under test and interacts only with external resources) + :attr:`job.deduplicate` may be set to ``true`` to ignore the + heuristic and deduplicate anyway. + .. attr:: workspace-scheme :default: golang diff --git a/doc/source/config/project.rst b/doc/source/config/project.rst index af5053295..301e0b08d 100644 --- a/doc/source/config/project.rst +++ b/doc/source/config/project.rst @@ -166,10 +166,6 @@ pipeline. stanza; it may appear in secondary instances or even in a :ref:`project-template` definition. - Pipeline managers other than `dependent` do not use this - attribute, however, it may still be used if - :attr:`scheduler.relative_priority` is enabled. - .. note:: This attribute is not evaluated speculatively and its setting shall be merged to be effective. diff --git a/doc/source/config/semaphore.rst b/doc/source/config/semaphore.rst index 74440ae79..91bfc9a4d 100644 --- a/doc/source/config/semaphore.rst +++ b/doc/source/config/semaphore.rst @@ -15,6 +15,10 @@ project as long as the value is the same. This is to aid in branch maintenance, so that creating a new branch based on an existing branch will not immediately produce a configuration error. +Zuul also supports global semaphores (see :ref:`global_semaphore`) +which may only be created by the Zuul administrator, but can be used +to coordinate resources across multiple tenants. + Semaphores are never subject to dynamic reconfiguration. If the value of a semaphore is changed, it will take effect only when the change where it is updated is merged. However, Zuul will attempt to validate diff --git a/doc/source/developer/model-changelog.rst b/doc/source/developer/model-changelog.rst index efdf50e3d..0d4cb5077 100644 --- a/doc/source/developer/model-changelog.rst +++ b/doc/source/developer/model-changelog.rst @@ -79,3 +79,10 @@ Version 7 Playbook secret references are now either an integer index into the job secret list, or a dict with a blob store key. This affects schedulers and executors. + +Version 8 +--------- + +:Prior Zuul version: 6.0.0 +:Description: Deduplicates jobs in dependency cycles. Affects + schedulers only. diff --git a/doc/source/developer/zookeeper.rst b/doc/source/developer/zookeeper.rst index c14047ad6..ee77f815f 100644 --- a/doc/source/developer/zookeeper.rst +++ b/doc/source/developer/zookeeper.rst @@ -414,6 +414,13 @@ This is a reference for object layout in Zookeeper. An election to decide which scheduler will report system-wide stats (such as total node requests). +.. path:: zuul/global-semaphores/<semaphore> + :type: SemaphoreHandler + + Represents a global semaphore (shared by multiple tenants). + Information about which builds hold the semaphore is stored in the + znode data. + .. path:: zuul/semaphores/<tenant>/<semaphore> :type: SemaphoreHandler diff --git a/doc/source/drivers/gerrit.rst b/doc/source/drivers/gerrit.rst index 8d746d465..7d052f946 100644 --- a/doc/source/drivers/gerrit.rst +++ b/doc/source/drivers/gerrit.rst @@ -51,6 +51,12 @@ The supported options in ``zuul.conf`` connections are: Fully qualified domain name of Gerrit server. + .. attr:: ssh_server + + If SSH access to the Gerrit server should be via a different + hostname than web access, set this value to the hostname to use + for SSH connections. + .. attr:: canonical_hostname The canonical hostname associated with the git repos on the diff --git a/doc/source/drivers/github.rst b/doc/source/drivers/github.rst index 42a56bf2a..9ca7a1f38 100644 --- a/doc/source/drivers/github.rst +++ b/doc/source/drivers/github.rst @@ -182,6 +182,37 @@ The supported options in ``zuul.conf`` connections are: Enable or disable GitHub rate limit logging. If rate limiting is disabled in GitHub Enterprise this can save some network round trip times. + .. attr:: repo_cache + + To configure Zuul to use a GitHub Enterprise `repository cache + <https://docs.github.com/en/enterprise-server@3.3/admin/enterprise-management/caching-repositories/about-repository-caching>`_ + set this value to the hostname of the cache (e.g., + ``europe-ci.github.example.com``). Zuul will fetch commits as + well as determine the global repo state of repositories used in + jobs from this host. + + This setting is incompatible with :attr:`<github + connection>.sshkey`. + + Because the repository cache may be several minutes behind the + canonical site, enabling this setting automatically sets the + default :attr:`<github connection>.repo_retry_timeout` to 600 + seconds. That setting may still be overidden to specify a + different value. + + .. attr:: repo_retry_timeout + + This setting is only used if :attr:`<github + connection>.repo_cache` is set. It specifies the amount of time + in seconds that Zuul mergers and executors should spend + attempting to fetch git commits which are not available from the + GitHub repository cache host. + + When :attr:`<github connection>.repo_cache` is set, this value + defaults to 600 seconds, but it can be overridden. Zuul retries + git fetches every 30 seconds, and this value will be rounded up + to the next highest multiple of 30 seconds. + Trigger Configuration --------------------- GitHub webhook events can be configured as triggers. diff --git a/doc/source/gating.rst b/doc/source/gating.rst index e6b4fed47..325313b54 100644 --- a/doc/source/gating.rst +++ b/doc/source/gating.rst @@ -405,7 +405,21 @@ change in project C. Cycles ~~~~~~ -If a cycle is created by use of cross-project dependencies, Zuul will -abort its work very early. There will be no message in Gerrit and no -changes that are part of the cycle will be enqueued into any pipeline. -This is to protect Zuul from infinite loops. +Zuul supports cycles that are created by use of cross-project dependencies. +However this feature is opt-in and can be configured on the queue. +See :attr:`queue.allow-circular-dependencies` for information on how to +configure this. + +.. _global_repo_state: + +Global Repo State +~~~~~~~~~~~~~~~~~ + +If a git repository is used by at least one job for a queue item, then +Zuul will freeze the repo state (i.e., branch heads and tags) and use +that same state for every job run for that queue item. Not every job +will get a git repo checkout of every repo, but for any repo that is +checked out, it will have the same state. Because of this, authors +can be sure that jobs running on the same queue item have a consistent +view of all involved git repos, even if one job starts running much +later than another. diff --git a/doc/source/howtos/openid-with-keycloak.rst b/doc/source/howtos/openid-with-keycloak.rst index e80376801..74d3a27c0 100644 --- a/doc/source/howtos/openid-with-keycloak.rst +++ b/doc/source/howtos/openid-with-keycloak.rst @@ -10,7 +10,7 @@ Prerequisites * The Zuul instance must be able to query Keycloak over HTTPS. * Authenticating users must be able to reach Keycloak's web UI. * Have a realm set up in Keycloak. - `Instructions on how to do so can be found here <https://www.keycloak.org/docs/latest/getting_started/index.html#creating-a-realm-and-user>`_ . + `Instructions on how to do so can be found here <https://www.keycloak.org/docs/latest/server_admin/#configuring-realms>`_ . By convention, we will assume the Keycloak server's FQDN is ``keycloak``, and Zuul's Web UI's base URL is ``https://zuul/``. We will use the realm ``my_realm``. @@ -81,7 +81,7 @@ Keycloak can delegate authentication to predefined social networks. Follow If you don't set up authentication delegation, make sure to create at least one user in your realm, or allow self-registration. See Keycloak's documentation section -on `user management <https://www.keycloak.org/docs/latest/server_admin/index.html#user-management>`_ +on `user management <https://www.keycloak.org/docs/latest/server_admin/index.html#assembly-managing-users_server_administration_guide>`_ for more details on how to do so. Setting up Zuul @@ -107,4 +107,4 @@ Further Reading --------------- This How-To is based on `Keycloak's documentation <https://www.keycloak.org/documentation.html>`_, -specifically `the documentation about clients <https://www.keycloak.org/docs/latest/server_admin/#_clients>`_. +specifically `the documentation about clients <https://www.keycloak.org/docs/latest/server_admin/#assembly-managing-clients_server_administration_guide>`_. diff --git a/doc/source/tenants.rst b/doc/source/tenants.rst index 7835eef46..fbbb458a5 100644 --- a/doc/source/tenants.rst +++ b/doc/source/tenants.rst @@ -211,6 +211,66 @@ configuration. Some examples of tenant definitions are: exclude-unprotected-branches. This currently only affects GitHub and GitLab projects. + .. attr:: include-branches + + A list of regexes matching branches which should be + processed. If omitted, all branches are included. + Operates after *exclude-unprotected-branches* and so may + be used to further reduce the set of branches (but not + increase it). + + It has priority over *exclude-branches*. + + .. attr:: exclude-branches + + A list of regexes matching branches which should be + processed. If omitted, all branches are included. + Operates after *exclude-unprotected-branches* and so may + be used to further reduce the set of branches (but not + increase it). + + It will not exclude a branch which already matched + *include-branches*. + + .. attr:: always-dynamic-branches + + A list of regular expressions matching branches which + should be treated as if every change newly proposes + dynamic Zuul configuration. In other words, the only time + Zuul will realize any configuration related to these + branches is during the time it is running jobs for a + proposed change. + + This is potentially useful for situations with large + numbers of rarely used feature branches, but comes at the + cost of a significant reduction in Zuul features for these + branches. + + Every regular expression listed here will also implicitly + be included in *exclude-branches*, therefore Zuul will not + load any static in-repo configuration from this branch. + These branches will not be available for use in overriding + checkouts of repos, nor will they be included in the git + repos that Zuul prepares for *required-projects* (unless + there is a change in the dependency tree for this branch). + + In particular, this means that the only jobs which can be + specified for these branches are pre-merge and gating jobs + (such as :term:`check` and :term:`gate`). No post-merge + or periodic jobs will run for these branches. + + Using this setting also incurs additional processing for + each change submitted for these branches as Zuul must + recalculate the configuration layout it uses for such a + change as if it included a change to a ``zuul.yaml`` file, + even if the change does not alter the configuration). + + With all these caveats in mind, this can be useful for + repos with large numbers of rarely used branches as it + allows Zuul to omit their configuration in most + circumstances and only calculate the configuration of a + single additional branch when it is used. + .. attr:: extra-config-paths Normally Zuul loads in-repo configuration from the first @@ -363,6 +423,57 @@ configuration. Some examples of tenant definitions are: to add finer filtering to admin rules, for example filtering by the ``iss`` claim (generally equal to the issuer ID). + .. attr:: semaphores + + A list of names of :attr:`global-semaphore` objects to allow + jobs in this tenant to access. + +.. _global_semaphore: + +Global Semaphore +---------------- + +Semaphores are normally defined in in-repo configuration (see +:ref:`semaphore`), however to support use-cases where semaphores are +used to represent constrained global resources that may be used by +multiple Zuul tenants, semaphores may be defined within the main +tenant configuration file. + +In order for a job to use a global semaphore, the semaphore must first +be defined in the tenant configuration file with +:attr:`global-semaphore` and then added to each tenant which should +have access to it with :attr:`tenant.semaphores`. Once that is done, +Zuul jobs may use that semaphore in the same way they would use a +normal tenant-scoped semaphore. + +If any tenant which is granted access to a global semaphore also has a +tenant-scoped semaphore defined with the same name, that definition +will be treated as a configuration error and subsequently ignored in +favor of the global semaphore. + +An example definition looks similar to the normal semaphore object: + +.. code-block:: yaml + + - global-semaphore: + name: global-semaphore-foo + max: 5 + +.. attr:: global-semaphore + + The following attributes are available: + + .. attr:: name + :required: + + The name of the semaphore, referenced by jobs. + + .. attr:: max + :default: 1 + + The maximum number of running jobs which can use this semaphore. + + .. _admin_rule_definition: Access Rule |