diff options
Diffstat (limited to 'doc')
20 files changed, 532 insertions, 18 deletions
diff --git a/doc/source/discussion/components.rst b/doc/source/discussion/components.rst index 5d93bd932..ba36c89c6 100644 --- a/doc/source/discussion/components.rst +++ b/doc/source/discussion/components.rst @@ -64,7 +64,7 @@ TCP port 4730 by default. The Zuul scheduler communicates with Nodepool via the ZooKeeper protocol. Nodepool requires an external ZooKeeper cluster, and the Zuul scheduler needs to be able to connect to the hosts in that -cluster on TCP port 2181. +cluster on TCP port 2181 or 2281. Both the Nodepool launchers and Zuul executors need to be able to communicate with the hosts which nodepool provides. If these are on @@ -193,6 +193,17 @@ The following sections of ``zuul.conf`` are used by all Zuul components: The ZooKeeper session timeout, in seconds. + .. attr:: tls_cert + + If using TLS, the path to the PEM encoded certificate file. + + .. attr:: tls_key + + If using TLS, the path to the PEM encoded key file. + + .. attr:: tls_ca + + If using TLS, the path to the PEM encoded CA certificate file. .. _scheduler: diff --git a/doc/source/discussion/github-checks-api.rst b/doc/source/discussion/github-checks-api.rst new file mode 100644 index 000000000..73788a5d2 --- /dev/null +++ b/doc/source/discussion/github-checks-api.rst @@ -0,0 +1,170 @@ +:title: Github Checks API + +Github Checks API +================= + +Using the `Github Checks API`_ to report job results back to a PR provides +some additional features compared to the status API like file comments and +custom actions. The latter one could be used to e.g. cancel a running +build. + +Design decisions +----------------- + +The github checks API consists mainly of two entities: `Check Suites`_ and +`Check Runs`_. Check suites are a collection of check runs for a specific +commit and summarize their status and conclusion. + +Following this description, one might think that the check suite is a +perfect mapping for a pipeline execution in zuul and a check run could map +to a single job execution that is part of the pipeline run. Unfortunately, +there are a few restrictions that don't allow this kind of mapping. + +First of all, check suites are completely managed by Github. Apart from +creating a check suite for a commit SHA, we can't do anything with it. +The current status, duration and the conclusion are all calculated and +set by Github automatically whenever an included check run is updated. + +There can only be one check suite per commit sha per app. Thus, even if +we could update the check suite, we wouldn't be able to create one check +suite for each pipeline, e.g. check and gate. + +When configuring the branch protection in Github, only a check run can +be selected as required status check. Having each job as a dedicated +check run would result in a huge list of status checks one would have to +enable to make the branch protection work. Additionally, we would then +loose some of Zuul's features like non-voting jobs and it would break +Zuul's gating capabilities as they are working on a pipeline level, not on +a job level. + +Zuul can only report the whole buildset, but no individual jobs. With +that we wouldn't be able to update individual check runs on a job level. + +Having said the above, the only possible integration of the checks API is +on a pipeline level, so each pipeline execution maps to a check run in +Github. + +Behaviour in Zuul +----------------- + +Reporting +~~~~~~~~~ + +The Github reporter is able to report both a status +:attr:`pipeline.<reporter>.<github source>.status` or a check +:attr:`pipeline.<reporter>.<github source>.check`. While it's possible to +configure a Github reporter to report both, it's recommended to use only one. +Reporting both might result in duplicated status check entries in the Github +PR (the section below the comments). + +Trigger +~~~~~~~ + +The Github driver is able to trigger on a reported check +(:value:`pipeline.trigger.<github source>.event.check_run`) similar to a +reported status (:value:`pipeline.trigger.<github source>.action.status`). + +Requirements +~~~~~~~~~~~~ + +While trigger and reporter differentiates between status and check, the Github +driver does not differentiate between them when it comes to pipeline +requirements. This is mainly because Github also doesn't differentiate between +both in terms of branch protection and `status checks`_. + +Actions / Events +---------------- + +Github provides a set of default actions for check suites and check runs. +Those actions are available as buttons in the Github UI. Clicking on those +buttons will emit webhook events which will be handled by Zuul. + +These actions are only available on failed check runs / check suites. So +far, a running or successful check suite / check run does not provide any +action from Github side. + +Available actions are: + +Re-run all checks + Github emits a webhook event with type ``check_suite`` and action + ``rerequested`` that is meant to re-run all check-runs contained in this + check suite. Github does not provide the list of check-runs in that case, + so it's up to the Github app what should run. + +Re-run failed checks + Github emits a webhook event with type ``check_run`` and action + ``rerequested`` for each failed check run contained in this suite. + +Re-run + Github emits a webhook event with type ``check_run`` and action + ``rerequested`` for the specific check run. + +Zuul will handle all events except for the `Re-run all checks` event as +this is not suitable for the Zuul workflow as it doesn't make sense to +trigger all pipelines to run simultaniously. + +The drawback here is, that we are not able to customize those events in Github. +Github will always say "You have successfully requested ..." although we aren't +listening to the event at all. Therefore, it might be a solution to handle the +`Re-run all checks` event in Zuul similar to `Re-run failed checks` just to +not do anything while Github makes the user believe an action was really +triggered. + + +File comments (annotations) +--------------------------- + +Check runs can be used to post file comments directly in the files of the PR. +Those are similar to user comments, but must provide some more information. + +Zuul jobs can already return file comments via ``zuul_return`` +(see: :ref:`return_values`). We can simply use this return value, build the +necessary annotations (how Github calls it) from it and attach them to the +check run. + + +Custom actions +~~~~~~~~~~~~~~ + +Check runs can provide some custom actions which will result in additional +buttons being available in the Github UI for this specific check run. +Clicking on such a button will emit a webhook event with type ``check_run`` +and action ``requested_action`` and will additionally contain the id/name of +the requested action which we can define when creating the action on the +check run. + +We could use these custom actions to provide some "Re-run" action on a +running check run (which might otherwise be stuck in case a check run update +fails) or to abort a check run directly from the Github UI. + + +Restrictions and Recommendations +-------------------------------- + +Although both the checks API and the status API can be activated for a +Github reporter at the same time, it's not recommmended to do so as this might +result in multiple status checks to be reported to the PR for the same pipeline +execution (which would result in duplicated entries in the status section below +the comments of a PR). + +In case the update on a check run fails (e.g. request timeout when reporting +success or failure to Github), the check run will stay in status "in_progess" +and there will be no way to re-run the check run via the Github UI as the +predefined actions are only available on failed check runs. +Thus, it's recommended to configure a +:value:`pipeline.trigger.<github source>.action.comment` trigger on the +pipeline to still be able to trigger re-run of the stuck check run via e.g. +"recheck". + +The check suite will only list check runs that were reported by Zuul. If +the requirements for a certain pipeline are not met and it is not run, the +check run for this pipeline won't be listed in the check suite. However, +this does not affect the required status checks. If the check run is enabled +as required, Github will still show it in the list of required status checks +- even if it didn't run yet - just not in the check suite. + + +.. _Github Checks API: https://developer.github.com/v3/checks/ +.. _Check Suites: https://developer.github.com/v3/checks/suites/ +.. _Check Runs: https://developer.github.com/v3/checks/runs/ +.. _status checks: https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-status-checks#types-of-status-checks-on-github diff --git a/doc/source/discussion/index.rst b/doc/source/discussion/index.rst index a02b5ff97..efa59d6a9 100644 --- a/doc/source/discussion/index.rst +++ b/doc/source/discussion/index.rst @@ -16,3 +16,4 @@ configure it to meet your needs. gating encryption tenant-scoped-rest-api + github-checks-api diff --git a/doc/source/examples/docker-compose.yaml b/doc/source/examples/docker-compose.yaml index 839e9cf9e..ccb85ca1c 100644 --- a/doc/source/examples/docker-compose.yaml +++ b/doc/source/examples/docker-compose.yaml @@ -27,11 +27,17 @@ services: - "sshkey:/var/ssh:z" - "nodessh:/var/node:z" - "./playbooks/:/var/playbooks/:z" + - "certs:/var/certs:z" + - "../../../tools/:/var/zuul-tools/:z" # NOTE(pabelanger): Be sure to update this line each time we change the # default version of ansible for Zuul. command: "/usr/local/lib/zuul/ansible/2.8/bin/ansible-playbook /var/playbooks/setup.yaml" zk: image: zookeeper + hostname: examples_zk_1.examples_default + volumes: + - "certs:/var/certs:z" + - "./zoo.cfg:/conf/zoo.cfg:z" mysql: image: mariadb environment: @@ -59,6 +65,7 @@ services: - "./etc_zuul/:/etc/zuul/:z" - "./playbooks/:/var/playbooks/:z" - "sshkey:/var/ssh:z" + - "certs:/var/certs:z" web: command: "sh -c '/var/playbooks/wait-to-start-gearman.sh && zuul-web -f'" depends_on: @@ -72,6 +79,7 @@ services: volumes: - "./etc_zuul/:/etc/zuul/:z" - "./playbooks/:/var/playbooks/:z" + - "certs:/var/certs:z" executor: privileged: true environment: @@ -103,6 +111,7 @@ services: image: zuul/nodepool-launcher volumes: - "./etc_nodepool/:/etc/nodepool/:z" + - "certs:/var/certs:z" ports: - "8022:8022" logs: @@ -122,3 +131,4 @@ volumes: sshkey: nodessh: logs: + certs: diff --git a/doc/source/examples/etc_nodepool/nodepool.yaml b/doc/source/examples/etc_nodepool/nodepool.yaml index fd4252daa..e5d292b7c 100644 --- a/doc/source/examples/etc_nodepool/nodepool.yaml +++ b/doc/source/examples/etc_nodepool/nodepool.yaml @@ -1,5 +1,10 @@ zookeeper-servers: - host: zk + port: 2281 +zookeeper-tls: + cert: /var/certs/certs/client.pem + key: /var/certs/keys/clientkey.pem + ca: /var/certs/certs/cacert.pem labels: - name: ubuntu-bionic diff --git a/doc/source/examples/etc_zuul/zuul.conf b/doc/source/examples/etc_zuul/zuul.conf index 8b8c34c23..518fd57f3 100644 --- a/doc/source/examples/etc_zuul/zuul.conf +++ b/doc/source/examples/etc_zuul/zuul.conf @@ -5,7 +5,10 @@ server=scheduler start=true [zookeeper] -hosts=zk +hosts=zk:2281 +tls_cert=/var/certs/certs/client.pem +tls_key=/var/certs/keys/clientkey.pem +tls_ca=/var/certs/certs/cacert.pem [scheduler] tenant_config=/etc/zuul/main.yaml diff --git a/doc/source/examples/pipelines/gerrit-reference-pipelines.yaml b/doc/source/examples/pipelines/gerrit-reference-pipelines.yaml index a20647a03..4e0fccaf5 100644 --- a/doc/source/examples/pipelines/gerrit-reference-pipelines.yaml +++ b/doc/source/examples/pipelines/gerrit-reference-pipelines.yaml @@ -100,6 +100,24 @@ sqlreporter: - pipeline: + name: deploy + description: | + This pipeline runs jobs that operate after each change is merged + in order to deploy to production. + manager: serial + precedence: high + post-review: True + trigger: + gerrit: + - event: change-merged + success: + gerrit: {} + sqlreporter: + failure: + gerrit: {} + sqlreporter: + +- pipeline: name: release description: | When a commit is tagged as a release, this pipeline runs jobs diff --git a/doc/source/examples/playbooks/setup.yaml b/doc/source/examples/playbooks/setup.yaml index d6e35365c..c17a971d8 100644 --- a/doc/source/examples/playbooks/setup.yaml +++ b/doc/source/examples/playbooks/setup.yaml @@ -1,6 +1,10 @@ - hosts: localhost gather_facts: false tasks: + - name: Generate ZooKeeper certs + shell: | + /var/zuul-tools/zk-ca.sh /var/certs examples_zk_1.examples_default + chmod -R a+rX /var/certs - name: Wait for Gerrit to start wait_for: host: gerrit diff --git a/doc/source/examples/zoo.cfg b/doc/source/examples/zoo.cfg new file mode 100644 index 000000000..4d4fcc3ea --- /dev/null +++ b/doc/source/examples/zoo.cfg @@ -0,0 +1,15 @@ +dataDir=/data +dataLogDir=/datalog +tickTime=2000 +initLimit=5 +syncLimit=2 +autopurge.snapRetainCount=3 +autopurge.purgeInterval=0 +maxClientCnxns=60 +standaloneEnabled=true +admin.enableServer=true +server.1=examples_zk_1.examples_default:2888:3888 +serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory +secureClientPort=2281 +ssl.keyStore.location=/var/certs/keystores/examples_zk_1.examples_default.pem +ssl.trustStore.location=/var/certs/certs/cacert.pem diff --git a/doc/source/howtos/admin.rst b/doc/source/howtos/admin.rst index a829d2528..9fd4933f2 100644 --- a/doc/source/howtos/admin.rst +++ b/doc/source/howtos/admin.rst @@ -7,3 +7,4 @@ Admin How-to Guides installation zuul-from-scratch troubleshooting + zookeeper diff --git a/doc/source/howtos/zookeeper.rst b/doc/source/howtos/zookeeper.rst new file mode 100644 index 000000000..ef127a599 --- /dev/null +++ b/doc/source/howtos/zookeeper.rst @@ -0,0 +1,118 @@ +ZooKeeper Administration +======================== + +This section will cover some basic tasks and recommendations when +setting up ZooKeeper for use with Zuul. A complete tutorial for +ZooKeeper is out of scope for this documentation. + +Configuration +------------- + +The following general configuration setting in +``/etc/zookeeper/zoo.cfg`` is recommended: + +.. code-block:: + + autopurge.purgeInterval=6 + +This instructs ZooKeeper to purge old snapshots every 6 hours. This +will avoid filling the disk. + +Encrypted Connections +--------------------- + +ZooKeeper version 3.5.1 or greater is required for TLS support. +ZooKeeper performs hostname validation for all ZooKeeper servers +("quorum members"), therefore each member of the ZooKeeper cluster +should have its own certificate. This does not apply to clients which +may share a certificate. + +ZooKeeper performs certificate validation on all connections (server +and client). If you use a private Certificate Authority (CA) (which +is generally recommended and discussed below), then these TLS +certificates not only serve to encrypt traffic, but also to +authenticate and authorize clients to the cluster. Only clients with +certificates authorized by a CA explicitly trusted by your ZooKeeper +installation will be able to connect. + +.. note:: The instructions below direct you to sign certificates with + a CA that you create specifically for Zuul's ZooKeeper + cluster. If you use a CA you share with other users in your + organization, any certificate signed by that CA will be able + to connect to your ZooKeeper cluster. In this case, you may + need to take additional steps such as network isolation to + protect your ZooKeeper cluster. These are beyond the scope + of this document. + +The ``tools/zk-ca.sh`` script in the Zuul source code repository can +be used to quickly and easily generate self-signed certificates for +all ZooKeeper cluster members and clients. + +Make a directory for it to store the certificates and CA data, and run +it once for each client: + +.. code-block:: + + mkdir /etc/zookeeper/ca + tools/zk-ca.sh /etc/zookeeper/ca zookeeper1.example.com + tools/zk-ca.sh /etc/zookeeper/ca zookeeper2.example.com + tools/zk-ca.sh /etc/zookeeper/ca zookeeper3.example.com + +Add the following to ``/etc/zookeeper/zoo.cfg``: + +.. code-block:: + + # Necessary for TLS support + serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory + + # Client TLS configuration + secureClientPort=2281 + ssl.keyStore.location=/etc/zookeeper/ca/keystores/zookeeper1.example.com.jks + ssl.keyStore.password=keystorepassword + ssl.trustStore.location=/etc/zookeeper/ca/certs/cacert.pem + + # Server TLS configuration + sslQuorum=true + ssl.quorum.keyStore.location=/etc/zookeeper/ca/keystores/zookeeper1.example.com.jks + ssl.quorum.keyStore.password=keystorepassword + ssl.quorum.trustStore.location=/etc/zookeeper/ca/certs/cacert.pem + +Change the name of the certificate filenames as appropriate for the +host (e.g., ``zookeeper1.example.com.jks``). Note that the keystore +password ``keystorepassword``, which is set by the ``zk-ca.sh`` +script, does not need to be changed as long as file permissions +provide sufficient protection. The password is present because many +Java utilities misbehave when interacting with keystores with empty or +missing passwords. + +In order to disable plaintext connections, ensure that the +``clientPort`` option does not appear in ``zoo.cfg``. Use the new +method of specifying Zookeeper quorum servers, which looks like this: + +.. code-block:: + + server.1=zookeeper1.example.com:2888:3888 + server.2=zookeeper2.example.com:2888:3888 + server.3=zookeeper3.example.com:2888:3888 + +This format normally includes ``;2181`` at the end of each line, +signifying that the server should listen on port 2181 for plaintext +client connections (this is equivalent to the ``clientPort`` option). +Omit it to disable plaintext connections. The earlier addition of +``secureClientPort`` to the config file instructs ZooKeeper to listen +for encrypted connections on port 2281. + +Be sure to specify port 2281 rather than the standard 2181 in the +:attr:`zookeeper.hosts` setting in ``zuul.conf``. + +Finally, add the :attr:`zookeeper.tls_cert`, +:attr:`zookeeper.tls_key`, and :attr:`zookeeper.tls_ca` options. Your +``zuul.conf`` file should look like: + +.. code-block:: + + [zookeeper] + hosts=zookeeper1.example.com:2281,zookeeper2.example.com:2281,zookeeper3.example.com:2281 + tls_cert=/etc/zookeeper/ca/certs/client.pem + tls_key=/etc/zookeeper/ca/keys/clientkey.pem + tls_ca=/etc/zookeeper/ca/certs/cacert.pem diff --git a/doc/source/reference/developer/specs/kubernetes-operator.rst b/doc/source/reference/developer/specs/kubernetes-operator.rst index b49f4e0f6..18e3e6cc1 100644 --- a/doc/source/reference/developer/specs/kubernetes-operator.rst +++ b/doc/source/reference/developer/specs/kubernetes-operator.rst @@ -191,7 +191,7 @@ needs should be in ConfigMaps and referenced. config: nodepoolYamlConfig builder: config: nodepoolYamlConfig - external_config: + externalConfig: openstack: secretName: cloudsYaml kubernetes: @@ -343,6 +343,46 @@ pods that need them. keyfile: secretName: mqttSecrets +Executor job volume +------------------- + +To manage the executor job volumes, the CR also accepts a list of volumes +to be bind mounted in the job bubblewrap contexts: + +:: + + name: Text + context: <trusted | untrusted> + access: <ro | rw> + path: /path + volume: Kubernetes.Volume + + +For example, to expose a GCP authdaemon token, the Zuul CR can be defined as + +:: + + apiVersion: zuul-ci.org/v1alpha1 + kind: Zuul + spec: + ... + jobVolumes: + - context: trusted + access: ro + path: /authdaemon/token + volume: + name: gcp-auth + hostPath: + path: /var/authdaemon/executor + type: DirectoryOrCreate + +Which would result in a new executor mountpath along with this zuul.conf change: + +:: + + trusted_ro_paths=/authdaemon/token + + Logging ------- diff --git a/doc/source/reference/drivers/github.rst b/doc/source/reference/drivers/github.rst index 474c8ac82..c2c085eca 100644 --- a/doc/source/reference/drivers/github.rst +++ b/doc/source/reference/drivers/github.rst @@ -499,6 +499,21 @@ enqueued into the pipeline. request. The syntax is ``user:status:value``. This can also be a regular expression. + Zuul does not differentiate between a status reported via + status API or via checks API (which is also how Github behaves + in terms of branch protection and `status checks`__). + Thus, the status could be reported by a + :attr:`pipeline.<reporter>.<github source>.status` or a + :attr:`pipeline.<reporter>.<github source>.check`. + + When a status is reported via the status API, Github will add + a ``[bot]`` to the name of the app that reported the status, + resulting in something like ``user[bot]:status:value``. For a + status reported via the checks API, the app's slug will be + used as is. + + .. __: https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-status-checks#types-of-status-checks-on-github + .. attr:: label A string value indicating that the pull request must have the diff --git a/doc/source/reference/drivers/pagure.rst b/doc/source/reference/drivers/pagure.rst index a12f77985..7845ae79a 100644 --- a/doc/source/reference/drivers/pagure.rst +++ b/doc/source/reference/drivers/pagure.rst @@ -24,7 +24,6 @@ Each project to be integrated with Zuul needs: - "Web hook target" set to http://<zuul-web>/zuul/api/connection/<conn-name>/payload -- "Notify on pull-request flag" set to on - "Pull requests" set to on - "Open metadata access to all" set to off (optional, expected if approval based on PR a metadata tag) @@ -32,9 +31,11 @@ Each project to be integrated with Zuul needs: the score requierement (optional, expected if score requierement is defined in a pipeline) -Furthermore, the user must be added as project collaborator **admin** to -be able to read the project's webhook token. This token is used -to validate webhook's payload. +Furthermore, the user must be added as project collaborator +(**ticket** access level), to be able to read the project's +webhook token. This token is used to validate webhook's payload. But +if Zuul is configured to merge pull requests then the access level +must be **commit**. Connection Configuration ------------------------ diff --git a/doc/source/reference/glossary.rst b/doc/source/reference/glossary.rst index 994fdf2fd..536c80034 100644 --- a/doc/source/reference/glossary.rst +++ b/doc/source/reference/glossary.rst @@ -66,7 +66,10 @@ Glossary yet seen any human review, so care must be taken in selecting the kinds of jobs to run, and what resources will be available to them in order to avoid misuse of the system or credential - compromise. + compromise. It usually has an :value:`independent + <pipeline.manager.independent>` pipeline manager since the final + sequence of changes to merge is not generally known at the time + of upload. config-project @@ -94,6 +97,20 @@ Glossary incorporate any cross-project dependencies into the context for the change declaring that dependency relationship. + deploy + + By convention, the name of a continuous-deployment pipeline. + Such a pipeline typically interacts with production systems + rather than ephemeral test nodes. By triggering on merge events + the results of deployment can be reported back to the + originating change. The :value:`serial + <pipeline.manager.serial>` pipeline manager, is recommended if + multiple repositories are involved and only some jobs (based on + file matchers) will be run for each change. If a single repo is + involved and all deployment jobs run on every change merged, + then :value:`supercedent <pipeline.manager.supercedent>` may be + a better fit. + executor The component of Zuul responsible for executing a sandboxed @@ -224,6 +241,21 @@ Glossary of reusing the same sets of jobs in the same pipelines across multiple projects. + promote + + By convention, the name of a pipeline which uploads previously + built artifacts. These artifacts should be constructed in a + :term:`gate` pipeline and uploaded to a temporary location. + When all of the jobs in the gate pipeline succeed, the change + will be merged and may then be enqueued into a promote pipeline. + Jobs running in this pipeline do so with the understanding that + since the change merged as it was tested in the gate, any + artifacts created at that time are now safe to promote to + production. It is a good choice to use a :value:`supercedent + <pipeline.manager.supercedent>` pipeline manager so that if many + changes merge in rapid sequence, Zuul may skip promoting all but + the latest artifact to production. + provided artifact A named artifact which builds of a job are expected to @@ -232,6 +264,30 @@ Glossary same name, allowing these relationships to be defined independent of the specific jobs which provide them. + post + + By convention, the name of a pipeline which runs after a branch + is updated. By triggering on a branch update (rather than a + merge) event, jobs in this pipeline may run with the final git + state after the merge (including any merge commits generated by + the upstream code review system). This is important when + building some artifacts in order that the exact commit ids are + present in the git repo. The downside to this approach is that + jobs in this pipeline run without any connection to the + underlying changes which created the commits. If only the + latest updates to a branch matter, then the :value:`supercedent + <pipeline.manager.supercedent>` pipeline manager is recommended; + otherwise :value:`independent <pipeline.manager.independent>` + may be a better choice. See also :term:`tag` and + :term:`release`. + + release + + By convention, the name of a pipeline which runs after a + release-formatted tag is updated. Other than the matching ref, + this is typically constructed the same as a :term:`post` + pipeline. See also :term:`tag`. + reporter A reporter is a :ref:`pipeline attribute <reporters>` which @@ -276,6 +332,13 @@ Glossary queue items to no longer include it in their respective contexts. + tag + + By convention, the name of a pipeline which runs after a tag is + updated. Other than the matching ref, this is typically + constructed the same as a :term:`post` pipeline. See also + :term:`release`. + tenant A set of projects on which Zuul should operate. Configuration diff --git a/doc/source/reference/governance.rst b/doc/source/reference/governance.rst index 69e6cffc4..64f5f4f3d 100644 --- a/doc/source/reference/governance.rst +++ b/doc/source/reference/governance.rst @@ -69,7 +69,6 @@ Name Freenode Nick ====================== ============= Clark Boylan clarkb Clint Byrum SpamapS -David Shrewsbury Shrews Ian Wienand ianw James E. Blair corvus Jens Harbott frickler @@ -129,6 +128,6 @@ Current Zuul-Jobs Maintainers (in addition to Zuul Maintainers): Name Freenode Nick ====================== ============= Andreas Jaeger AJaeger -David Moreau Simard dmsimard Mohammed Naser mnaser +Albin Vass avass ====================== ============= diff --git a/doc/source/reference/job_def.rst b/doc/source/reference/job_def.rst index b5fe63627..d90dbfc62 100644 --- a/doc/source/reference/job_def.rst +++ b/doc/source/reference/job_def.rst @@ -703,7 +703,7 @@ Here is an example of two job definitions: or *soft* dependency. A *hard* dependency will cause an error if the specified job is not run. That is, if job B depends on job A, but job A is not run for any reason (for - example, it containes a file matcher which does not match), + example, it contains a file matcher which does not match), then Zuul will not run any jobs and report an error. A *soft* dependency will simply be ignored if the dependent job is not run. diff --git a/doc/source/reference/jobs.rst b/doc/source/reference/jobs.rst index b39629604..bd4ee2d11 100644 --- a/doc/source/reference/jobs.rst +++ b/doc/source/reference/jobs.rst @@ -881,10 +881,12 @@ change, set the **zuul.file_comments** value. For example: path/to/file.py: - line: 42 message: "Line too long" + level: info - line: 82 message: "Line too short" - line: 119 message: "This block is indented too far." + level: warning range: start_line: 117 start_character: 0 @@ -893,7 +895,8 @@ change, set the **zuul.file_comments** value. For example: Not all reporters currently support line comments (or all of the features of line comments); in these cases, reporters will simply -ignore this data. +ignore this data. The ``level`` is optional, but if provided must +be one of ``info``, ``warning``, ``error``. Zuul will attempt to automatically translate the supplied line numbers to the corresponding lines in the original change as written (they may diff --git a/doc/source/reference/pipeline_def.rst b/doc/source/reference/pipeline_def.rst index 8b73282e2..e2c2d9587 100644 --- a/doc/source/reference/pipeline_def.rst +++ b/doc/source/reference/pipeline_def.rst @@ -63,7 +63,18 @@ success, the pipeline reports back to Gerrit with ``Verified`` vote of .. attr:: manager :required: - There are three schemes for managing pipelines: + There are several schemes for managing pipelines. The following + table summarizes their features; each is described in detail + below. + + =========== ============================= ============ ===== ============= ========= + Manager Use Case Dependencies Merge Shared Queues Window + =========== ============================= ============ ===== ============= ========= + Independent :term:`check`, :term:`post` No No No Unlimited + Dependent :term:`gate` Yes Yes Yes Variable + Serial :term:`deploy` No No Yes 1 + Supercedent :term:`post`, :term:`promote` No No Project-ref 1 + =========== ============================= ============ ===== ============= ========= .. value:: independent @@ -107,6 +118,22 @@ success, the pipeline reports back to Gerrit with ``Verified`` vote of For more detail on the theory and operation of Zuul's dependent pipeline manager, see: :doc:`/discussion/gating`. + .. value:: serial + + This pipeline manager supports shared queues (like depedent + pipelines) but only one item in each shared queue is + processed at a time. + + This may be useful for post-merge pipelines which perform + partial production deployments (i.e., there are jobs with + file matchers which only deploy to affected parts of the + system). In such a case it is important for every change to + be processed, but they must still be processed one at a time + in order to ensure that the production system is not + inadvertently regressed. Support for shared queues ensures + that if multiple projects are involved deployment runs still + execute sequentially. + .. value:: supercedent This is like an independent pipeline, in that every item is @@ -124,11 +151,12 @@ success, the pipeline reports back to Gerrit with ``Verified`` vote of these cases, build resources can be conserved by avoiding building intermediate versions. - .. note:: Since this pipeline filters intermediate buildsets using - it in combination with file filters on jobs is dangerous. - In this case jobs of in between buildsets can be - unexpectedly skipped entirely. If file filters are needed - the independent pipeline manager should be used. + .. note:: Since this pipeline filters intermediate buildsets + using it in combination with file filters on jobs + is dangerous. In this case jobs of in between + buildsets can be unexpectedly skipped entirely. If + file filters are needed the ``independent`` or + ``serial`` pipeline managers should be used. .. attr:: post-review :default: false diff --git a/doc/source/tutorials/quick-start.rst b/doc/source/tutorials/quick-start.rst index e7c227518..e640ff261 100644 --- a/doc/source/tutorials/quick-start.rst +++ b/doc/source/tutorials/quick-start.rst @@ -530,6 +530,15 @@ output from the job's playbook. Further Steps ------------- +You now have a Zuul system up and running, congratulations! + +The Zuul community would love to hear about how you plan to use Zuul. +Please take a few moments to fill out the `Zuul User Survey +<https://www.surveymonkey.com/r/K2B2MWL>`_ to provide feedback and +information around your deployment. All information is confidential +to the OpenStack Foundation unless you designate that it can be +public. + If you would like to make further changes to Zuul, its configuration files are located in the ``zuul/doc/source/examples`` directory and are bind-mounted into the running containers. You may edit them |