summaryrefslogtreecommitdiff
path: root/doc/source/howtos
Commit message (Collapse)AuthorAgeFilesLines
* docs: remove orphaned howto pagesIan Wienand2022-11-1110-876/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The zuul-from-scratch page was removed with I3c6327c9bc1a924f076ded06afc0afc4e3024384, but all these files it linked to were left behind. At first glance this seemed a bit odd, because sphinx should warn when pages aren't linked to from a TOC. It took me a while to realise these pages were already marked with :orphan: at the top which stopped this happening. So they really are orphans now, but we haven't noticed. This appears to go back well before the zuul-from-scratch removal to some of the original organisation several years ago in I206a2acf09eb8a2871ec61a00226654c798bb3eb -- it's not clear if this flag was intended to be left in the files or was a temporary step; but it seems that as we've gone on we've copied it into all the other files as they got created too. Most of this is all old and part of the bit-rot as described in the original zuul-from-scratch removal. The only recent part is some console streaming docs added with I5bfb61323bf3219168d4d014cbb9703eed230e71 -- upon reevaluating this I've moved it into the executor docs where it seems to fit. The other orphaned files are removed. Change-Id: Id3669418189f1083a2fb690ada0b60043a77b1d6
* Fix typo for zuul_console_disabled flagIan Wienand2022-10-111-1/+1
| | | | | | | | This was introduced with I0273993c3ece4363098e4bf30bfc4308bb69a8b4. The variable being checked is "zuul_console_disabled". Correct this in the documentation. Change-Id: Ib45ec943d4b227ba254354d116440aa521fb6b9e
* zuul-stream: Add variable to disable writing streaming filesIan Wienand2022-09-141-6/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Upon further discussion we recently found another case of leaking console streaming files; if the zuul_console is not running on port 19885, or can not be reached, the streaming spool files will still be leaked. The prior work in I823156dc2bcae91bd6d9770bd1520aa55ad875b4 has the receiving side indicate to the zuul_console daemon that it should remove the spool file. If this doesn't happen, either because the daemon was never there, or it is firewalled off, the streaming spool files are left behind. This modifies the command action plugin to look for a variable "zuul_console_disable" which will indicate to the library running the shell/command task not to write out the spool file at all, as it will not be consumed. It is expected this would be set at a host level in inventory for nodes that you know can not or will not have access to zuul_console daemon. We already have a mechanism to disable this for commands running in a loop; we expand this with a new string type. The advantage of this is it leaves the library/command.py side basically untouched. Documentation is updated, and we cover this with a new test. Change-Id: I0273993c3ece4363098e4bf30bfc4308bb69a8b4
* docs: update console streaming docsIan Wienand2022-09-011-6/+26
| | | | | | | | Update the developer Ansible docs to give some more details on how the zuul_console daemon streaming happens. In a couple of places where it is mentioned, rework things and point them at this explaination. Change-Id: I5bfb61323bf3219168d4d014cbb9703eed230e71
* Update keycloak demo, documentation for keycloak 18Matthieu Huin2022-07-221-0/+13
| | | | | | | | | | Make the compose demo work for keycloak's latest major version at this time. Document unsupported key signing algorithm type, and how to disable it in keycloak. Change-Id: I102c0fe7abd25b8a98e9c19e6f9ce1f2e7f3f502
* Update links in keycloak integration docDr. Jens Harbott2022-06-031-3/+3
| | | | | | | Some URLs in the keycloak docs were no longer working properly, update them to current version. Change-Id: I5b501af1ec08c4840f2a6861c9a767a4187477be
* Add support for Microsoft loginJames E. Blair2022-02-111-0/+84
| | | | | | | | | | | Microsoft login (or Azure AD) nearly supports the OpenID Connect protocol. With extensive configuration and some additional customization of the values expected in the tokens, it will work with Zuul. Add the necessary additional configuration and a HOWTO with the procedure. Change-Id: I445a0494796572762fbf78e580d7d6dd17be7239
* Improve documentation around ZK requirementsClark Boylan2022-01-121-0/+1
| | | | | | | | | Zuul requires ZK connections are encrypted and ZK 3.5.1 or newer is required to make that happen. Make this a bit more clear in the documentation. Story: 2009317 Change-Id: Icb335f69446f7db3d3e1e018d031c31c9a2be98b
* Remove zuul-from-scratch howtoJames E. Blair2021-12-162-143/+0
| | | | | | This has not been updated in a while and in starting to bitrot. Change-Id: I3c6327c9bc1a924f076ded06afc0afc4e3024384
* Remove cross-project-gating howtoJames E. Blair2021-12-163-27/+2
| | | | | | | This howto has no instructional content. Retaining it may mislead users. Change-Id: I5594bfa3e225ee94c20c9bb8df9ddb3c1483e1bf
* Reorganize docsJames E. Blair2021-12-157-463/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an attempt to reorganize docs based on what we've learned so far: * Audience is important -- help users find the job syntax reference without getting bogged down in how to run zookeeper. * Having distinct tutorials, howtos, and reference documentation is helpful. * Grouping by subject matter is important; users shouldn't have to open tabs with howto, reference, and tutorial to synthesize all the information on a subject. This reorg reduces the use of explicit reference/howto/tutorial/discussion divisions since in some cases they never got sufficiently fleshed out (eg, user tutorials), and in others the information was spread too thinly across them all (eg authentication). However, those distinctions are still useful, and the new organization reflects that somewhat. I have made only some changes to content (generally in introductory sections in order to make things make sense) and added a new "about Zuul" page. We should still go through the documentation and update it and tweak the organization further. This is mostly an attempt to get a new framework in place. The theme is switched from alabaster to RTD. That's because RTD has really good support for a TOC tree in the side bar with expansion. That makes a big difference when trying to navigate large documentation like this. The new framework is intended to have very good left-hand navigation for users. Change-Id: I5ef88536acf1a1e58a07827e06b07d06588ecaf1
* web UI: user login with OpenID ConnectMatthieu Huin2021-11-184-0/+197
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Under the hood, this uses AuthProvider as supplied by oidc-react. Most of the theory is explained in the comment in ZuulAuthProvider.jsx The benefit of doing this is that we allow the AuthProvider and userManager to handle the callback logic, so we don't need to handle the callback logic ourselves. A callback page is still required though in order to deal with the parameters passed in a successful redirection from the Identity Provider. The challenge in using these classes as-is is that our authority endpoints (eg, the IDP itself) may change from one tenant to the next; these classes aren't set up for that. So we need to be careful about how and when we change those authority URLs. In terms of functionalities: if the default realm's authentication driver is set to "OpenIDConnect", display a "Sign in" button. If the the user is logged in, redirect to the last page visited prior to logging in; fetch user authorizations and add them to the redux store; display the user's preferred username in the upper right corner. Clicking on the user icon in the right corner displays a modal with user information such as the user's zuul-client configuration, and a sign out button. Clicking on the sign out button removes user information from the store (note that it does not log the user out from the Identity Provider). Add some basic documentation explaining how to configure Zuul with Google's authentication, and with a Keycloak server. (This squashes https://review.opendev.org/c/zuul/zuul/+/816208 into https://review.opendev.org/c/zuul/zuul/+/734082 ) Co-authored-by: James E. Blair <jim@acmegating.com> Change-Id: I31e71f2795f3f7c4253d0d5b8ed309bfd7d4f98e
* Update Matrix instructionsJames E. Blair2021-08-212-41/+59
| | | | | | This updates the instructions with the new room name. Change-Id: Ice5874d903d2c6338c8f5fd259c9d18cd68a6006
* Split optional matrix howto steps into new docsJames E. Blair2021-07-083-106/+118
| | | | | | | So that the "How to matrix" doc isn't so large, split the optional add-on steps into separate documents. Change-Id: I1ef1c8c0280dccda12fe0fd9c6498d202f769ab5
* Correct URL in matrix howtoJames E. Blair2021-06-141-1/+1
| | | | | | Sphinx did not like auto-parsing this URL; this will correct it. Change-Id: I96a6a5d44e58b28fbe4a56199ef812b387cb460b
* Add Matrix HOWTOJames E. Blair2021-06-122-0/+246
| | | | | | | | Add a HOWTO that walks through setting up a Matrix account with Element and matrix.org, joining #zuul, and some optional additional steps to save encryption keys and register with a 3pid provider. Change-Id: I217c44a7270033998113e434f40a49c2dbb6aaef
* Move key_store_password to keystore section in zuul.confJames E. Blair2021-04-142-2/+6
| | | | | | | | | | | | | | | | | This is likely to be needed by executors as well since passing decrypted secrets to the executors via zookeeper has the same encrypted-at-rest concerns as they keystore itself. To avoid confusion around executors needing a zuul.conf with a scheduler section, start a new keystore section which we can later indicate is used by schedulers and executors. It also makes it convenient to add new options (like those dealing with rotation, or even using an external keystore). Also change some log levels from debug to info where it's useful for the operator to know that the backup keystore was used (or a key was generated). Change-Id: If2491bbe4eb80b76435a274cf5354a4918315e65
* Store secrets keys and SSH keys in ZookeeperSimon Westphahl2021-04-142-0/+2
| | | | | | | | | | | As a preparation for the HA scheduler, project secrets keys and SSH keys will now also be stored in Zookeeper. All private data in Zookeeper will be encrypted at rest. Existing keys on the filesystem will be automatically imported into Zookeeper and new keys will still be available as files for backup. Change-Id: I2a7d1a555f1db1f2178d3bb2f06756ecc8bc7a81
* Resync github permissions docsJames E. Blair2021-03-191-7/+7
| | | | | | We have duplicate github setup instructions and one had bitrotted. Change-Id: Ie38ba581f9e23b40fa243b729f3094742e52844c
* Required SQL reportersJan Kubovy2021-02-031-0/+3
| | | | | | | | | | | | | | | On the way towards a fully scale out scheduler we need to move the times database from the local filesystem into the SQL database. Therefore we need to make at least one SQL connection mandatory. SQL reporters are required (an implied sql reporter is added to every pipeline, explicit sql reporters ignored) Change-Id: I30723f9b320b9f2937cc1d7ff3267519161bc380 Depends-On: https://review.opendev.org/621479 Story: 2007192 Task: 38329
* Doc updates for Github interactionIan Wienand2020-08-201-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | This updates a few things that have come up when using Github with Zuul * The use "github" for the reference pipeline for consistency with everything else. * No need to set the status-url after If5c5fa3c8b33b349d95eb52175e4faf17f5bbb36 * Switch the example pipelines to use the check API, rather than the older status API. Comments are left in the reference pipeline to show both, depending on what a project prefers. * Add the check_run trigger event to examples, which is emitted when a failed run has it's "Re-Run" button pushed in the checks UI page. * Add a note on the Depends-On: format for Github, because if you're used to Gerrit where the Depends-On: works in the change description always you may not realise you have to update the PR description. Change-Id: I049e801c5a36666367db8f9e4fa4cb59b5f08933
* zookeeper-tls-howto: Remove unused keystorepasswordAlbin Vass2020-06-041-1/+0
| | | | Change-Id: I0675216760e1404b5b7e9b0e519264bc8a5312b7
* zookeeper-howto: update to match zk-ca.shAlbin Vass2020-05-281-9/+3
| | | | | | | zk-ca.sh doesn't generate jks files, it generates pem files without any password. Change-Id: Idcd404787f0d47da363f47347d429e814a2c6d3b
* Add release note about the final 3.x releaseJames E. Blair2020-05-271-0/+2
| | | | Change-Id: I8e83d8c40893a395dedb6afcbec3dc86f8be0ac9
* Support dynamic badgesTobias Henkel2020-05-211-0/+7
| | | | | | | | | | Zuul currently has a zuul/gated badge that can be linked e.g. in a readme of a project. This is sufficient for many use cases. However if a project has periodic jobs that do extended testing which is not possible in check/gate this is not sufficient. For use cases like those we can add support for dynamic badges in zuul itself. Change-Id: I449fa9f38ca251522789b6075fbc876d21bd0200
* add installation manual for UbuntuMichiel Piscaer2020-05-164-0/+61
| | | | | | | | Change-Id: I1c66e3fae65f740866202e84fcfce34f5830f155 Use the --user to install bindep Change-Id: I8a1c79a6d591c4e43416f4afc3dba87f78ef612e
* Add TLS support for ZooKeeperJames E. Blair2020-03-182-0/+119
| | | | | | | | | | | | | | | | This adds a script to generate TLS certs for zookeeper. It also adds new config file options for specifying certs for a TLS connection, adds a howto document to advise admins on how to configure ZK for TLS. It also removes the 'required' flag for the SASL auth parameters, since they are not actually required. Include the default openssl.cnf file since some distros modify it to specify paths that are incompatbile with the zk-ca.sh script. Change-Id: Icd976cc32dfd9f75f8cfb1c9ad11e08af31723d6
* Add notes on thread dumping and yappiJeremy Stanley2020-01-181-0/+18
| | | | | | | | | | Copy the bits of the Nodepool Operation and Developer's Guide which talk about SIGUSR2 usage for producing thread dumps and performing profiling. While these are technically development-related topics, they're a useful tool in the admin's troubleshooting belt for filing informative defect reports. Change-Id: I07f78be92d1c77ca1e68affd1852de0e9b8630e3
* Docs: flatten directory structureJames E. Blair2020-01-1717-0/+0
| | | | | | | | | | | | | | | This drops the admins/ and users/ directories, so we are only left with one level of organization (tutorials/howto/reference/discussion). This way as we move things around, we may have slightly fewer redirects to deal with. This does not change the navigation structure or indexes. Note, an alternative would be to drop tutorials/howto/etc and keep admins/users, which was the original orginization, thus requiring even fewer redirects. Change-Id: I580088f5556330b76bc978be79baa0e9a2c552da
* Documentation reorgDavid Shrewsbury2020-01-1417-0/+1595
Reorganizing docs as recommended in: https://www.divio.com/blog/documentation/ This is simply a reorganization of the existing documents and changes no content EXCEPT to correct the location of sphinx doc references. Expect followup changes to change document names (to reflect the new structure) and to move content from existing guides (e.g., to move the pipeline/project/job structure definitions out of the "Project Configuration" reference guide into their own reference documents for easier locatability). All documents are now located in either the "overview", "tutorials", "discussions", or "references" subdirectories to reflect the new structure presented to the user. Code examples and images are moved to "examples" and "images" root-level directories. Developer specific documents are located in the "references/developer" directory. Change-Id: I538ffd7409941c53bf42fe64b7acbc146023c1e3