summaryrefslogtreecommitdiff
path: root/etc
Commit message (Collapse)AuthorAgeFilesLines
* authentication config: add optional max_validity_time, skewMatthieu Huin2019-12-101-0/+2
| | | | | | | | | | | | | | | | | | The Zuul admin can configure authenticators with an optional "max_validity_time" field, which is the maximum age in seconds for a valid authentication token. By default there is no maximum age set for tokens, except the one deduced from the token's "exp" claim. If "max_validity" is set, tokens without an "iat" claim will be rejected. This is meant as an extra security to avoid accidentally issueing very long lived tokens through the CLI. The "skew" field can be used to mitigate clocks discrepancies between Zuul and a JWT emitter. Change-Id: I9351ca016b60050b5f3b3950b840d5f719e919ce
* Add propagate=0 for loggers in logging.conf-sampleVitaliy Lotorev2019-11-141-0/+2
| | | | | | | Otherwise messages from zuul and gerrit loggers are printed twice: from the logger and from root logger. Change-Id: I377758d4b70e250f8b2ba281cd0dabcf12555b62
* Web: plug the authorization enginemhuin2019-07-301-1/+0
| | | | | | | | Add an "authorize_user" RPC call allowing to test a set of claims against the rules of a given tenant. Make zuul-web use this call to authorize access to tenant-scoped privileged actions. Change-Id: I50575f25b6db06f56b231bb47f8ad675febb9d82
* Add Authorization Rules configurationmhuin2019-07-301-0/+1
| | | | | | | | | | Allow an operator to define authorization rules. Allow an operator to add authorization rules to a tenant. Add a rule parser and a rule registry. The authZ engine is not plugged in yet. Change-Id: I3a86c6c7d62ad2bce68a98dbd2fff18549b94fb9
* Zuul CLI: allow access via RESTmhuin2019-07-301-0/+4
| | | | | | | | Users can set the [webclient] section in their zuul.conf file so that the CLI relies on REST calls rather than RPC. The CLI accepts a new --auth-token argument allowing remote users to use privileged REST endpoints. Change-Id: I5a07fccfd787246c4c494db592b575fbdf90ddb1
* web: add tenant and project scoped, JWT-protected actionsMatthieu Huin2019-07-101-0/+9
| | | | | | | | | | | | | | | | A user with the right JSON Web Token (JWT) can trigger a autohold, reenqueue or dequeue a buildset from the web API. The Token is expected to include a key called "zuul.admin" that contains a list of the tenants the user is allowed to perform these actions on. The Token must be passed as a bearer token in an Authorization header. The Token is validated thanks to authenticator declarations in Zuul's configuration file. Change-Id: Ief9088812f44368f14234ddfa25ba872526b8735
* Fix indent error for layout.yaml-sample.Fergus Yu2018-06-201-1/+1
| | | | Change-Id: I01ac226e7873bf9531a221471431e2d5dcf6aed7
* Move SQL web handler to driverJames E. Blair2018-05-311-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The only rest API endpoint that uses sql queries is /api/tenant/{tenant}/builds. There's no connection in there, which means it doesn't make sense for that to be attached to a sql connection (which is currently the case). Moreover, it doesn't make sense for *every* tenant's endpoint to be attached to the *same* connection. In other words, the current situation only allows for a single sql connection system-wide, even if someone is using different connections per tenant. Moving the handler for the endpoint into the sql driver means that it can dispatch the query to the appropriate connection for a given tenant (since a tenant is always implied by the REST endpoint). Moreover, the *rest* of the system actually allows multiple connections within a single tenant, and we should support that here, but I don't immediately have a solution of how to handle pagination across queries that span multiple connections. This is an improvement in that it is now tenant-scoped, but it's not ideal. This also removes the (undocumented!) sql_connection_name config file option. It also uses the tenant name from the path to constructe the query so that it always includes the correct tenant (this eliminates the inadvertant ability for one tenant to query another tenant's builds). The internal API here isn't great, but it will get cleaned up in the next patch which converts to cherrypy. Change-Id: Ie1f19f0b392d4c010ef43dc6220ff1c8667f5a4a
* Merge "mqtt: add basic reporter"Zuul2018-05-311-0/+7
|\
| * mqtt: add basic reporterTristan Cacqueray2018-05-021-0/+7
| | | | | | | | | | | | This change adds a MQTT reporter to publish build results message. Change-Id: I5a9937a7952beac5c77d83ab791d48ff000b447b
* | Add zuul systemd drop-in files for CentOS 7Fatih Degirmenci2018-04-293-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | The existing service files are for Fedora27 and it does not work on CentOS 7 since pip installs zuul-executor, zuul-scheduler, and zuul-web in /usr/bin on CentOS 7. This change adds the systemd drop-in files for this distro so ExecStart and ExecStop commands are overriden by systemd based on where zuul componenst are installed. This change is needed in order to have complete installation instructions for Centos7 in zuul-from-scratch document. Change-Id: Ib9766ba2a825486f05464be6a85968b5158e2e34
* | Fix zuul systemd filesPaul Belanger2018-04-273-6/+3
|/ | | | | | | | | Our current systemd unit files do not work with type=forking, it does work with type=simple. This also means we can drop the PIDFile, as it is only used by forking. Change-Id: Ia9a2e54b91e9b2430c612b2edfa07077f7248539 Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Merge "Add sample systemd service files."Zuul2018-04-163-0/+45
|\
| * Add sample systemd service files.David Shrewsbury2018-04-133-0/+45
| | | | | | | | | | | | | | | | Also modify the Zuul From Scratch doc to reference these new files in the service installation documents. Depends-On: https://review.openstack.org/560993 Change-Id: Ifa0d1cd48ec5e1ef3358ada55b1b2769f392dfe9
* | Remove zuul-cloner commandPaul Belanger2018-04-121-16/+0
|/ | | | | | | | With the release of zuul 3.0.0 is command is no longer needed. Jobs are setup to push the known repo state on to the remove nodes. Change-Id: I0df6e41dc05276e648d393ec62329a85f1b8c415 Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Use yarn and webpack to manage zuul-web javascriptMonty Taylor2018-03-0419-1714/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | yarn drives package and dependency management. webpack handles bundling, minification and transpiling down to browser-acceptable javascript but allows for more modern javascript like import statements. There are some really neat things in the webpack dev server. CSS changes, for instance, get applied immediately without a refresh. Other things, like the jquery plugin do need a refresh, but it's handled just on a file changing. As a followup, we can also consider turning the majority of the status page into a webpack library that other people can depend on as a mechanism for direct use. Things like that haven't been touched because allowing folks to poke at the existing known status page without too many changes using the tools seems like a good way for people to learn/understand the stack. Move things so that the built content gets put into zuul/web/static so that the built-in static serving from zuul-web will/can serve the files. Update MANIFEST.in so that if npm run build:dist is run before the python setup.py sdist, the built html/javascript content will be included in the source tarball. Add a pbr hook so that if yarn is installed, javascript content will be built before the tarball. Add a zuul job with a success url that contains a source_url pointing to the live v3 data. This adds a framework for verifying that we can serve the web app urls and their dependencies for all of the various ways we want to support folks hosting zuul-web. It includes a very simple reverse proxy server for approximating what we do in openstack to "white label" the Zuul service -- that is, hide the multitenancy aspect and present the single tenant at the site root. We can run similar tests without the proxy to ensure the default, multi-tenant view works as well. Add babel transpiling enabling use of ES6 features ECMAScript6 has a bunch of nice things, like block scoped variables, const, template strings and classes. Babel is a javascript transpiler which webpack can use to allow us to write using modern javascript but the resulting code to still work on older browsers. Use the babel-plugin-angularjs-annotate so that angular's dependency injection doesn't get borked by babel's transpiling things (which causes variables to otherwise be renamed in a way that causes angular to not find them) While we're at it, replace our use of var with let (let is the new block-scoped version of var) and toss in some use of const and template strings for good measure. Add StandardJS eslint config for linting JavaScript Standard Style is a code style similar to pep8/flake8. It's being added here not because of the pep8 part, but because the pyflakes equivalent can catch real errors. This uses the babel-eslint parser since we're using Babel to transpile already. This auto-formats the existing code with: npm run format Rather than using StandardJS directly through the 'standard' package, use the standardjs eslint plugin so that we can ignore the camelCase rule (and any other rule that might emerge in the future) Many of under_score/camelCase were fixed in a previous version of the patch. Since the prevailing zuul style is camelCase methods anyway, those fixes were left. That warning has now been disabled. Other things, such as == vs. === and ensuring template strings are in backticks are fixed. Ignore indentation errors for now - we'll fix them at the end of this stack and then remove the exclusion. Add a 'format' npm run target that will run the eslint command with --fix for ease of fixing reported issues. Add a 'lint' npm run target and a 'lint' environment that runs with linting turned to errors. The next patch makes the lint environment more broadly useful. When we run lint, also run the BundleAnalyzerPlugin and set the success-url to the report. Add an angular controller for status and stream page Wrap the status and stream page construction with an angular controller so that all the javascripts can be bundled in a single file. Building the files locally is wonderful and all, but what we really want is to make a tarball that has the built code so that it can be deployed. Put it in the root source dir so that it can be used with the zuul fetch-javascript-tarball role. Also, replace the custom npm job with the new build-javascript-content job which naturally grabs the content we want. Make a 'main.js' file that imports the other three so that we just have a single bundle. Then, add a 'vendor' entry in the common webpack file and use the CommonsChunkPlugin to extract dependencies into their own bundle. A second CommonsChunkPlugin entry pulls out a little bit of metadata that would otherwise cause the main and vendor chunks to change even with no source change. Then add chunkhash into the filename. This way the files themselves can be aggressively cached. This all follows recommendations from https://webpack.js.org/guides/caching/ https://webpack.js.org/guides/code-splitting/ and https://webpack.js.org/guides/output-management/ Change-Id: I2e1230783fe57f1bc3b7818460463df1e659936b Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com> Co-Authored-By: James E. Blair <jeblair@redhat.com>
* Remove .json suffix from web routesMonty Taylor2018-02-192-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | For the existing simple cases, like builds, jobs and status, having the json suffix is a perfectly reasonable thing. However, in the next patch it starts to get weird. When we add support for specific changes or specific jobs, we we grow URLs like: /openstack/status.json /openstack/status/change/537010,2.json Those read weird, because change/537010,2 is much more like an argument or specialiation of status. The thing that reads weird is the status call having .json but the change-specific call just being status/, not the trailing .json on the change url. Removing the json suffix gets us: /openstack/status /openstack/status/change/537010,2 which feels better as the status portion of the url remains consistent. This is done first in the stack so that as we add tests for new endpoints we can get them right the first time rather than having a big rename patch at the end (which is what this started as) Change-Id: I4baf33fdacaf46943fbd192743551bb27bd618de
* Remove webappTobias Henkel2018-01-291-4/+0
| | | | | | | The webapp has been superseeded by zuul-web now so remove it completely. Change-Id: I8125a0d7f3aef8fa7982c75d4650776b6906a612
* Move status_url from webapp to web sectionTobias Henkel2018-01-291-1/+1
| | | | | | | The webapp will be fully replaced by zuul-web so also move the status_url setting there. Change-Id: I8278d9ca81ed7b0a2a2189d42b8b69c5eea2bab5
* Remove large status header and taglineMonty Taylor2017-12-081-2/+0
| | | | | | | | | With the dashboard navbar above, the words "Zuul Status" aren't really neeed. Also, "Real-time status monitor of Zuul, the pipeline manager between Gerrit and Workers." seems like text that's a little out of place now. Change-Id: I5741f1e7edf721eec3a445c054f036a1d17bb172
* web: add /{tenant}/builds routeTristan Cacqueray2017-12-061-0/+1
| | | | | | | | | This change adds a SqlHandler to query the sql reporter database from zuul-web through the /{tenant}/builds.json controller. This change also adds a /{tenant}/builds.html basic web interface. Change-Id: I423a37365316cc96ed07ad0895c7198d9cff8be5
* web: add Cache-Control to static filesTristan Cacqueray2017-11-291-0/+1
| | | | | | This change add the Cache-Control header to static files' response. Change-Id: Ibdf1c35bad378507162d807cf5acdf13fc3fab88
* Add management queue length to status pageTobias Henkel2017-10-192-1/+6
| | | | | | | The status.json now exposes the management queue length. This should also be displayed on the status page. Change-Id: I9dbf34e3b477f82605076be95b4e6c02f253abc7
* Switch statsd config to zuul.confJames E. Blair2017-10-131-0/+3
| | | | | | | | The automatic statsd configuration based on env variables has proven cumbersome and counter-intuitive. Move its configuration into zuul.conf in preparation for other components emitting stats. Change-Id: I3f6b5010d31c05e295f3d70925cac8460d334283
* Avoid JS error when a change has no idJens Harbott2017-10-041-9/+9
| | | | | | | | This may cause the UI to display only a subset of the total changes, leading to some confusion. Use the local variable that we define to be set even when change.id is null consistently. Change-Id: I7c5ff2d9c6ba83e8a8265df3fd83afabe1984fe2
* Status: Don't toggle panel when clicking patch linkTimo Tijhof2017-09-191-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Each patchset panel is clickable in its entirety. Clicks anywhere on the header of the widget propagate to the wrapper element and result in the panel being toggled. This works as expected when clicking on non-interactive areas, such as the title, ETA, background, or progress bar. However, this doesn't work as expected when clicking on the Gerrit patch link, which inadvertently also causes the panel to be toggled. Add logic to the click event handler to ignore the event if it bubbled up from an anchor link. Test Plan: * http://localhost/zuul/etc/status/public_html/?demo=basic * Click on a Gerrit patch link (e.g. "10101,1") in a panel. Previously, it triggers a panel toggle. Now it doesn't. Clicking elsewhere still toggles the panel. Cherry-picked from: Ifadf3dfabfc361e1e7d775a62c4c2643b0e2b2c2 Change-Id: Ifadf3dfabfc361e1e7d775a62c4c2643b0e2b2c2
* Status: Remove use of deprecated jQuery jqXHR `complete` methodTimo Tijhof2017-09-191-2/+2
| | | | | | | | | | | This has been deprecated for a long time, and was removed in jQuery 3.0 per <https://api.jquery.com/jQuery.ajax/#jqXHR> jqXHR.always() has been available since jQuery 1.6. Cherry-picked from: I4498e3b63d07d019db51b4f04a1dd20ebc7c1167 Change-Id: I4498e3b63d07d019db51b4f04a1dd20ebc7c1167
* Revert "Only add changes to status page with jobs"James E. Blair2017-09-051-8/+5
| | | | | | | | | This reverts commit d9f1f82b4e6fb85829c392337c339282d8a3cf96. This change has been confusing because it makes it hard to detect when a change is sitting in a pipeline waiting for a merge. Change-Id: I3cc74db8c13f73d0a609f15a42bbe765a163084b
* Only add changes to status page with jobsMonty Taylor2017-08-301-5/+8
| | | | | | | | | | | | | | In v3, changes get added to pipelines and then it is determined what, if any, jobs they have. This leads to two UX 'issues'. The first is that a change can flash into the check pipeline for a second even if zuul isn't doing anything with it. The second is that for changes that zuul IS doing things with, the change box will appear before it is expandable. If we only append changes that have jobs, then they won't show up in a pipeline if they aren't going to do anything, nor will the user be presented with a box that looks expandable but isn't. Change-Id: I4c39505dbba5c02790132a3f52afad5d68578a2e
* Add gearman server port configurationTristan Cacqueray2017-08-291-0/+2
| | | | | | | This change adds the port configuration option to set a custom port for the gearman server. Change-Id: I1b65f93fa0403ff10e00a97afcdb4a3b512eb372
* Remove zuul_url from merger configTobias Henkel2017-07-311-1/+0
| | | | | | | Currently the zuul_url is not used anywhere but still a required merger setting. This removes it. Change-Id: I627c8a18015f4c148c28d2f7e735b30cc1ef3862
* Change name and document the bind_mount config pathsMonty Taylor2017-07-281-2/+2
| | | | | | | The content in these can be a file or a directory - so _dirs is confusing. Change it to _paths and document it. Change-Id: Ida38766cd3d440d75a6dc55035a54e0804e03760
* Case sensitive label matchingTobias Henkel2017-07-271-8/+8
| | | | | | | | | | | | | | | | | | | | After upgrading Gerrit to 2.13 our gate stopped working. The reason for this is that after a successful gate run zuul does something like 'gerrit review --label verified=2 --submit'. The verified label in Gerrit by default is configured as 'Verified'. The newer version of gerrit behaves different now. It accepts the +2 vote on verified but doesn't submit the patch anymore if the casing is not correct. This forces us to specify the label in the same casing as gerrit expects. In that case the tolower() in canMerge prevents the patch from entering the gate. In order to avoid confusion and be consistent, avoid any case conversions and use the labels exactly as defined in Gerrit. Note that this patch requires changes to the pipelines such that the labels are spelled exactly as defined in Gerrit. Change-Id: I9713a075e07b268e4f2620c0862c128158283c7c
* Display log url when a job completesMonty Taylor2017-07-211-1/+9
| | | | | | | When the job has a result we need to swap to showing the report_url and not the url. Change-Id: Iae9edf906c5dcb692550ea887286bc8a49490d9b
* Fix status_url section in zuul.conf-sampleTristan Cacqueray2017-07-131-3/+1
| | | | | | The status_url needs to be in the [webapp] section Change-Id: I1568d7e6e62682bfca87979fc71db5648f1ed72e
* Merge "Add web-based console log streaming" into feature/zuulv3Jenkins2017-07-101-0/+4
|\
| * Add web-based console log streamingMonty Taylor2017-07-101-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zuul now provides socket-based console streaming, which is super cool. In order to have jenkins parity with web streaming, we need to provide a websocket (javascript in browsers can't really connect to random ports on servers) After surveying the existing python websocket options, basically all of them are based around twisted, eventlet, gevent or asyncio. It's not just a thing we can easily deal with from our current webob/paste structure, because it is a change to the fundamental HTTP handling. While we could write our own websocket server implementation that was threaded like the rest of zuul, that's a pretty giant amount of work. Instead, we can run an async-based server that's just for the websockets, so that we're not all of a sudden putting async code into the rest of zuul and winding up frankensteined. Since this is new code, using asyncio and python3 seems like an excellent starting place. aiohttp supports running a websocket server in a thread. It also supports doing other HTTP/REST calls, so by going aiohttp we can set ourselves up for a single answer for the HTTP tier. In order to keep us from being an open socket relay, we'll expect two parameters as the first message on the websocket - what's the zuul build uuid, and what log file do we want to stream. (the second thing, multiple log files, isn't supported yet by the rest of zuul, but one can imagine a future where we'd like to support that too, so it's in the protocol) The websocket server will then ask zuul over gearman for the IP and port associated with the build and logfile and will start streaming it to the socket. Ultimately we'll want the status page to make links of the form: /console.html?uuid=<uuid>&logfile=console.log and we'll want to have apache map the websocket server to something like /console. Co-Authored-By: Monty Taylor <mordred@inaugust.com> Change-Id: Idd0d3f9259e81fa9a60d7540664ce8d5ad2c298f
* | Move zookeeper_hosts to zookeeper sectionJames E. Blair2017-07-051-1/+3
| | | | | | | | | | | | | | | | | | To anticipate future use of zookeeper by more than just the scheduler, create a new 'zookeeper' section in zuul.conf similar to the 'gearman' section. The 'hosts' entry is the only setting there now, but others may be added later. Change-Id: Ib0bb2dfdc96fa0eb4aad2b4700de2f2004ea15f2
* | Correct sample zuul.confJames E. Blair2017-07-051-3/+5
|/ | | | | | And add missing zuul.conf config entries. Change-Id: I776826e7ff1fff2edd6f6d332c20a24af1a4eb58
* executor: run trusted playbook in a bubblewrapTristan Cacqueray2017-06-171-0/+2
| | | | | | | | | | | | | | This change renames untrusted_wrapper to execution_wrapper and uses bubblewrap for both trusted and untrusted playbooks by default. This change adds new options to the zuul.conf executor section to let operators define what directories to mount ro or rw for both context: * trusted_ro_dirs/trusted_rw_dirs, and * untrusted_ro_dirs/untrusted_rw_dirs Change-Id: I9a8a74a338a8a837913db5e2effeef1bd949a49c Story: 2001070 Task: 4687
* Add ssl support to gearman / gearman_serverPaul Belanger2017-06-141-0/+6
| | | | | | | | | Enable SSL support for gearman. We also created an new SSLZuulBaseTest class to provide a simple way to use SSL end to end where possible. A future patch will enable support in zookeeper. Change-Id: Ia8b89bab475d758cc6a021988f8d79ead8836a9d Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Merge "Re-add the ability to set username on zuul-executor" into feature/zuulv3Jenkins2017-06-071-0/+3
|\
| * Re-add the ability to set username on zuul-executorJamie Lennox2017-04-191-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes we need to log in to a nodepool node using a username of something other than zuul. This used to be possible by setting that [launcher] username= property. Re-enable it with the [executor] default_username= property. default_username is used instead of simply username as it is likely in future this information will be able to be supplied by nodepool or other in a node or image specific way. At which time that information will be used in priority to the default specified in zuul. Change-Id: Icf657b4f0bbe34e182307b9eea0cd64a8d813464
* | Merge "Fix bad text wrap in status page" into feature/zuulv3Jenkins2017-06-011-2/+1
|\ \
| * | Fix bad text wrap in status pageMonty Taylor2017-05-261-2/+1
| | | | | | | | | | | | Change-Id: I63fa6c25aff15a016b57924963971dacbe48d010
* | | Represent github change ID in status page by PR numberJamie Lennox2017-05-231-1/+10
|/ / | | | | | | | | | | | | | | | | | | | | | | The change.id given to the status.json output for github PRs is a full "PR#,SHA" combo. This gives it uniqueness when we can have multiple commits and versions of commits per PR. When we display this in the status page though i think all we care about is the PR number, so trim it out and just display that. Change-Id: I73385a6b0743a6764fdfdd51cd1d8ee02b71c53c Signed-off-by: Jamie Lennox <jamielennox@gmail.com>
* | Add javascript license informationTristan Cacqueray2017-05-213-0/+15
| | | | | | | | | | | | | | This change adds javascript license information according to: https://www.gnu.org/software/librejs/free-your-javascript.html Change-Id: Iaf9c48e1edfea8f3043a4056ce2d76460b3a42d9
* | Set filter according to PR/Change in URLJan Hruban2017-05-111-0/+4
|/ | | | | | | | | | E.g. https://zuul-server/#333 will set filter to 333 Introduce related configuration option And always pass the full change url in github status payloads Allow the status_url to be used in a pipeline start-message. Change-Id: I7b9eccb1d218d65d8c6b01e042e6cce4f8aec89a
* Remove more swift configurationsJames E. Blair2017-03-171-9/+0
| | | | | | These slipped through the earlier swift removal. Change-Id: Iecd71563e1904f3bb8028fdafb00b10d95bc8ab2
* Merge branch 'master' into feature/zuulv3Joshua Hesketh2017-03-062-4/+7
|\ | | | | | | Change-Id: I37a3c5d4f12917b111b7eb624f8b68689687ebc4
View Lorry Controller Status