| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
| |
Resolve the following warnings:
UserWarning: Usage of dash-separated 'description-file' will not be
supported in future versions. Please use the underscore name
'description_file' instead
Change-Id: Ibd13b66a1283714c2ceaecdfc0e1de7a5338df5f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This adds python3.10 testing on Jammy and switches the docker images to
python3.10 from 3.8.
We run sudo for postgres with -Hi to avoid non fatal errors when
postres' client attempts to write command history to Zuul's homedir (it
is running as the postgres user which can't write to zuul's homedir). We
also need to update the libffi package version for jammy to 8 in
bindep.txt. Finally, python_version values need to be quoted as "3.10"
is different than 3.10 which is equivalent to 3.1 when serialized by
yaml as a float.
Force setuptools to use stdlib (shipped by the distro) distutils to
avoid problems with virtualenvs not actually being virtualenvs.
Finally we switch the bulk of jobs over to using nodeset: ubuntu-jammy
as the default python there is 3.10.
Change-Id: I97b90bb7a23c90f108f23dda9fdd0e89f9f4dbca
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We have two CLIs: zuul-client for REST-related operations, which cover
tenant-scoped, workflow modifying actions such as enqueue, dequeue and
promote; and zuul which supercedes zuul-client and covers also true admin
operations like ZooKeeper maintenance, config checking and issueing auth tokens.
This is a bit confusing for users and operators, and can induce code
duplication.
* Rename zuul CLI into zuul-admin. zuul is still a valid endpoint
and will be removed after next release.
* Print a deprecation warning when invoking the admin CLI as zuul
instead of zuul-admin, and when running autohold-*, enqueue-*,
dequeue and promote subcommands. These subcommands will need to be
run with zuul-client after next release.
* Clarify the scopes and deprecations in the documentation.
Change-Id: I90cf6f2be4e4c8180ad0f5e2696b7eaa7380b411
|
| |/
|
|
|
|
|
| |
We actually reqire 3.8 or newer now so ensure that is reflected in the
package metadata.
Change-Id: Ia6ef56729b4ca3e93a694bdf6ba375ae4c34e86a
|
| |
|
|
|
|
|
| |
Python 3.6 is EOL and Ansible 5 does not support it. Bump our
testing to 3.8 + 3.9 to match Nodepool.
Change-Id: Ica6094c05afb87cd5bbc8752c6362d89c13eca0a
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
An SQL database is an additional requirement so we should also include
the client libraries for it.
Change-Id: Id3a8b55928d9ce9e8764f18a64a42bb2f14f5ecb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |/
|
|
|
|
|
|
| |
This adds the python-logstash-async module to the list of packages
that should be present in container images. This allows operators
to configure direct-to-logstash logging from Zuul.
Change-Id: I6764876c7e64afa94dcdf8f49215cf7abc607a57
|
| |
|
|
|
|
|
|
|
| |
This was intended as a one-time helper program to help people upgrade
from Zuul v2 to v3. It did not cover all use cases, and has not been
kept up to date or improved. It's time to remove it before the v4
release.
Change-Id: I12cdcedb5baabd8fa0937a6ea21590259093ead1
|
| |
|
|
|
|
|
|
|
| |
This allows mypy to produce the same results regardless how is called.
For example IDEs will produce the same results as tox -e linters
instead of displaying violations which the core team decided to
disable.
Change-Id: I20eeafc0549f78e3d4f9636793cfe031389b6bbe
|
| |
|
|
|
|
|
| |
The currently used version 0.740 throws syntax errors when used with
python 3.9.
Change-Id: If5cd0da2d5a77126e9ac4c4d6db7a6552f1bd067
|
| |
|
|
|
|
|
|
| |
There are now increasing numbers of dependencies like cryptocraphy
that will drop support for Python 3.5 soon. Thus we should consider
dropping the support for Python 3.5 as well.
Change-Id: I830ec3e97cfaac5d336d02755fe788572f36fb07
|
| |
|
|
|
|
|
| |
If people want to do debugging of scheduler issues, these
are useful to have installed.
Change-Id: I93e949f75d38cc86c799dc52a1b939e61e81d74b
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Replace our tox-py37 job with tox-py38, extend the list of trove
classifiers for Python versions in package metadata, and replace the
"py35" in the tox.ini envlist with just "py3" so that folks running
`tox` unqualified on their systems will use whatever python3
interpreter they have on hand (odds are it's in our supported range
these days). Also uncap python-daemon so we use a version compatible
with Python >=3.8.
Change-Id: I4b382bb89847d3ae2b20292d923c3c2211dfb254
|
| |
|
|
|
|
|
| |
Ara and openstacksdk are installed into the ansible virtualenvs now so
there is no need to pull them as executor specific dependencies.
Change-Id: I3aeac278b8e1f4bb796e0f3a3f475b3c97208a85
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As a first step towards supporting multiple ansible versions we need
tooling to manage ansible installations. This moves the installation
of ansible from the requirements.txt into zuul. This is called as a
setup hook to install the ansible versions into
<prefix>/lib/zuul/ansible. Further this tooling abstracts knowledge
that the executor must know in order to actually run the correct
version of ansible.
The actual usage of multiple ansible versions will be done in
follow-ups.
For better maintainability the ansible plugins live in
zuul/ansible/base where plugins can be kept in different versions if
necessary. For each supported ansible version there is a specific
folder that symlinks the according plugins.
Change-Id: I5ce1385245c76818777aa34230786a9dbaf723e5
Depends-On: https://review.openstack.org/623927
|
| |
|
|
|
|
|
|
| |
This adds "Requires-Python: >=3.5" to the PKG-INFO file, which will then
in turn be processed by pypi and pip and prevent users on python2 from
being able to pip install zuul.
Change-Id: Ia2fe96bd151faf4f0c15be79872d4ddf9505b771
|
| |
|
|
|
|
| |
Helps to name the option correctly.
Change-Id: Iedd0bdda37a6a6e14c327e1a7a914ec554a489d8
|
| |
|
|
|
|
|
| |
So that the pbrx image builds install openstacksdk on the executors
so that it can be used by the swift logs role in zuul-jobs.
Change-Id: Ie775169e9230108f7f76ef31953171f5a4881a69
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The pbrx project has a command to build container images based on the
python:alpine base image for any project using pbr and bindep.
Use it to start building container images for zuul. A followup patch
will add support for publishing the images to dockerhub.
The images want the database drivers in them to be useful out of the
box. Also, the executor wants ara installed. By adding those to extra
entries named after the resulting image name, pbrx will install the
extra packages at the right time, as well as produce wheels for them
during the wheel cache stage.
Depends-On: https://review.openstack.org/583047
Change-Id: I453b0ab2c4916cb77249b5f1039e6e18fde47ab1
|
| |
|
|
|
|
|
|
| |
With the release of zuul 3.0.0 is command is no longer needed. Jobs
are setup to push the known repo state on to the remove nodes.
Change-Id: I0df6e41dc05276e648d393ec62329a85f1b8c415
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |
|
|
|
|
|
|
| |
Refresh our metadata for pypi to use our shiny new https://zuul-ci.org
domain.
Change-Id: I5fdcf86c716f869840265aa40b1046bbb9f1b75e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |
|
|
|
|
|
|
| |
Noticed we haven't updated this information to point to new URL for
documentation or summary.
Change-Id: I4594876888de9aef2259ab9c324fa0b7ed2ebaaa
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
yarn drives package and dependency management. webpack handles
bundling, minification and transpiling down to browser-acceptable
javascript but allows for more modern javascript like import statements.
There are some really neat things in the webpack dev server. CSS
changes, for instance, get applied immediately without a refresh. Other
things, like the jquery plugin do need a refresh, but it's handled just
on a file changing.
As a followup, we can also consider turning the majority of the status page
into a webpack library that other people can depend on as a mechanism
for direct use. Things like that haven't been touched because allowing
folks to poke at the existing known status page without too many changes
using the tools seems like a good way for people to learn/understand the
stack.
Move things so that the built content gets put
into zuul/web/static so that the built-in static serving from zuul-web
will/can serve the files.
Update MANIFEST.in so that if npm run build:dist is run before the
python setup.py sdist, the built html/javascript content will be
included in the source tarball.
Add a pbr hook so that if yarn is installed, javascript content will be
built before the tarball.
Add a zuul job with a success url that contains a source_url
pointing to the live v3 data.
This adds a framework for verifying that we can serve the web app
urls and their dependencies for all of the various ways we want to
support folks hosting zuul-web.
It includes a very simple reverse proxy server for approximating
what we do in openstack to "white label" the Zuul service -- that
is, hide the multitenancy aspect and present the single tenant
at the site root.
We can run similar tests without the proxy to ensure the default,
multi-tenant view works as well.
Add babel transpiling enabling use of ES6 features
ECMAScript6 has a bunch of nice things, like block scoped variables,
const, template strings and classes. Babel is a javascript transpiler
which webpack can use to allow us to write using modern javascript but
the resulting code to still work on older browsers.
Use the babel-plugin-angularjs-annotate so that angular's dependency
injection doesn't get borked by babel's transpiling things (which causes
variables to otherwise be renamed in a way that causes angular to not
find them)
While we're at it, replace our use of var with let (let is the new
block-scoped version of var) and toss in some use of const and template
strings for good measure.
Add StandardJS eslint config for linting
JavaScript Standard Style is a code style similar to pep8/flake8. It's
being added here not because of the pep8 part, but because the pyflakes
equivalent can catch real errors. This uses the babel-eslint parser
since we're using Babel to transpile already.
This auto-formats the existing code with:
npm run format
Rather than using StandardJS directly through the 'standard' package,
use the standardjs eslint plugin so that we can ignore the camelCase
rule (and any other rule that might emerge in the future)
Many of under_score/camelCase were fixed in a previous version of the patch.
Since the prevailing zuul style is camelCase methods anyway, those fixes
were left. That warning has now been disabled.
Other things, such as == vs. === and ensuring template
strings are in backticks are fixed.
Ignore indentation errors for now - we'll fix them at the end of this
stack and then remove the exclusion.
Add a 'format' npm run target that will run the eslint command with
--fix for ease of fixing reported issues.
Add a 'lint' npm run target and a 'lint' environment that runs with
linting turned to errors. The next patch makes the lint environment more
broadly useful.
When we run lint, also run the BundleAnalyzerPlugin and set the
success-url to the report.
Add an angular controller for status and stream page
Wrap the status and stream page construction with an angular controller
so that all the javascripts can be bundled in a single file.
Building the files locally is wonderful and all, but what we really want
is to make a tarball that has the built code so that it can be deployed.
Put it in the root source dir so that it can be used with the zuul
fetch-javascript-tarball role.
Also, replace the custom npm job with the new build-javascript-content
job which naturally grabs the content we want.
Make a 'main.js' file that imports the other three so that we just have
a single bundle. Then, add a 'vendor' entry in the common webpack file
and use the CommonsChunkPlugin to extract dependencies into their own
bundle. A second CommonsChunkPlugin entry pulls out a little bit of
metadata that would otherwise cause the main and vendor chunks to change
even with no source change. Then add chunkhash into the filename. This
way the files themselves can be aggressively cached.
This all follows recommendations from https://webpack.js.org/guides/caching/
https://webpack.js.org/guides/code-splitting/ and
https://webpack.js.org/guides/output-management/
Change-Id: I2e1230783fe57f1bc3b7818460463df1e659936b
Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com>
Co-Authored-By: James E. Blair <jeblair@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This adds the zuul-fingergw app that should be run as root (so that
it can connect to the standard finger port 79), but changes user privs
immediately after binding that port.
Common streaming functions have been moved to streamer_utils.py to
be shared among modules.
Support for CommandSocket has been included.
Change-Id: Ia35492fe951e7b9367eeab0b145d96189d72c364
|
| |
|
|
|
|
|
|
| |
We need to be able to generate a new v3 config based on our old v2
config. It won't be perfect, but should ultimately be no worse than the
v2.5 auto-generated playbooks.
Change-Id: I9cb676ceff01bbdb845d22774edbaa718323db27
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
zuul now provides socket-based console streaming, which is super cool.
In order to have jenkins parity with web streaming, we need to provide a
websocket (javascript in browsers can't really connect to random ports
on servers)
After surveying the existing python websocket options, basically all of
them are based around twisted, eventlet, gevent or asyncio. It's not
just a thing we can easily deal with from our current webob/paste
structure, because it is a change to the fundamental HTTP handling.
While we could write our own websocket server implementation that was
threaded like the rest of zuul, that's a pretty giant amount of work.
Instead, we can run an async-based server that's just for the
websockets, so that we're not all of a sudden putting async code into
the rest of zuul and winding up frankensteined. Since this is new code,
using asyncio and python3 seems like an excellent starting place.
aiohttp supports running a websocket server in a thread. It also
supports doing other HTTP/REST calls, so by going aiohttp we can set
ourselves up for a single answer for the HTTP tier.
In order to keep us from being an open socket relay, we'll expect two
parameters as the first message on the websocket - what's the zuul build
uuid, and what log file do we want to stream. (the second thing,
multiple log files, isn't supported yet by the rest of zuul, but one can
imagine a future where we'd like to support that too, so it's in the
protocol) The websocket server will then ask zuul over gearman for the
IP and port associated with the build and logfile and will start
streaming it to the socket.
Ultimately we'll want the status page to make links of the form:
/console.html?uuid=<uuid>&logfile=console.log
and we'll want to have apache map the websocket server to something like
/console.
Co-Authored-By: Monty Taylor <mordred@inaugust.com>
Change-Id: Idd0d3f9259e81fa9a60d7540664ce8d5ad2c298f
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
OpenStack Infra is running Zuul v3 on Python3. We're adding things, like
the websockets streaming, that are python3 only - and testing them turns
out to be extra hard even with skipTest because of the syntax changes.
There are no legacy Zuul v3 installations to worry about, and the
Software Factory team has verified that Python3 on CentOS 7 is not a
problem. It's time to pull the plug.
Remove python2 jobs from running in v3. Make the tox defaults python3
for all testenvs and update setup.cfg to indicate that Zuul v3 supports
python3 and not python2.
Also fixed a python3 bug in zuul command line client that shows up
when running zuul --help under python3.
Change-Id: Icaccab9f6edcdba0b87ec0b5f0411d3dc8030b42
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will be the minimum "batteries included" bubblwrap driver. It does
not do any MAC configuration, since these vary by system. Operators
may wish to wrap it further in a MAC wrapper driver.
Because we set bubblewrap as the default wrapper, test_playbooks tests
it. However, it lacks a negative test, so we won't know if we're not
actually containing things.
Users who don't have bubblewrap or don't wish to use it can set the
untrusted_wrapper to 'nullwrap' which will just execute things as
they're done before this change.
Change-Id: I84dd7c8cc55d2110b58609784007ffda0d135716
Story: 2000910
Task: 3540
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The docs builds throw quite a few exceptions and warnings. Most of this
is due to code that has moved around so autodoc can't import code to
document that code. Update the names for code that has moved. There were
also so small docstring indentation issues that are corrected.
In addition to fixing these issues, update the minimum version of sphinx
and set warning-is-error so that we avoid these problems in the future
by gating on making sure they don't exist.
Change-Id: Ic06775e7d1c8fbbfc5a8e6c412a2ad15f8699413
|
| |
|
|
|
|
|
|
| |
To avoid confusion with nodepool-launcher, we've decided to rename
zuul-launcher to zuul-executor.
Change-Id: I7d03cf0f0093400f4ba2e4beb1c92694224a3e8c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |\
| |
| |
| | |
Change-Id: I37a3c5d4f12917b111b7eb624f8b68689687ebc4
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This will allow us to enter results from all jobs for
use with the openstack-health dashboard.
Depends-On: I08dbbb64b3daba915a94e455f75eef61ab392852
Change-Id: I28056d84a3f6abcd8d9038a91a6c9a3902142f90
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |/
|
|
|
|
| |
This is what we call it.
Change-Id: If4ae79eddff9151979271ae7faf4348014528e46
|
| |
|
|
|
|
|
| |
As part of the Zuulv2.5 effort, this service is not designed to be
long-lived.
Change-Id: Idbac79f60cb694a9a707d204cd274328289bc20d
|
| |
|
|
|
|
|
|
|
| |
The http://ci.openstack.org/ documentation site has been deprecated,
replaced by redirects to corresponding paths within
http://docs.openstack.org/infra/ where other Project Infrastructure
documentation already resides.
Change-Id: I765c42294c478d88076959462c5316a620a4f24a
|
| |
|
|
|
|
|
| |
The intent is to replace the devstack shell script with a python utility
which would be easy to reuse.
Change-Id: I3c8748e2544af80e72fdd0b2e3627b4fc2212c01
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Connect it to Zuul via Gearman. Any number of mergers may be
deployed.
Directly find the pipeline for a build when processing a result,
so that the procedure is roughly the same for build and merge
results.
The timer trigger currently requires the gerrit trigger also be
configured. Make that explicit inside of the timer trigger so
that the scheduler API interaction with triggers is cleaner.
Change-Id: I69498813764753c97c426e42d17596c2ef1d87cf
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a command line client called 'zuul' that supports one command
to start with: 'enqueue'. It allows an operator (one with access
to the gearman server) to enqueue an arbitrary change in a specified
pipeline. It uses gearman to communicate with the Zuul server, which
now has an added RPC listener component to answer such requests via
gearman.
Add tests for the client RPC interface.
Raise an exception if a Gerrit query does not produce a change. Unlike
events from Gerrit, user (or admin) submitted events over the RPC bus
are more likely to reference invalid changes. To validate those, the
Gerrit trigger will raise an exception (and remove from its cache) changes
which prove to be invalid.
Change-Id: Ife07683a736c15f4db44a0f9881f3f71b78716b2
|
| |
|
|
|
|
|
| |
d2to1 pulls in setuptools, which trips the unhappy bugs with setuptools
updating. Move past that and just use new pbr.
Change-Id: I2609eda10ed781116940c3607ff85a14fc4b7f58
|
| |
|
|
|
|
|
|
| |
With this change, we now should properly gate on Sphinx errors /
warnings.
Change-Id: I7e789ad83e81286575c74e1e5256da20a717e982
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Needed to move some directory creation around to be contained within the
testcase, but with parallel testing, we shave 48 seconds off the run
time and go from around 60s to around 11. We're also now compatible with
future subunit-based fast-fail semantics when we grow them.
Change-Id: I6c7148c29d1edb5d9469a8c2afe4b31b2b340009
Reviewed-on: https://review.openstack.org/33352
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
|
| |
|
|
|
|
|
|
|
| |
Change-Id: I734f0f8237fb603ee41a39f06e63c007e79825a9
Reviewed-on: https://review.openstack.org/33350
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Doug Hellmann <doug.hellmann@dreamhost.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
|
| |
|
|
|
|
|
|
|
|
|
| |
We are not concerned with the code coverage of upstream libraries that
are used in zuul. Restrict code coverage gathering to the zuul package.
Change-Id: I62eb752c5fd0fe0289253c0b5c950382d760ea95
Reviewed-on: https://review.openstack.org/12725
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
|
|
|
Change-Id: I8197ec2e52596fa4136f8af9aa93ea06e56d4d0d
|
