| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The reason for this is that containers for zuul services need to run
privileged in order to successfully run bwrap. We currently only expect
users to run the executor as privilged and the new bwrap execution
checks have broken other services as a result. (Other services load the
bwrap system bceause it is a normal zuul driver and all drivers are
loaded by all services).
This works around this by add a check_bwrap flag to connection setup and
only setting it to true on the executor. A better longer term followup
fixup would be to only instantiate the bwrap driver on the executor in
the first place. This can probably be accomplished by overriding the
ZuulApp configure_connections method in the executor and dropping bwrap
creation in ZuulApp.
Temporarily stop running the quick-start job since it's apparently not
using speculative images.
Change-Id: Ibadac0450e2879ef1ccc4b308ebd65de6e5a75ab
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Newer bwrap has added the ability to disable additional nested user
namespace creation from with the bwrap execution context. Take advantage
of this feature in Zuul if we are able to in order to fortify Zuul's
security position.
In particular we need two conditions to take advantage of this. 1) bwrap
must be new enough to support the feature (>=0.8.0) and 2) we must be
running with user namespaces enabled. We explicitly check for both
conditions and add the appropriate invocation flags to bwrap when the
conditions are met.
Change-Id: Idf933a0847cb8570b551892186ca9c0057be127f
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A new version of pyjwt was released which alters an internal call
it makes to urllib which we have mocked in the unit tests. Our
mock must be updated to match.
The previous version would call urlopen with a string url argument,
newer versions call it with a Request object (urllib accepts both).
This change updates the mock to accept both.
There has also been a recent alembic release which alters the function
signature of alter_column so that most arguments are now required to
be keyword arguments. This causes one of our migrations to fail
since it was not passing the new type in as a positional argument.
Notably, the type was not the next positional argument in the
signature ("nullable" is), so this explains why we had two
"change patchset to string" migrations: the first one did not actually
work.
This change alters the migration to do what it effectively did rather
that what it was intended to do. The later migration continues to
correct the error.
Change-Id: I10cdffa43147f7f72e431e7c64e10e9416dfb295
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The SQL queries are designed to be highly optimized and should return
in milliseconds even with millions of rows. However, sometimes
query planners are misled by certain characteristics and can end
up performing suboptimally.
To protect the web server in case that happens, set a statement or
query timeout for the queries which list builds or buildsets. This
will instruct mysql or postgresql to limit execution of the buildset
or build listing queries to 30 seconds -- but only if these queries
originate in zuul-web. Other users (such as the admin tools) may
still run these queries without an explicit time limit (though the
server may still have one).
Unfortunately (or perhaps fortunately) the RDBMSs can occasionally
satisfy the queries we use in testing in less than 1ms, making a
functional test of this feature impractical (we are unable to set
the timeout to 0ms).
Change-Id: If2f01b33dc679ab7cf952a4fbf095a1f3b6e4faf
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This adds a theme selection in the preferences in the
config modal and adds a new dark theme.
Removes the line.png image and instead uses CSS
linear-gradient that is available in all browsers
since around 2018, also fixes the 15 pixels spacing
issue that is there today.
You can select between three different themes.
Auto will use your system preference to choose either the
light or dark theme, changes dynamically based on your
system preference.
Light is the current theme.
Dark is the theme added by this patch series.
The UX this changes is that if somebody has their system
preferences set to dark, for example in Mac OS X that is
in System Settings -> Appearance -> Dark the user will
get the Zuul web UI in dark by default and same for the
opposite.
This uses a poor man's dark mode for swagger-ui
as per the comment in [1].
[1] https://github.com/swagger-api/swagger-ui/issues/5327#issuecomment-742375520
Change-Id: I01cf32f3decdb885307a76eb79d644667bbbf9a3
|
| |\ \ \ |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This updates the Gerrit driver to match the pattern in the GitHub
driver where instead of specifying individual trigger
requirements such as "require-approvals", instead a complete ref
filter (a la "requirements") can be embedded in the trigger
filter.
The "require-approvals" and "reject-approvals" attributes are
deprecated in favor of the new approach.
Additionally, all require filters in Gerrit are now available as
reject filters.
And finally, the Gerrit filters are updated to return
FalseWithReason so that log messages are more useful, and the
Github filters are updated to improve the language, avoid
apostraphes for ease of grepping, and match the new Gerrit
filters.
Change-Id: Ia9c749f1c8e318fe01e84e52831a9d0d2c10b203
|
| |\ \ \ \
| |/ / / |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This mimics a useful feature of the Gerrit driver and allows users
to configure pipelines that trigger on events but only if certain
conditions of the PR are met.
Unlike the Gerrit driver, this embeds the entire require/reject
filter within the trigger filter (the trigger filter has-a require
or reject filter). This makes the code simpler and is easier for
users to configure. If we like this approach, we should migrate the
gerrit driver as well, and perhaps the other drivers.
The "require-status" attribute already existed, but was undocumented.
This documents it, adds backwards-compat handling for it, and
deprecates it.
Some documentation typos are also corrected.
Change-Id: I4b6dd8c970691b1e74ffd5a96c2be4b8075f1a87
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This adds extra bits of info that the ensure-quay-repo role expects to
be present.
Change-Id: I581c667515b5ee2eec4ee79e9f2b485247711195
|
| |\ \ \ \ |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Systems with very large numbers of job variants can suffer slow
performance due to inefficiencies in layout generation. This change
updates two locations to be more efficient:
1) When adding a job variant, we check to make sure that the variant
we are adding is from the same repo as all the previous variants.
Due to the transitive property of this check, we only need to compare
the new variant to one of the prior variants, not all of them.
2) When validating references on a job variant, we created 3 list
comprehensions with all of the variant's parents which matched certain
criteria. Since most variants match all three criteria, if we
short-circuit this check as soon as all three criteria have been
satisfied, we can complete the process much sooner.
Change-Id: Iaa9cd663581b3b888fb1ee95c0c25a3484d3e828
|
| |\ \ \ \ \ |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
The test helper method that handles fake gerrit queries had a bug
which would cause the "topic:" queries to return all open changes.
When we correct that, we can see, by virtue of a newly raised
execption that there was some unexercised code in getChangesByTopic
which is now exercised. This change also corrects the exception
that is raised when mutating a set while iterating over it.
Change-Id: I1874482b2c28fd1082fcd56036afb20333232409
|
| |\ \ \ \ \ \
| |/ / / / / |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Before getting a change by key we need to convert the reference to a
`ChangeKey` object.
Traceback (most recent call last):
File "/opt/src/zuul/zuul/scheduler.py", line 2423, in process_tenant_trigger_queue
self._forward_trigger_event(event, tenant)
File "/opt/src/zuul/zuul/scheduler.py", line 2538, in _forward_trigger_event
pipeline.manager.updateCommitDependencies(change, event)
File "/opt/src/zuul/zuul/manager/__init__.py", line 924, in updateCommitDependencies
for dep in source.getChangesByTopic(change.topic):
File "/opt/src/zuul/zuul/driver/gerrit/gerritsource.py", line 171, in getChangesByTopic
git_change = self.getChange(git_key)
File "/opt/src/zuul/zuul/driver/gerrit/gerritsource.py", line 86, in getChange
return self.connection.getChange(change_key, refresh=refresh,
File "/opt/src/zuul/zuul/driver/gerrit/gerritconnection.py", line 791, in getChange
if change_key.connection_name != self.connection_name:
AttributeError: 'str' object has no attribute 'connection_name'
Change-Id: I20663b538ccb48bc13191a134caf3326b6f5b76e
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | | |
Change-Id: I142c99a5469fd5af5e558619cd9a4acc400221ff
|
| |/ / / / /
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
If a commit has no parents (e.g. the first commit in a repository), the
HTTP request to gerrit to the revision files endpoint must not specify
the parent=1 query parameter. Otherwise, gerrit will return an HTTP 400
error with the message 'invalid parent number: 1'.
Explicitly check for this by examining if any parents are present for
the current revision before adding the query parameter
Change-Id: I3f93899dc79ebc716223ecc6f5adecd0f1ce9b3e
|
| |\ \ \ \ \ |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
From the previous log messages related to a cat job it wasn't clear
which HEAD SHA was used to get the file content.
This change adds a log message to the `getFiles()` method that contains
the HEAD commit SHA.
Change-Id: I02a3a97f9b3dfa70f6e55954ea6ef365289f0046
|
| |\ \ \ \ \ \ |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
This reverts commit 1c188ad440e82bb356832497c387aace71653dc8.
Reason for revert: This change increases test failures; investigation is ongoing.
Change-Id: I3a101656ee0904e1cf6c856dba845385c333260f
|
| |\ \ \ \ \ \ \
| |/ / / / / / |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
I'm not sure if this is a good idea or not yet. Testing with MySQL
actually exposed the alembic transaction thing that is fixed in a
grandparent change. That said much of the world seems to be moving to
MariaDB so it might be a good idea for us to test that more explicitly?
We update bindep.txt which will install mariadb during CI job runs. We
also update our local test setup tooling to roughly match what will be
used in CI. For example the mariadb:10.6 docker image is chosen because
this is roughly equivalent to what Jammy packages.
Another good reason for this change is that Debian ARM64 does not
package MySQL and only packages MariaDB. This should give us better
compatibility for testing on architectures other than x86-64.
Change-Id: I9d00557ca5823da34278c3e9ecda2daaf13440b7
|
| |/ / / / / /
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
When a user sets zuul_console_disabled, we don't need to try to
connect to the streaming daemon. In fact, they may have set it
because they know it won't be running. Check for this and avoid
the connection step in that case and therefore avoid the extraneous
"Waiting on logger" messages and extra 30 second delay at the end
of each task.
Change-Id: I86af231f1ca1c5b54b21daae29387a8798190a58
|
| |\ \ \ \ \ \ |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
With newer versions of Gerrit, we are increasingly likely to encounter
systems where the traditional label requirements are minimized in favor
of the new submit requirements rules. If Gerrit is configured to use
a submit requirement instead of a traditional label blocking rule, that
is typically done by switching the label function to "NoBlock", which,
like the "NoOp" function, will still cause the label to appear in the
"submit_record" field, but with a value of "MAY" instead of "OK", "NEED",
or "REJECT".
Instead, the interesting information will be in the "submit_requirements"
field. In this field we can see the individual submit requirement rules
and whether they are satisfied or not.
Since submit requirements do not have a 1:1 mapping with labels, determining
whether an "UNSATISFIED" submit requirement should be ignored (because it
pertains to a label that Zuul will alter, like "Verified") is not as
straightforward is it is for submit records. To be conservative, this
change looks for any of the "allow needs" labels (typically "Verified") in
each unsatisfied submit record and if it finds one, it ignores that record.
With this change in place, we can avoid enqueing changes which we are certain
can not be merged into gate pipelines, and will continue to enqueue changes
about which we are uncertain.
Change-Id: I667181565684d97c1d036e2db6193dc606c76c57
|
| |\ \ \ \ \ \ \
| |/ / / / / /
|/| | | | | | |
|
| | | |/ / / /
| |/| | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
This command had two problems:
* It would only delete the first 50 buildsets
* Depending on DB configuration, it may not have deleted anything
or left orphan data.
We did not tell sqlalchemy to cascade delete operations, meaning that
when we deleted the buildset, we didn't delete anything else.
If the database enforces foreign keys (innodb, psql) then the command
would have failed. If it doesn't (myisam) then it would have deleted
the buildset rows but not anything else.
The tests use myisam, so they ran without error and without deleting
the builds. They check that the builds are deleted, but only through
the ORM via a joined load with the buildsets, and since the buildsets
are gone, the builds weren't returned.
To address this shortcoming, the tests now use distinct ORM methods
which return objects without any joins. This would have caught
the error had it been in place before.
Additionally, the delet operation retained the default limit of 50
rows (set in place for the web UI), meaning that when it did run,
it would only delete the most recent 50 matching builds.
We now explicitly set the limit to a user-configurable batch size
(by default, 10,000 builds) so that we keep transaction sizes
manageable and avoid monopolizing database locks. We continue deleting
buildsets in batches as long as any matching buildsets remain. This
should allow users to remove very large amounts of data without
affecting ongoing operations too much.
Change-Id: I4c678b294eeda25589b75ab1ce7c5d0b93a07df3
|
| |\ \ \ \ \ \
| | |_|_|/ /
| |/| | | | |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
This migrates the Labels, Nodes and
Projects pages to using the react-table
component from patternfly.
Change-Id: Iaa75e70f4b0c25113369939f97a297571f2ea7a5
|
| |\ \ \ \ \ \
| |_|_|/ / /
|/| | | | | |
|
| | | |_|_|/
| |/| | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
When we detect newly protected branche we also need to set the branch
cache ltime accordingly. Otherwise we might end up with schedulers using
an outdated branch cache during reconfig and layout update which can
result in config not being loaded.
Change-Id: Ie18ef0ce9664e58d25f34018f8eb4513bc8b559a
|
| |\ \ \ \ \ |
|
| | |/ / / /
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Occasionally we need to look into hanging event processing, with the installation_id
to be included in the log, it would be easier to find out which events are blocked
in waiting for the lock.
Change-Id: I824e299501642b61a57883f4b37dc121f5ea0979
|
| |\ \ \ \ \ |
|
| | | |_|/ /
| |/| | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
We are occassionally seeing different types of IO errors when updating
repos on an executor. Currently those exceptions will abort the build
and result in an error being reported.
Since those errors are usually transient and point to some
infrastructure problem we should retry those builds instead.
We'll catch all IOErrors which includes request related exceptions from
the "requests" Python package. See:
https://github.com/psf/requests/blob/main/requests/exceptions.py
Traceback (most recent call last):
File "/opt/zuul/lib/python3.10/site-packages/zuul/executor/server.py", line 3609, in _innerUpdateLoop
self.merger.updateRepo(
File "/opt/zuul/lib/python3.10/site-packages/zuul/merger/merger.py", line 994, in updateRepo
repo = self.getRepo(connection_name, project_name,
File "/opt/zuul/lib/python3.10/site-packages/zuul/merger/merger.py", line 966, in getRepo
url = source.getGitUrl(project)
File "/opt/zuul/lib/python3.10/site-packages/zuul/driver/github/githubsource.py", line 154, in getGitUrl
return self.connection.getGitUrl(project)
File "/opt/zuul/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1744, in getGitUrl
self._github_client_manager.get_installation_key(
File "/opt/zuul/lib/python3.10/site-packages/zuul/driver/github/githubconnection.py", line 1126, in get_installation_key
response = github.session.post(url, headers=headers, json=None)
File "/opt/zuul/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
return self.request("POST", url, data=data, json=json, **kwargs)
File "/opt/zuul/lib/python3.10/site-packages/github3/session.py", line 171, in request
response = super().request(*args, **kwargs)
File "/opt/zuul/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/opt/zuul/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/opt/zuul/lib/python3.10/site-packages/cachecontrol/adapter.py", line 53, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/opt/zuul/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError:
HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded
with url: /api/v3/app/installations/123/access_tokens (Caused by
NewConnectionError('<urllib3.connection.HTTPSConnection object at
0x7f44f6136ef0>: Failed to establish a new connection: [Errno -3]
Temporary failure in name resolution'))
Change-Id: I4e07e945c88b9ba61f83131076fbf7b9768a61f9
|
| |\ \ \ \ \ |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Github will use the PR title as the commit subject for squash merges, so
we don't need include the title once again in the commit description.
Change-Id: Id5a00701c236235f5a49abd025bcfad1b2da916c
|
| |\ \ \ \ \ \ |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
So far we've aborted all cat jobs when any of the cat jobs failed.
However, since we ignore those exceptions anyways unless we are
validating the tenant configuration, we should continue processing the
rest of the job results that haven't failed.
If an early cat job failed we might otherwise not load configuration
from repositories and branches even if those cat jobs were successful.
Change-Id: I34f2a23641de9138b1e887df86ae2602ca190277
|
| |\ \ \ \ \ \ \ |
|
| | |/ / / / / /
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
File annotations that are posted to a PR as part of a check run have a
size limit of 64KB for the message field.
Since it's unclear if this should be 64KiB or 64KB, we'll use KB as a
unit to be on the safe side.
Change-Id: I43e4cfbc3a96bf1e8a9828c55150216940a64728
|
| |\ \ \ \ \ \ \ |
|
| | | |_|_|/ / /
| |/| | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Similar to zuul-web not starting the Github event connector, we also
don't want it to connect to the configured MQTT broker as the web
component won't be using it.
Change-Id: If61da19ab0af39bc68d12c9b6613bf6c41d7efaa
|
| |\ \ \ \ \ \ \
| |_|_|_|_|/ /
|/| | | | | | |
|
| | |/ / / / /
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Traceback (most recent call last):
File "/opt/zuul/lib/python3.10/site-packages/zuul/zk/job_request_queue.py", line 612, in cleanup
"Unable to delete lock %s", path)
UnboundLocalError: local variable 'path' referenced before assignment
Change-Id: I9d76abce0f6e539374765bc7a988484548fda3f6
|
