| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This adds the "draft" PR status as a pipeline requirement to the
GitHub driver. It is already used implicitly in dependent pipelines,
but this will allow it to be added explicitly to other pipelines
(for example, check).
This also fixes some minor copy/pasta errors in debug messages related
to github pipeline requirements.
Change-Id: I05f8f61aee251af24c1479274904b429baedb29d
|
| |/
|
|
|
|
|
| |
Ansible 5 is no longer supported and 6 is available and working.
Deprecate Ansible 5.
Change-Id: I8c152f7c0818bccd07f50e85bef9a82ddb863a68
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
This updates the OpenAPI docs to include the semaphores endpoint,
and adds a Semaphores tab to the web UI to show information about
semaphores within a tenant.
Change-Id: If78b27131ac76aff93c47a986fce6eae3e068668
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | |
| | | |
Versions 2.8 and 2.9 are no longer supported by the Ansible project.
Change-Id: I888ddcbecadd56ced83a27ae5a6e70377dc3bf8c
|
| |\ \ \ |
|
| | | |/
| |/|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This feature instructs Zuul to attempt a second or more node request
with a different node configuration (ie, possibly different labels)
if the first one fails.
It is intended to address the case where a cloud provider is unable
to supply specialized high-performance nodes, and the user would like
the job to proceed anyway on lower-performance nodes.
Change-Id: Idede4244eaa3b21a34c20099214fda6ecdc992df
|
| |\ \ \ |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This was deprecated quite some time ago and we should remove it as
part of the next major release.
Also remove a very old Zuul v1 layout.yaml from the test fixtures.
Change-Id: I40030840b71e95f813f028ff31bc3e9b3eac4d6a
|
| |\ \ \ \
| |/ / / |
|
| | | |/
| |/|
| | |
| | |
| | |
| | |
| | | |
This was previously deprecated and should be removed shortly before
we release Zuul v7.
Change-Id: Idbdfca227d2f7ede5583f031492868f634e1a990
|
| |\ \ \
| |/ /
|/| | |
|
| | |/
| |
| |
| |
| |
| |
| |
| | |
This adds an option to include result data from a job in the MQTT
reporter. It is off by default since it may be quite large for
some jobs.
Change-Id: I802adee834b60256abd054eda2db834f8db82650
|
| | |
| |
| |
| | |
Change-Id: I0d450d9385b9aaab22d2d87fb47798bf56525f50
|
| |/
|
|
| |
Change-Id: I2576d0dcec7c8f7bbb76bdd469fd992874742edc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a config-error pipeline reporter configuration option and
now also reports config errors and merge conflicts to the database
as buildset failures.
The driving use case is that if periodic pipelines encounter config
errors (such as being unable to freeze a job graph), they might send
email if configured to send email on merge conflicts, but otherwise
their results are not reported to the database.
To make this more visible, first we need Zuul pipelines to report
buildset ends to the database in more cases -- currently we typically
only report a buildset end if there are jobs (and so a buildset start),
or in some other special cases. This change adds config errors and
merge conflicts to the set of cases where we report a buildset end.
Because of some shortcuts previously taken, that would end up reporting
a merge conflict message to the database instead of the actual error
message. To resolve this, we add a new config-error reporter action
and adjust the config error reporter handling path to use it instead
of the merge-conflicts action.
Tests of this as well as the merge-conflicts code path are added.
Finally, a small debug aid is added to the GerritReporter so that we
can easily see in the logs which reporter action was used.
Change-Id: I805c26a88675bf15ae9d0d6c8999b178185e4f1f
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently the timer trigger accepts an optional "jitter" specification
which can delay the start of a pipeline timer trigger by up to a
certain number of seconds. It applies uniformly to every project-branch
that participates in the pipeline. For example, if a periodic pipeline
with nova and glance is configured to trigger at midnight, and has
a jitter of 30 seconds, then the master and stable branches of nova and
glance will all be enqueued at the same time (perhaps 00:00:17).
While this provides some utility in that if other systems are configured
to do things around midnight, this pipeline may not join a thundering
herd with them. Or if there are many periodic pipelines configured for
midnight (perhaps across different tenants, or just with slightly different
purposes), they won't be a thundering hurd.
But to the extent that jobs within a given pipeline might want to avoid
a thundering herd with other similar jobs in the same pipeline, it offers
no relief. While Zuul may be able to handle it (especially since multiple
schedulers allows other pipelines to continue to operate), these jobs
may interact with remote systems which would appreciate not being DoS'd.
To alleviate this, we change the jitter from applying to the pipeline
as a whole to individual project-branches. To be clear, it is still the
case that the pipeline has only a single configured trigger time (this
change does not allow projects to configure their own triggers). But
instead of firing a single event for the entire pipeline, we will fire
a unique event for every project-branch in that pipeline, and these
events will have the jitter applied to them individually. So in our
example above, nova@master might fire at 00:00:05, nova@stable/zulu
may fire at 00:00:07, glance@master at 00:00:13, etc.
This behavior is similar enough in spirit to the current behavior that
we can consider it a minor implementation change, and it doesn't require
any new configuration options, feature flags, deprecation notice, etc.
The documentation is updated to describe the new behavior, as well as
correct an error relating to the description of jitter (it only delays,
not advances, events).
We currently add a single job to APScheduler for every timer triggered
pipeline in every tenant (so the number of jobs is the sum of the
periodic pipelines in every tenant). OpenDev for example may have on
the order of 20 APScheduler jobs. With the new approach, we will
enqueue a job for each project-branch in a periodic pipeline. For a
system like OpenDev, that could potentially be thousands of jobs.
In reality, based on current configuration and pipeline participation,
it should be 176.
Even though it will result in the same number of Zuul trigger events,
there is overhead to having more APScheduler jobs. To characterize
this, I performed a benchmark where I added a certain number of
APScheduler jobs with the same trigger time (and no jitter) and
recorded the amount of time needed to add the jobs and also, once the
jobs began firing, the elapsed time from the first to the last job.
This should charactize the additional overhead the scheduler will
encounter with this change.
Time needed to add jobs to APScheduler (seconds)
1: 0.00014448165893554688
10: 0.0009338855743408203
100: 0.00925445556640625
1000: 0.09204769134521484
10000: 0.9236903190612793
100000: 11.758053541183472
1000000: 223.83168983459473
Time to run jobs (last-first in seconds)
1: 2.384185791015625e-06
10: 0.006863832473754883
100: 0.09936022758483887
1000: 0.22670435905456543
10000: 1.517075777053833
100000: 19.97287678718567
1000000: 399.24730825424194
Given that this operates primarily at the tenant level (when a tenant
reconfiguration happens, jobs need to be removed and added), I think
it makes sense to consider up to 10,000 jobs a reasonable high end.
It looks like we can go a little past that (between 10,000 and 100,000)
while still seeing something like a linear increase. As we approach
1,000,000 jobs it starts looking more polynomial and I would not conisder
the performance to be acceptable. But 100,000 is already an unlikely
number, so I think this level of performance is okay within the likely
range of jobs.
The default executor used by APScheduler is a standard python
ThreadPoolExecutor with a maximum of 10 simultaneous workers. This
will cause us to fire up to 10 Zuul event simultaneously (whereas before
we were only likely to fire simultaneous events if multiple tenants
had identical pipeline timer triggers). This could result in more
load on the connection sources and change cache as they update the
branch tips in the change cache. It seems plausible that 10 simulatenous
events is something that the sources and ZK can handle. If not, we
can reduce the granularity of the lock we use to prevent updating the
same project at the same time (to perhaps a single lock for all
projects), or construct the APScheduler with a lower number of max_workrs.
Change-Id: I27fc23763da81273eb135e14cd1d0bd95964fd16
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
formatStatusUrl() return build set URL when the
item.current_build_set.result exist (zuul/model.py)
Also updated the quick-start to continue to look for the
build URL instead of the buildset URL.
Change-Id: I5f8433e2926da5a8d14b966d89cc943be1ecfca9
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
It is possible to merge an invalid Zuul config by having a dependency
on a job which does not exist (e.g., due to a typo in the job name).
If this happens, the problem will only be detected when the job graph
is frozen. At this point, the user will need to try to find out where
the erroneus reference is defined.
To try to avoid this situation, we can validate job dependency references
early, as we do most other configuration objects (such as references to
nodesets from jobs). This should help users avoid merging new errors.
Existing jobs and project stanzas with reference errors will become
invalid and ineffective, which may cause some collatoral damage, so a
release note is included.
This may have escaped notice so far because within a single repo,
attempting to add a bad config would trigger an error freezing the job
graph and therefore bringing the error to the attention of the user.
Having a project stanza in a config project is much more likely to
produce the issue since the job graph would not be frozen as part of
that change.
Change-Id: I196a2fb13e93847bc4db4b20f30dea8fecba6325
|
| |\ \ \ |
|
| | |/ /
| | |
| | |
| | |
| | |
| | |
| | | |
'web' section is used by zuul-web component while zuul REST API
client uses 'webclient' section.
Change-Id: I145c9270ca6676abd0d4977ce1c4c637d304a264
|
| |\ \ \ |
|
| | |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
In a multi-host Gerrit environment (HA or failover) it's plausible
that admins may use one mechanism for managing ingress for HTTP
requests and a different for SSH requests. Or admins may have
different firewall rules for each. To accomodate these situations,
Add an "ssh_server" configuration item for Gerrit. This makes the
set of hostname-like items the following:
* server: the HTTP hostname and default for all others
* canonical_hostname: what to use for golang-style git paths
* ssh_server: the hostname to use for SSH connections
* baseurl: the base URL for HTTP connections
The following are equivalent:
server=review.example.com
ssh_server=ssh-review.example.com
and:
server=ssh-review.example.com
baseurl=https://review.example.com
Change-Id: I6e9cd9f48c1a78d8d24bfe176efbb932a18ec83c
|
| |\ \ \ |
|
| | |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This instructs the scheduler to wait until all tenants have been
initialized before processing pipelines. This can be useful for
large systems with excess scheduler capacity to speed up a rolling
restart.
This also removes an unused instance variable from
SchedulerTestManager.
Change-Id: I19e733c881d1abf636674bf572f4764a0d018a8a
|
| |\ \ \ |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This adds an option to specify that certain branches should always trigger
dynamic configuration and never be included in static configuration.
The use case is a large number of rarely used feature branches, where
developers would still like to be able to run pre-merge check jobs, and alter
those jobs on request, but otherwise not have the configuration clogged up
with hundreds of generally unused job variants.
Change-Id: I60ed7a572d66a20a2ee014f72da3cb7132a550da
|
| |\ \ \ \
| |_|/ /
|/| | | |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This adds support for global semaphores which can be used by multiple tenants.
This supports the use case where they represent real-world resources which
operate independentyl of Zuul tenants.
This implements and removes the spec describing the feature. One change from
the spec is that the configuration object in the tenant config file is
"global-semaphore" rather than "semaphore". This makes it easier to distinguish
them in documentation (facilitating easier cross-references and deep links),
and may also make it easier for users to understand that they have distinct
behavoirs.
Change-Id: I5f2225a700d8f9bef0399189017f23b3f4caad17
|
| |\ \ \ \
| |/ / /
|/| | | |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This adds a zuul-admin command which allows operators to delete old
database entries.
Change-Id: I4e277a07394aa4852a563f4c9cdc39b5801ab4ba
|
| |\ \ \ \
| |/ / /
|/| | | |
|
| | | | |
| | | |
| | | |
| | | | |
Change-Id: Iec87857aa58f71875d780da3698047dae01120d7
|
| |\ \ \ \ |
|
| | |/ / /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This adds support for deduplicating jobs within dependency cycles.
By default, this will happen automatically if we can determine that the
results of two builds would be expected to be identical. This uses a
heuristic which should almost always be correct; the behavior can be
overidden otherwise.
Change-Id: I890407df822035d52ead3516942fd95e3633094b
|
| |\ \ \ \ |
|
| | |/ / /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Merges cannot be cherry-picked in git, so if a change is a merge, do a
`git merge` instead of a cherry-pick to match how Gerrit will merge the
change.
Change-Id: I9bc7025d2371913b63f0a6723aff480e7e63d8a3
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
|
| |\ \ \ \ |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
We have two CLIs: zuul-client for REST-related operations, which cover
tenant-scoped, workflow modifying actions such as enqueue, dequeue and
promote; and zuul which supercedes zuul-client and covers also true admin
operations like ZooKeeper maintenance, config checking and issueing auth tokens.
This is a bit confusing for users and operators, and can induce code
duplication.
* Rename zuul CLI into zuul-admin. zuul is still a valid endpoint
and will be removed after next release.
* Print a deprecation warning when invoking the admin CLI as zuul
instead of zuul-admin, and when running autohold-*, enqueue-*,
dequeue and promote subcommands. These subcommands will need to be
run with zuul-client after next release.
* Clarify the scopes and deprecations in the documentation.
Change-Id: I90cf6f2be4e4c8180ad0f5e2696b7eaa7380b411
|
| |\ \ \ \ \
| |/ / / /
|/| | / /
| | |/ /
| |/| | |
|
| | |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This allows operators to filter the set of branches from which
Zuul loads configuration. They are similar to exclude-unprotected-branches
but apply to all drivers.
Change-Id: I8201b3a19efb266298decb4851430b7205e855a1
|
| |/ /
| |
| |
| |
| |
| |
| | |
This was missed in the preparation for Zuul v6. Let's include it now to
avoid further confusion.
Change-Id: Id5b999e3012510a3ab28e383aa8e245f5f2b9cb2
|
| |\ \ |
|
| | | |
| | |
| | |
| | | |
Change-Id: I0358608cb588000f6f9c0ec8ac0c4db179f8fab7
|
| |\ \ \ |
|
| | |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Zuul ignores nodes with network_cli connections when running the
setup playbook in order to support the use-case where a network
appliance begins a job in an unreachable state.
Starting with Zuul v4.6.0, for security reasons, host variables
require a functioning Ansible connection in order to be set.
This includes the Nodepool variables such as public and private
IP addresses. Causing a network_cli host to become online after
the start of a job requires this information, and so this
functionality has not worked since 4.6.0.
To correct this, we now always add in the nodepool host vars as
originally set regardless of whether the variable freeze playbook
runs for a particular host. This means that even if we have no
other variables set for that host, we at least know the IP address
of the host and can interact with it directly in order to bring it
online.
Additionally, the freeze playbook did not correctly check the
return value, due to a typo. This accidentally allowed Zuul to
continue to function in this situation (but without the nodepool
variables accessible). With this change we continue to ignore
the return value, but intentionally so.
A test for this use-case is added, along with a releaso note.
Change-Id: Icd8a2c035e6c04a7c198281adbd07fef422a6c63
Story: 2009226
|
