| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
If one of the queue items finished while a job was still running
for the other, then Zuul might abort that job instead of getting
a SUCCESS result for it.
Change-Id: I855015b884d42792a632fc2138f48efe9ec034d0
|
| |\ \ \ \ |
|
| | |/ / /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The Zuul Ansible callback stream plugin assumed that the ansible loop
var was always called 'item' in the result_dict. You can override this
value (and it is often necessary to do so to avoid collisions) to
something less generic. In those cases we would get errors like:
b'[WARNING]: Failure using method (v2_runner_item_on_ok) in callback plugin'
b'(<ansible.plugins.callback.zuul_stream.CallbackModule object at'
b"0x7fbecc97c910>): 'item'"
And stream output would not include the info typically logged.
Address this by checking if ansible_loop_var is in the results_dict and
using that value for the loop var name instead. We still fall back to
'item' as I'm not sure that ansible_loop_var is always present.
Change-Id: I408e6d4af632f8097d63c04cbcb611d843086f6c
|
| |\ \ \ \
| |/ / /
|/| | | |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
We previously only reported statsd timing info for SUCCESS and FAILURE
builds. But it would be useful to get info for POST_FAILURE and
TIMED_OUT builds as well. In particular with POST_FAILURE builds we can
track how long they are running before they fail.
Change-Id: I2fe443ac2f69f40b7419e5280a38958d3ac7c080
|
| |\ \ \ \ |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
These two tests exercise the LookupError code paths in the branch
cache a bit more. In particular, we now have a test that verifies
that a failure in fetching will result in a retry later.
Change-Id: Ia83b0233ccc31f018a61093d02fd1ce68547f9c0
|
| |\ \ \ \ \
| |/ / / / |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
When exclude-unprotected-branche is in effect and a project doesn't have
any protected branches (e.g. a wrong branch protection rule in Github or
none at all) the branch cache will contain an empty list.
Since the empty list was so far used to indicate a fetch error, those
projects showed up with a config error in zuul-web ("Will not fetch
project branches as read-only is set").
For the branch cache we need to distinguish three different cases:
1. branche cache miss (no fetch yet)
2. branch cache hit (including empty list of branches)
3. fetch error
Instead of storing an empty list in case of a fetch error we will store
a sentinel value of `None` in those cases. However, with this change we
can't use `None` as an indicator for a cache miss anymore, so we are now
raising a `LookupError` instead.
Change-Id: I5b51805f7d0a5d916a46fe89db7b32f14063d7aa
|
| |\ \ \ \ \ |
|
| | | |_|_|/
| |/| | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
The fake Gerrit did not actually merge all of the changes that
should be submitted together simultaneously. This meant our testing
that the git repo branch pointer moved forward was not correct.
This change improves the fake Gerrit so that all changes under
submitWholeTopic are merged simultaneously.
This then shows the error where after two changes are merged
simultaneously, the git repo check fails.
That is because our check is performed by simply ensuring that the
branch pointer is at some sha other than the one it was at right
before we started the merge operation. The sequence is this:
1) Start reporting change #1
2) Get branch sha and store on change #1: abc1
3) Submit change #1 (Gerrit also merges change #2)
4) Get branch sha: abc2
5) Verify abc1 != abc2 (success)
6) Start reporting change #2
7) Get branch sha and store on change #2: abc2
8) Submit change #2 (already merged)
9) Get branch sha: abc2
A) Verify abc2 != abc2 (failure)
This is corrected by using only the first branch sha in a bundle.
When we store the branch sha on the change, we check to see if
any other changes in the bundle for the same project+branch already
have a sha; if they do, we use that. Otherwise we know we are
the first, so we fetch it.
This changes the steps above to:
2) Get branch sha for all Gerrit changes in bundle
7) [not needed -- branch sha already stored in step 2]
A) Verify abc1 != abc2 (success)
Change-Id: Ia9ef411cbf24d1e4e31456ddc660e5b2a6eb5321
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Disable caching for the remaining zuul-web tests that use the tenant
list.
Change-Id: I03c80a2171ee02fba41319b21f78003e4c8554b4
|
| |/ / / /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Change I101fc42e29f4b376272c417ec536c17f05cb1a60 introduced a race
in test_tenant_add_remove. If the test takes longer than 1 second,
it will succeed, but less than the 1 second cache interval and it
will fail.
Correct that by disabling caching for the test.
Change-Id: I313e519ab26cdbdad0cc17cd4d3489b49482f0b3
|
| |\ \ \ \ |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
formatStatusUrl() return build set URL when the
item.current_build_set.result exist (zuul/model.py)
Also updated the quick-start to continue to look for the
build URL instead of the buildset URL.
Change-Id: I5f8433e2926da5a8d14b966d89cc943be1ecfca9
|
| |\ \ \ \ \ |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
It is possible to merge an invalid Zuul config by having a dependency
on a job which does not exist (e.g., due to a typo in the job name).
If this happens, the problem will only be detected when the job graph
is frozen. At this point, the user will need to try to find out where
the erroneus reference is defined.
To try to avoid this situation, we can validate job dependency references
early, as we do most other configuration objects (such as references to
nodesets from jobs). This should help users avoid merging new errors.
Existing jobs and project stanzas with reference errors will become
invalid and ineffective, which may cause some collatoral damage, so a
release note is included.
This may have escaped notice so far because within a single repo,
attempting to add a bad config would trigger an error freezing the job
graph and therefore bringing the error to the attention of the user.
Having a project stanza in a config project is much more likely to
produce the issue since the job graph would not be frozen as part of
that change.
Change-Id: I196a2fb13e93847bc4db4b20f30dea8fecba6325
|
| |\ \ \ \ \ \ |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
If the command didn't exist:
- Popen would throw an exception
- 't' would not be updated (t is None)
- return code would not be written to the console
- zuul_stream would wait unecessary for 10 seconds
As rc is defined in normal case or in both exceptions, it can be written
in each case to the console.
Change-Id: I77a4e1bdc6cd163143eacda06555b62c9195ee38
|
| |\ \ \ \ \ \ \ |
|
| | | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
In previous change support for the gitlab merge was added, but the
parameters dict was not properly passed to the invocation method.
Fix this now and add corresponding test.
Change-Id: I781c02848abc524ca98e03984539507b769d19fe
|
| |\ \ \ \ \ \ \ \ |
|
| | | | | | | | | |
| | | | | | | | |
| | | | | | | | |
| | | | | | | | |
| | | | | | | | |
| | | | | | | | |
| | | | | | | | | |
This verifies that we aren't performing unecessary tenant reconfigs
with the new dynamic-only project feature.
Change-Id: Ied8f0166c17ad9262d36a956be1cd5e06342a8ae
|
| |\ \ \ \ \ \ \ \ \
| |_|_|_|_|_|/ / /
|/| | | | | | | | |
|
| | | |_|_|/ / / /
| |/| | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
When Zuul dequeues a change because it no longer meets the code
review systems merge requirements, it says it was dequeued
"due to a missing requirement". To clarify it's the code review
system's merge requirements (as opposed to some kind of job,
artifact, or other change), update the text to "due to a
missing merge requirement".
Change-Id: Ia76c0564726c0e1317302e76f3f2bdae2c1ff431
|
| |\ \ \ \ \ \ \ \
| |_|_|_|_|_|/ /
|/| | | | | | | |
|
| | | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
Previously support for Gerrit's submitWholeTopic feature was added
so that when it is enabled, changes that are submitted together are
treated as circular dependencies in Zuul. However, this feature did
not work in a gating pipeline because when that setting is enabled,
Gerrit requires all changes to be mergable at once so that it can
attempt to atomically merge all of them. That means that every
change would need a Verified+2 vote before any change can be
submitted. Zuul leaves the vote and submits each change one at a
time.
(Note, this does not affect the emulated submitWholeTopic feature
in Zuul, since in that case, Gerrit will merge each change alone.)
To correct this, a two-phase option is added to reporters. In phase1,
reporters will report all information about the change but not submit.
In phase2, they will only submit. In the cases where we are about
to submit a successful bundle, we enable the two-phase option and
report the entire bundle without submitting first, then proceed to
submit each change in the bundle in sequence as normal. In Gerrit,
if submitWholeTopic is enabled, this will cause all changes to be
submitted as soon as the first one is, but that's okay because we
handle the case where we try to submit a change and it is already
submitted.
The fake Gerrit used in the Zuul unit tests is updated to match
the real Gerrit in these cases. If submitWholeTopic is enabled,
it will return a 409 to submit requests if the whole topic is not
able to be submitted.
One unit test of failed bundle reporting is adjusted since we will
now report the buildset result to all changes before potentially
reporting a second time if the bundle later fails to merge.
While this does mean that some changes will have extra report
messages, it actually makes the behavior consistent (before, some
changes would have 2 reports and some would have only 1; now all
changes will have 2 reports: the expected result and then a second
report of the unexpected merge failure).
All reporters are updated to handle the two-phase reporting. Since
all reporters have different API methods to leave comments and merge
changes, this won't actually cause any extra API calls even for
reporters which don't need two-phase reporting.
Non-merging reporters (MQTT, SMTP, etc) simply ignore phase2.
Change-Id: Ibf377ab5b7141fe60ecfd5cbbb296bb4f9c24265
|
| |\ \ \ \ \ \ \ \ |
|
| | | |_|_|_|/ / /
| |/| | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
In a multi-host Gerrit environment (HA or failover) it's plausible
that admins may use one mechanism for managing ingress for HTTP
requests and a different for SSH requests. Or admins may have
different firewall rules for each. To accomodate these situations,
Add an "ssh_server" configuration item for Gerrit. This makes the
set of hostname-like items the following:
* server: the HTTP hostname and default for all others
* canonical_hostname: what to use for golang-style git paths
* ssh_server: the hostname to use for SSH connections
* baseurl: the base URL for HTTP connections
The following are equivalent:
server=review.example.com
ssh_server=ssh-review.example.com
and:
server=ssh-review.example.com
baseurl=https://review.example.com
Change-Id: I6e9cd9f48c1a78d8d24bfe176efbb932a18ec83c
|
| |\ \ \ \ \ \ \ \
| |_|/ / / / / /
|/| | | | | | | |
|
| | |/ / / / / /
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
This instructs the scheduler to wait until all tenants have been
initialized before processing pipelines. This can be useful for
large systems with excess scheduler capacity to speed up a rolling
restart.
This also removes an unused instance variable from
SchedulerTestManager.
Change-Id: I19e733c881d1abf636674bf572f4764a0d018a8a
|
| |\ \ \ \ \ \ \ |
|
| | |/ / / / / /
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Loading config involves significant network operations for each project:
* Loading project keys
* Asking the source for the list of branches for each project
* Retrieving the config file contents from the ZK cache (if present)
* Retrieving the config file contents from git (otherwise)
Only the third item in that list is parallelized currently; the others
are serialized. To parallelize the remainder, use a thread pool executor.
The value of max_workers=4 is chosen as it appears in practice on OpenDev
to make the most significant reduction in startup time while higher values
make little difference (and could potentially contribute to DoS scenarios
or local thread contention). Observed config priming times for various
worker counts:
1: 282s
2: 181s
4: 144s
8: 146s
Change-Id: I65472a8af96ed95eb28b88cc623ef103be76a46f
|
| |\ \ \ \ \ \ \
| | |_|/ / / /
| |/| | | | | |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
This adds an option to specify that certain branches should always trigger
dynamic configuration and never be included in static configuration.
The use case is a large number of rarely used feature branches, where
developers would still like to be able to run pre-merge check jobs, and alter
those jobs on request, but otherwise not have the configuration clogged up
with hundreds of generally unused job variants.
Change-Id: I60ed7a572d66a20a2ee014f72da3cb7132a550da
|
| |\ \ \ \ \ \ \
| |_|/ / / / /
|/| | | | | | |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
This adds support for global semaphores which can be used by multiple tenants.
This supports the use case where they represent real-world resources which
operate independentyl of Zuul tenants.
This implements and removes the spec describing the feature. One change from
the spec is that the configuration object in the tenant config file is
"global-semaphore" rather than "semaphore". This makes it easier to distinguish
them in documentation (facilitating easier cross-references and deep links),
and may also make it easier for users to understand that they have distinct
behavoirs.
Change-Id: I5f2225a700d8f9bef0399189017f23b3f4caad17
|
| | |_|/ / / /
|/| | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
TestZuulStream28 and TestZuulStream29 have 2 base classes.
As class variable wait_timeout is defined in both classes, the value
from the first one (AnsibleZuulTestCase) was used. define wait_timeout
in setUp function to initialize to the expected value.
Change-Id: Id9626190a72ce8b0f2e5a2847396bad8abfcbc32
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | | |
Change-Id: I03e78a1774e20af6f6895faa089017cdcf62bb48
|
| |\ \ \ \ \ \
| |_|_|/ / /
|/| | | | | |
|
| | |/ / / /
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
When a project configuration defined a queue, but did not directly
specify any pipeline configuration (e.g. only referenced templates), the
relative priority queues were not setup correctly.
This could happen in pipelines using the independent and supercedent
manager. Other pipelines using the shared change queue mixin handle this
correctly.
This edge case will be tested in
`test_scheduler.TestScheduler.test_nodepool_relative_priority_check` by
slightly modifying the config to use a template for one of the projects.
Change-Id: I1f682e6593ccdad3cfacf5817fc1a1cf7de8856b
|
| |\ \ \ \ \
| |/ / / /
|/| | | | |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This adds a zuul-admin command which allows operators to delete old
database entries.
Change-Id: I4e277a07394aa4852a563f4c9cdc39b5801ab4ba
|
| |\ \ \ \ \ |
|
| | | |_|_|/
| |/| | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
When an operator issues a full-reconfigure or tenant-reconfigure
command to the scheduler, the branch cache is cleared. It will
be fully populated again during the subsequent full-reconfiguration,
or eventually populated more slowly over time after a
tenant-reconfiguration. But during the time the branch cache is
clear, zuul-web will be unable to update layouts since it is incapable
of populating the branch cache itself.
This produces a short time period of errors during a full-reconfig
or a potentially long time period of errors after a tenant-reconfig.
To correct this, we now detect whether a layout update in zuul-web
is incomplete due to a branch cache error, and retry the update
later. In the scheduler, we only clear the projects from the branch
cache that are affected by the tenants we are reloading (still all the
projects for a full-reconfigure). This limits the time during which
zuul-web will be unable to update the layout to only the time that
the scheduler spends actually performing a reconfiguration.
(Note that in general, the error here is not because zuul-web is
loading the layout for the tenant that zuul-scheduler is reconfiguring,
but rather it is loading a layout for a tenant which has projects
in common with the tenant that is being reconfigured.)
Change-Id: I6794da4d2316f7df6ab302c74b3efb5df4ce461a
|
| |\ \ \ \ \
| |_|/ / /
|/| | | | |
|
| | | | | |
| | | | |
| | | | |
| | | | | |
Change-Id: Iec87857aa58f71875d780da3698047dae01120d7
|
| |\ \ \ \ \ |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
This adds support for deduplicating jobs within dependency cycles.
By default, this will happen automatically if we can determine that the
results of two builds would be expected to be identical. This uses a
heuristic which should almost always be correct; the behavior can be
overidden otherwise.
Change-Id: I890407df822035d52ead3516942fd95e3633094b
|
| |\ \ \ \ \ \
| |/ / / / / |
|
