| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
| |
This causes e.g. web requests that should be trackable via the event
id in the logs.
Change-Id: Iade2558f2312aedca7480b4ea1d3df60735cfc90
|
| |
|
|
|
|
|
| |
This helps with debugging from logs in case something doesn't enter a
gate as expected.
Change-Id: Ia0c7e84812d479c455d72f8e4c367975ea0bd709
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The current attempt to caches branches is ineffective -- we
query the list of branches during every tenant reconfiguration.
The list of branches for a project is really global information;
we might cache it on the Abide, however, drivers may need to filter
that list based on tenant configuration (eg, github protected
branches). To accomodate that, just allow/expect the drivers to
perform their own caching of branches, and to generally keep
the list up to date (or at least invalidate their caches) by
observing branch create/delete events.
A full reconfiguration instructs the connections to clear their
caches so that we perform a full query. That way, an operator
can correct from a situation where the cache is invalid.
Change-Id: I3bd0cda5875dd21368e384e3704a61ebb5dcedfa
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
In GitHub with many app installations the getChangesDependingOn
currently iterates over all installations within the system and fires
up a search query. In larger deployments this can sum up to hundreds
of queries for a single parent-change-enqueued event. At least for
multi-tenant deployments this can be greatly improved when limiting
the scope just to the installations related to the tenant. With this
improvement in most tenants this can be accomplished with a handful of
requests then.
Change-Id: Ibfad750a685d2ec58f3e452bfe2098bbdc294e37
|
| |
|
|
|
|
|
|
| |
Additional tests and docs in later patches.
Change-Id: I3b86a1e3dd507fa5e584680fb6c86d35f9ff3e23
Story: 2001334
Task: 5885
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using a branch and pull model on a shared repository there are
usually one or more protected branches which are gated and a dynamic
number of temporary personal/feature branches which are the source for
the pull requests. These temporary branches while ungated can
potentially include broken zuul config and therefore break the global
tenant wide configuration.
In order to deal with this model add support for excluding unprotected
branches. This can be configured on tenant level and overridden per
project.
Change-Id: I8a45fd41539a3c964a84142f04c1644585c0fdcf
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
It exists only for py2/py3 compat. We do not need it any more.
This will explicitly break Zuul v3 for python2, which is different than
simply ceasing to test it and no longer declaring we support it. Since
we're not testing it any longer, it's bound to degrade overtime without
us noticing, so hopefully a clean and explicit break will prevent people
from running under python2 and it working for a minute, then breaking
later.
Change-Id: Ia16bb399a2869ab37a183f3f2197275bb3acafee
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As we expand the Github driver, we're seeing a need to specify driver-specific
pipeline requirements. To accomplish this, bump the require/reject pipeline
keywords down a level underneath connection names. This lets users specify
per-source pipeline requirements.
This adds new API methods for sources to create the new pipeline filters
(by returning instances or subclasses of RefFilter, which used to be called
ChangeishFilter).
This change also creates and/or moves driver-specific subclasses of EventFilter
and TriggerEvent in(to) their respective drivers.
Change-Id: Ia56c254e3aa591a688103db5b04b3dddae7b2da4
|
| |
|
|
|
|
|
|
|
|
|
| |
This is the start of the implementation of:
http://lists.openstack.org/pipermail/openstack-infra/2017-March/005208.html
It lets us associate a canonical hostname with every connection
that we will later use to uniquely identify source code repos.
Story: 2000953
Change-Id: I7f2e64944d46f304e63a54078e682fd5e1682f27
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Instatiations of sources all passed (driver, connection) to the
constructor, which was expecting (config, connection). The
config option is currently ignored (sources currently have no
additional configuration), which is why we didn't notice the
discrepancy.
Update the constructor to (driver, connection, optional config)
to match the rest of the driver-related classes.
Change-Id: Ibc878b51b81950559d39b00b1591864c7661fe7c
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change, while substantial, is mostly organizational.
Currently, connections, sources, triggers, and reporters are
discrete concepts, and yet are related by virtue of the fact that
the ConnectionRegistry is used to instantiate each of them. The
method used to instantiate them is called "_getDriver", in
recognition that behind each "trigger", etc., which appears in
the config file, there is a class in the zuul.trigger hierarchy
implementing the driver for that trigger. Connections also
specify a "driver" in the config file.
In this change, we redefine a "driver" as a single class that
organizes related connections, sources, triggers and reporters.
The connection, source, trigger, and reporter interfaces still
exist. A driver class is responsible for indicating which of
those interfaces it supports and instantiating them when asked to
do so.
Zuul instantiates a single instance of each driver class it knows
about (currently hardcoded, but in the future, we will be able to
easily ask entrypoints for these). That instance will be
retained for the life of the Zuul server process.
When Zuul is (re-)configured, it asks the driver instances to
create new connection, source, trigger, reporter instances as
necessary. For instance, a user may specify a connection that
uses the "gerrit" driver, and the ConnectionRegistry would call
getConnection() on the Gerrit driver instance.
This is done for two reasons: first, it allows us to organize all
of the code related to interfacing with an external system
together. All of the existing connection, source, trigger, and
reporter classes are moved as follows:
zuul.connection.FOO -> zuul.driver.FOO.FOOconnection
zuul.source.FOO -> zuul.driver.FOO.FOOsource
zuul.trigger.FOO -> zuul.driver.FOO.FOOtrigger
zuul.reporter.FOO -> zuul.driver.FOO.FOOreporter
For instance, all of the code related to interfacing with Gerrit
is now is zuul.driver.gerrit.
Second, the addition of a single, long-lived object associated
with each of these systems allows us to better support some types
of interfaces. For instance, the Zuul trigger maintains a list
of events it is required to emit -- this list relates to a tenant
as a whole rather than individual pipelines or triggers. The
timer trigger maintains a single scheduler instance for all
tenants, but must be able to add or remove cron jobs based on an
individual tenant being reconfigured. The global driver instance
for each of these can be used to accomplish this.
As a result of using the driver interface to create new
connection, source, trigger and reporter instances, the
connection setup in ConnectionRegistry is much simpler, and can
easily be extended with entrypoints in the future.
The existing tests of connections, sources, triggers, and
reporters which only tested that they could be instantiated and
have names have been removed, as there are functional tests which
cover them.
Change-Id: Ib2f7297d81f7a003de48f799dc1b09e82d4894bc
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This lets us ask a source for all of the branches for a project.
This uses the git protocol for now, but this can get much nicer
in the future if we switch to using Gerrit's REST API. It should
also be easy to do with github.
The included comment indicates why it's being added -- implementation
to follow in subsequent changes.
Change-Id: I0dfcd61f343a235dcf935aea434b9772d6e746d9
|
| |/
|
|
|
|
|
|
| |
Expose the ability to refresh a change in zuul/connection/gerrit.py
too, which is needed for our unit testing.
Change-Id: Iefd09d9b8deef563299e0f209d95e25b61aa4c1e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The connection registry should not have to know about the scheduler,
rather, the inverse is true.
(NB, connections themselves still know about the scheduler, but
that's okay, that happens after the connection registry is created.)
Drivers should be able to access the global configuration when
being created, so store that when the connection registry configures
itself.
Change-Id: Iea4b8fe3888b5eefd3df9ce385225b885f2caa0b
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Sources and connections are very tightly coupled in reality. Rather
than trying to maintain them as two abstractions, consider the
connection to hold all of the information and logic about the reality
of the external resource it represents. Make sources mere local
data structures that are used to interface a connection with a pipeline.
As seen in subsequent changes, this will allow us to simplify the
interconnections between objects.
Change-Id: I2dd88e1165267e4f987a205ba55923eaec7ea9ce
|
| |\
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
zuul/model.py
zuul/scheduler.py
Change-Id: I2973bfae65b3658549dc13aa3ea0efe60669ba8e
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Add reconfigure test case. This test previously fails currently due to a
regression introduced with the connections changes.
Because multiple sources share a connection, a pipeline that does not hold
and therefore require any changes in the cache may clear a connections
cache before a pipeline that does need said change has an opportunity to
add it to the relevant list.
Allow connections to manage their cache directly rather than the source
doing it vicariously ignorant of other pipelines/sources. Collect the
relevant changes from all pipelines and ask any connections holding a
cache for that item to keep it on reconfiguration.
Co-Authored-By: James E. Blair <jeblair@linux.vnet.ibm.com>
Change-Id: I2bf8ba6b9deda58114db9e9b96985a2a0e2a69cb
|
| |\ \
| |/
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
zuul/connection/gerrit.py
zuul/lib/connections.py
zuul/model.py
zuul/scheduler.py
Change-Id: If1c8ac3bf26bd8c4496ac130958b966d9937519e
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Make sure we update the referenced change object on a new gerrit
event rather than waiting to remake the queue item.
This was a performance regression in the connection changes.
Change-Id: I2a967f0347352a7674deb550e34fb94d1d903e89
|
| | |
| |
| |
| |
| |
| |
| | |
This begins a lot of related changes refactoring config loading,
the data model, etc., which will continue in subsequent changes.
Change-Id: I2ca52a079a837555c1f668e29d5a2fe0a80c1af5
|
| | |
| |
| |
| |
| |
| | |
It still remains in the drivers that the connections utilize.
Change-Id: Ie6efb57af297fbd546eed3e1104299b2e1a5205e
|
| |/
|
|
| |
Change-Id: Ia6c21152c00c9380c17c559290ed98ff22cf767b
|
| |
|
|
|
|
|
|
| |
This is a large refactor and as small as I could feasibly make it
while keeping the tests working. I'll do the documentation and
touch ups in the next commit to make digesting easier.
Change-Id: Iac5083996a183d1d8a9b6cb8f70836f7c39ee910
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
and test the all sources adhere to the set contract.
Also standardise the source (triggers to come) class names
to NameSource.
This will make it easier to do more sources in the future and also
add the possibility of loading sources dynamically.
Co-Authored-By: Gregory Haynes <greg@greghaynes.net>
Change-Id: I15b32013904f60873601dd7cc8fce3c158787de4
|
|
|
This is to further differentiate between sources and triggers.
Eventually allowing for multiple triggers per pipeline.
Still to come is separating connections from everything.
Change-Id: I1d680dbed5f650165643842af450f16b32ec5ed9
|
