Cache branches in connections/sources

The current attempt to caches branches is ineffective -- we
query the list of branches during every tenant reconfiguration.

The list of branches for a project is really global information;
we might cache it on the Abide, however, drivers may need to filter
that list based on tenant configuration (eg, github protected
branches).  To accomodate that, just allow/expect the drivers to
perform their own caching of branches, and to generally keep
the list up to date (or at least invalidate their caches) by
observing branch create/delete events.

A full reconfiguration instructs the connections to clear their
caches so that we perform a full query.  That way, an operator
can correct from a situation where the cache is invalid.

Change-Id: I3bd0cda5875dd21368e384e3704a61ebb5dcedfa

1 files changed, 17 insertions, 2 deletions

diff --git a/zuul/source/__init__.py b/zuul/source/__init__.py
index 4e7fc71e4..439a40743 100644
--- a/zuul/source/__init__.py
+++ b/zuul/source/__init__.py
@@ -49,7 +49,14 @@ class BaseSource(object, metaclass=abc.ABCMeta):
 
     @abc.abstractmethod
     def getChange(self, event):
-        """Get the change representing an event."""
+        """Get the change representing an event.
+
+        This method is called very frequently, and should generally
+        return quickly.  The connection is expected to cache change
+        objects and automatically update them as related events are
+        received.
+
+        """
 
     @abc.abstractmethod
     def getChangeByURL(self, url):
@@ -91,7 +98,15 @@ class BaseSource(object, metaclass=abc.ABCMeta):
 
     @abc.abstractmethod
     def getProjectBranches(self, project, tenant):
-        """Get branches for a project"""
+        """Get branches for a project
+
+        This method is called very frequently, and should generally
+        return quickly.  The connection is expected to cache branch
+        lists for all projects queried, and further, to automatically
+        clear or update that cache when it observes branch creation or
+        deletion events.
+
+        """
 
     @abc.abstractmethod
     def getRequireFilters(self, config):