Check out more appropriate branches of role and playbook repos

Currently when a job adds a zuul role repo to a playbook, we only
use the master branch of the role repo, unless the role repo
appears in the dependency chain for the change under test.

That means that if the role repo appears in required-projects,
but not as a dependency, then we use the master branch instead of
what was specified in required-projects.  That doesn't seem to make
much sense and is likely an oversight.  We attempt to use the
prepared repos where possible (ie, the requested branches match
and the playbook is not trusted).  However, the current check for
that only looks at 'items', that is, the dependency chain.  Instead,
we should look at 'projects', which includes not only the projects
which appear in 'items', but also those that appear in
required-projects.

The same check is performed for playbooks, and therefore is also
updated.

Also, in the case where a role repo doesn't appear in either the
dependency chain or in required-projects, we were hard-coded to
check out the 'master' branch.  Instead, re-use some of the logic
used when preparing required-projects to attempt to find the best
branch to check out.  We will try the job override branch first,
then the zuul branch, then the project default branch.

All playbook project repos are now prepared outside of the work dir,
even in cases where their projects also appear in the work dir.  If
the playbook is untrusted, then the repo is cloned into the "untrusted/"
jobdir directory (with speculative changes applied).  To account for
this, the "allow_trusted" flag in the ansible safe path checker is
updated to allow access to both "trusted/" and "untrusted/" paths.

Change-Id: If95a9b0aaff982040cd4e6e957f9588b26ef7935

1 files changed, 15 insertions, 0 deletions

diff --git a/releasenotes/notes/role-checkouts-89632d2ff5eb8b78.yaml b/releasenotes/notes/role-checkouts-89632d2ff5eb8b78.yaml
new file mode 100644
index 000000000..86203c5b0
--- /dev/null
+++ b/releasenotes/notes/role-checkouts-89632d2ff5eb8b78.yaml
@@ -0,0 +1,15 @@
+---
+fixes:
+  - |
+    Zuul role repository checkouts now honor :attr:`job.override-checkout`.
+
+    Previously, when a Zuul role was specified for a job, Zuul would
+    usually checkout the master branch, unless that repository
+    appeared in the dependency chain for a patch.  It will now follow
+    the usual procedure for determining the branch to check out,
+    including honoring :attr:`job.override-checkout` options.
+
+    This may alter the behavior of currently existing jobs.  Depending
+    on circumstances, you may need to set
+    :attr:`job.override-checkout` or copy roles to other branches of
+    projects.