Add proc to bubblewrap

And set the AFS pag.

We would like to use AFS within our playbooks (generally in trusted
jobs on the executor).  Ideally, such usage should be, like everything
else in bubblewrap, completely separate from any other processes.
However, by default OpenAFS stores authentication credentials by UID,
meaning that once any process obtained tokens, any other process on
the executor would be able to use them.

Fortunately, the concept of a PAG (process authentication group) helps
us out here.  That scopes tokens to a single process and its children.

Normally this is done by PAM when a user logs in, but there is an ioctl
that we can use to request a new PAG at any time.  It is this method that
we use to ensure each ansible process runs in its own PAG.

When a new PAG is created, it is actually bound to the *thread* that
created it.  Because of this, we don't need to be concerned with thread
synchronization around PAG creation.  This is useful in the executor which
has potentially hundreds of threads in various stages of preparing to
execute a subprocess.  It is sufficient to request the new PAG at any time
before the Popen call, and that thread will use it during the next
invocation.

The --proc argument is added to the bubblewrap invocation in order to
permit aklog to run (it needs to access /proc/fs/openafs/afs_ioctl
in order to store the tokens).

Change-Id: I2687629f964af11c9da261875f2ec735082b8836

1 files changed, 12 insertions, 0 deletions

diff --git a/zuul/driver/bubblewrap/__init__.py b/zuul/driver/bubblewrap/__init__.py
index a8969cc18..149874b6d 100644
--- a/zuul/driver/bubblewrap/__init__.py
+++ b/zuul/driver/bubblewrap/__init__.py
@@ -15,6 +15,7 @@
 # under the License.
 
 import argparse
+import fcntl
 import grp
 import logging
 import os
@@ -24,6 +25,7 @@ import subprocess
 import sys
 import threading
 import re
+import struct
 
 from typing import Dict, List  # flake8: noqa
 
@@ -83,7 +85,16 @@ class BubblewrapExecutionContext(BaseExecutionContext):
         os.write(pipe, data)
         os.close(pipe)
 
+    def setpag(self):
+        # If we are on a system with AFS, ensure that each playbook
+        # invocation ends up in its own PAG.
+        # http://asa.scripts.mit.edu/trac/attachment/ticket/145/setpag.txt#L315
+        if os.path.exists("/proc/fs/openafs/afs_ioctl"):
+            fcntl.ioctl(open("/proc/fs/openafs/afs_ioctl"), 0x40084301,
+                        struct.pack("lllll", 0, 0, 0, 0, 21))
+
     def getPopen(self, **kwargs):
+        self.setpag()
         # Set zuul_dir if it was not passed in
         if 'zuul_dir' in kwargs:
             zuul_dir = kwargs['zuul_dir']
@@ -186,6 +197,7 @@ class BubblewrapDriver(Driver, WrapperInterface):
             '--ro-bind', '/etc/hosts', '/etc/hosts',
             '--ro-bind', '{ssh_auth_sock}', '{ssh_auth_sock}',
             '--bind', '{work_dir}', '{work_dir}',
+            '--proc', '/proc',
             '--dev', '/dev',
             '--chdir', '{work_dir}',
             '--unshare-all',