diff options
| author | Justus Rossmeier <veecue@ventos.tk> | 2020-05-09 18:30:43 +0200 |
|---|---|---|
| committer | Justus Rossmeier <veecue@ventos.tk> | 2020-05-09 19:50:12 +0200 |
| commit | 25bb26b1859fccf132fb7ac298d7f3cb413f0f7e (patch) | |
| tree | 67d41068f80ca2f1e132103ff37c49ad55dad049 /zuul | |
| parent | 8a43c5fddd666a317e0d85116f8265fd53606c1a (diff) | |
| download | zuul-25bb26b1859fccf132fb7ac298d7f3cb413f0f7e.tar.gz | |
executor: Catch error when reading cpu_times
On older Linux Kernels (especially 3.10 shipped with CentOS 7), having
the mount option hidepid=2 set for /proc causes all zombie-processes to
vanish from /proc for non-root users. We rely on /proc to get ansible's
cpu_times however. Currently a fatal psutil.NoSuchProcess exception is
thrown in the given case.
Now, the error is catched and a warning is printed instead. This comes
with the drawback that cpu_time measuring does not work anymore (it will
always print 0s).
Another possible solution for this would be to read the cpu_times of
/proc/self before and after the execution of the ansible subprocess.
This is the behaviour that is present in GNU time for example. For the
described edge-case with an old kernel, this is considered overkill.
Change-Id: I776f8588e1b0f6f3d145b0c6c54c8970fe864251
Diffstat (limited to 'zuul')
| -rw-r--r-- | zuul/executor/server.py | 28 |
1 files changed, 17 insertions, 11 deletions
diff --git a/zuul/executor/server.py b/zuul/executor/server.py index ac137a77f..381dc53d3 100644 --- a/zuul/executor/server.py +++ b/zuul/executor/server.py @@ -19,6 +19,7 @@ import json import logging import multiprocessing import os +import psutil import shutil import signal import shlex @@ -2231,17 +2232,22 @@ class AnsibleJob(object): line = line[:1024].rstrip() ansible_log.debug("Ansible output: %s" % (line,)) self.log.debug("Ansible output terminated") - cpu_times = self.proc.cpu_times() - self.log.debug("Ansible cpu times: user=%.2f, system=%.2f, " - "children_user=%.2f, " - "children_system=%.2f" % - (cpu_times.user, cpu_times.system, - cpu_times.children_user, - cpu_times.children_system)) - self.cpu_times['user'] += cpu_times.user - self.cpu_times['system'] += cpu_times.system - self.cpu_times['children_user'] += cpu_times.children_user - self.cpu_times['children_system'] += cpu_times.children_system + try: + cpu_times = self.proc.cpu_times() + self.log.debug("Ansible cpu times: user=%.2f, system=%.2f, " + "children_user=%.2f, " + "children_system=%.2f" % + (cpu_times.user, cpu_times.system, + cpu_times.children_user, + cpu_times.children_system)) + self.cpu_times['user'] += cpu_times.user + self.cpu_times['system'] += cpu_times.system + self.cpu_times['children_user'] += cpu_times.children_user + self.cpu_times['children_system'] += cpu_times.children_system + except psutil.NoSuchProcess: + self.log.warn("Cannot get cpu_times for process %d. Is your" + "/proc mounted with hidepid=2" + " on an old linux kernel?", self.proc.pid) ret = self.proc.wait() self.log.debug("Ansible exit code: %s" % (ret,)) finally: |