blob: c3135debe6ebc399ffa99f4cad221088cf85a1f4 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
|
Ansible Integration
===================
Zuul contains Ansible modules and plugins to control the execution of Ansible
Job content.
Zuul provides realtime build log streaming to end users so that users
can watch long-running jobs in progress.
Streaming job output
--------------------
All jobs run with the :py:mod:`zuul.ansible.base.callback.zuul_stream` callback
plugin enabled, which writes the build log to a file so that the
:py:class:`zuul.lib.log_streamer.LogStreamer` can provide the data on demand
over the finger protocol. Finally, :py:class:`zuul.web.LogStreamHandler`
exposes that log stream over a websocket connection as part of
:py:class:`zuul.web.ZuulWeb`.
.. autoclass:: zuul.ansible.base.callback.zuul_stream.CallbackModule
:members:
.. autoclass:: zuul.lib.log_streamer.LogStreamer
.. autoclass:: zuul.web.LogStreamHandler
.. autoclass:: zuul.web.ZuulWeb
In addition to real-time streaming, Zuul also installs another callback module,
:py:mod:`zuul.ansible.base.callback.zuul_json.CallbackModule` that collects all
of the information about a given run into a json file which is written to the
work dir so that it can be published along with build logs.
.. autoclass:: zuul.ansible.base.callback.zuul_json.CallbackModule
Since the streaming log is by necessity a single text stream, choices
have to be made for readability about what data is shown and what is
not shown. The json log file is intended to allow for a richer more
interactive set of data to be displayed to the user.
.. _zuul_console_streaming:
Capturing live command output
-----------------------------
As jobs may execute long-running shell scripts or other commands,
additional effort is expended to stream ``stdout`` and ``stderr`` of
shell tasks as they happen rather than waiting for the command to
finish.
The global job configuration should run the ``zuul_console`` task as a
very early prerequisite step.
.. automodule:: zuul.ansible.base.library.zuul_console
This will start a daemon that listens on TCP port 19885 on the testing
node. This daemon can be queried to stream back the output of shell
tasks as described below.
Zuul contains a modified version of Ansible's
:ansible:module:`command` module that overrides the default
implementation.
.. automodule:: zuul.ansible.base.library.command
This library will capture the output of the running
command and write it to a temporary file on the host the command is
running on. These files are named in the format
``/tmp/console-<uuid>-<task_id>-<host>.log``
The ``zuul_stream`` callback mentioned above will send a request to
the remote ``zuul_console`` daemon, providing the uuid and task id of
the task it is currently processing. The ``zuul_console`` daemon will
then read the logfile from disk and stream the data back as it
appears, which ``zuul_stream`` will then present as described above.
The ``zuul_stream`` callback will indicate to the ``zuul_console``
daemon when it has finished reading the task, which prompts the remote
side to remove the temporary streaming output files. In some cases,
aborting the Ansible process may not give the ``zuul_stream`` callback
the chance to send this notice, leaking the temporary files. If nodes
are ephemeral this makes little difference, but these files may be
visible on static nodes.
|
