summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJulia Kreger <juliaashleykreger@gmail.com>2021-01-12 13:50:14 -0800
committerJulia Kreger <juliaashleykreger@gmail.com>2021-01-12 13:50:14 -0800
commitd44b3b6fec932fc79ffa5de4ded98d6110d69ef1 (patch)
tree37c1ea0c5fab355515be45a42ca7dcdd95d581e2
parent07bdccea58daf7dd2e10e00db233e707489c06e4 (diff)
downloadironic-d44b3b6fec932fc79ffa5de4ded98d6110d69ef1.tar.gz
CI: Collect a snapshot of network connections
A recent magnum bug (https://storyboard.openstack.org/#!/story/2008494) when running with uwsgi has yielded an interesting question if Ironic is orphaning rpc clients or not. Ironic's code is slightly different but also very similar, and similar bugs have been observed in the past where the python garbage collection never cleans up the old connection objects and effectively orphans the connection. So we should likely at least try to collect some of the information so we can determine if this is the case in our CI jobs. Hence this change and attempt to collect that data after running CI. Change-Id: I4e80c56449c0b3d07b160ae6c933a7a16c63c5c5
-rw-r--r--playbooks/ci-workarounds/get_extra_logging.yaml8
-rw-r--r--playbooks/ci-workarounds/get_tftpd.yaml5
-rw-r--r--zuul.d/ironic-jobs.yaml2
3 files changed, 9 insertions, 6 deletions
diff --git a/playbooks/ci-workarounds/get_extra_logging.yaml b/playbooks/ci-workarounds/get_extra_logging.yaml
new file mode 100644
index 000000000..5f6a1f8fe
--- /dev/null
+++ b/playbooks/ci-workarounds/get_extra_logging.yaml
@@ -0,0 +1,8 @@
+- hosts: all
+ tasks:
+ - name: Get tftpd info from journald
+ shell: "journalctl -t in.tftpd > {{ zuul_output_dir }}/logs/tftpd-journal.txt"
+ become: yes
+ - name: Get network connection information for ironic process
+ shell: "netstat -apn |grep ironic > {{ zuul_output_dir }}/logs/post-job-network-connections.txt"
+ become: yes
diff --git a/playbooks/ci-workarounds/get_tftpd.yaml b/playbooks/ci-workarounds/get_tftpd.yaml
deleted file mode 100644
index 8a0e1f3b4..000000000
--- a/playbooks/ci-workarounds/get_tftpd.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-- hosts: all
- tasks:
- - name: Get tftpd info from journald
- shell: "journalctl -t in.tftpd > {{ zuul_output_dir }}/logs/tftpd-journal.txt"
- become: yes \ No newline at end of file
diff --git a/zuul.d/ironic-jobs.yaml b/zuul.d/ironic-jobs.yaml
index 489370a5b..c6f3ef854 100644
--- a/zuul.d/ironic-jobs.yaml
+++ b/zuul.d/ironic-jobs.yaml
@@ -4,7 +4,7 @@
description: Base job for devstack/tempest based ironic jobs.
parent: devstack-tempest
nodeset: openstack-single-node-focal
- post-run: playbooks/ci-workarounds/get_tftpd.yaml
+ post-run: playbooks/ci-workarounds/get_extra_logging.yaml
timeout: 10800
required-projects:
- openstack/ironic