diff options
50 files changed, 775 insertions, 715 deletions
@@ -18,10 +18,9 @@ Compute Engine [images](https://cloud.google.com/compute/docs/images). * [Daemons](#daemons) * [Accounts](#accounts) * [Clock Skew](#clock-skew) - * [IP Forwarding](#ip-forwarding) + * [Network](#network) * [Instance Setup](#instance-setup) * [Metadata Scripts](#metadata-scripts) -* [Network Setup](#network-setup) * [Configuration](#configuration) * [Packaging](#packaging) * [Version Updates](#version-updates) @@ -52,11 +51,11 @@ The guest environment is made up of the following components: * **Disk expand** scripts to expand the VM root partition for CentOS 6, CentOS 7, RHEL 6, and RHEL 7 images. * **Instance setup** scripts to execute VM configuration scripts during boot. -* **IP forwarding** daemon that integrates network load balancing with +* **Network** daemon that handles network setup for multiple network interfaces + on boot and integrates network load balancing with forwarding rule changes into the guest. * **Metadata scripts** to run user-provided scripts at VM startup and shutdown. -* **Network setup** service to enable multiple network interfaces on boot. The Linux guest environment is written in Python and is version agnostic between Python 2.6 and 3.5. There is complete unittest coverage for every Python @@ -173,15 +172,18 @@ The clock skew daemon is responsible for syncing the software clock with the hypervisor clock after a stop/start event or after a migration. Preventing clock skew may result in `system time has changed` messages in VM logs. -#### IP Forwarding +#### Network -The IP forwarding daemon uses IP forwarding metadata to setup or remove IP -routes in the guest. +The network daemon uses network interface metadata to manage the network +interfaces in the guest by performing the following tasks: -* Only IPv4 IP addresses are currently supported. -* Routes are set on the default Ethernet interface determined dynamically. -* Google routes are configured, by default, with the routing protocol ID `66`. - This ID is a namespace for daemon configured IP addresses. +* Enabled all associated network interfaces on boot. Network interfaces are + specified by MAC address in instance metadata. +* Uses IP forwarding metadata to setup or remove IP routes in the guest. + * Only IPv4 IP addresses are currently supported. + * Routes are set on the default Ethernet interface determined dynamically. + * Google routes are configured, by default, with the routing protocol ID + `66`. This ID is a namespace for daemon configured IP addresses. ## Instance Setup @@ -215,12 +217,6 @@ design details. `startup-script-url`) a URL is executed first. * The exit status of a metadata script is logged after completed execution. -## Network Setup - -A network setup service runs on boot and enables all associated network -interfaces. Network interfaces are specified by MAC address in instance -metadata. - ## Configuration Users of Google provided images may configure the guest environment behaviors @@ -246,7 +242,8 @@ Accounts | usermod\_cmd | Command string to modify a user's g Accounts | groupadd\_cmd | Command string to create a new group. Daemons | accounts\_daemon | `false` disables the accounts daemon. Daemons | clock\_skew\_daemon | `false` disables the clock skew daemon. -Daemons | ip\_forwarding\_daemon | `false` disables the IP forwarding daemon. +Daemons | ip\_forwarding\_daemon | `false` (deprecated) skips IP forwarding. +Daemons | network\_daemon | `false` disables the network daemon. InstanceSetup | host\_key\_types | Comma separated list of host key types to generate. InstanceSetup | optimize\_local\_ssd | `false` prevents optimizing for local SSD. InstanceSetup | network\_enabled | `false` skips instance setup functions that require metadata. @@ -259,9 +256,10 @@ IpForwarding | target\_instance\_ips | `false` disables internal IP addres MetadataScripts | run\_dir | String base directory where metadata scripts are executed. MetadataScripts | startup | `false` disables startup script execution. MetadataScripts | shutdown | `false` disables shutdown script execution. +NetworkInterfaces | setup | `false` skips network interface setup. +NetworkInterfaces | ip\_forwarding | `false` skips IP forwarding. NetworkInterfaces | dhclient\_script | String path to a dhclient script used by dhclient. NetworkInterfaces | dhcp\_command | String to execute to enable network interfaces. -NetworkInterfaces | setup | `false` disables network interface setup. Setting `network_enabled` to `false` will skip setting up host keys and the `boto` config in the guest. The setting may also prevent startup and shutdown diff --git a/debian/control b/debian/control index 3aecfaa..5bd9fc4 100644 --- a/debian/control +++ b/debian/control @@ -13,7 +13,8 @@ Build-Depends: debhelper (>= 9), python3-pytest, python-mock, python-boto, - python3-boto + python3-boto, + python3-distro Standards-Version: 3.9.8 Homepage: https://github.com/GoogleCloudPlatform/compute-image-packages diff --git a/debian/google-compute-engine.postinst b/debian/google-compute-engine.postinst index f1dbf03..94f65aa 100644 --- a/debian/google-compute-engine.postinst +++ b/debian/google-compute-engine.postinst @@ -27,8 +27,7 @@ if [ -d /run/systemd/system ]; then fi deb-systemd-invoke $_dh_action \ google-instance-setup.service \ - google-network-setup.service \ google-accounts-daemon.service \ google-clock-skew-daemon.service \ - google-ip-forwarding-daemon.service >/dev/null || true + google-network-daemon.service >/dev/null || true fi diff --git a/debian/google-compute-engine.prerm b/debian/google-compute-engine.prerm index 73747eb..44e2c7a 100644 --- a/debian/google-compute-engine.prerm +++ b/debian/google-compute-engine.prerm @@ -18,10 +18,22 @@ set -e if [ -d /run/systemd/system ] && [ "$1" = remove ]; then deb-systemd-invoke stop \ google-instance-setup.service \ - google-network-setup.service \ google-accounts-daemon.service \ google-clock-skew-daemon.service \ - google-ip-forwarding-daemon.service \ + google-network-daemon.service \ google-shutdown-scripts.service \ google-startup-scripts.service >/dev/null + + # Remove old service. + if [ -f /lib/systemd/system/google-ip-forwarding-daemon.service ]; then + deb-systemd-invoke stop google-ip-forwarding-daemon.service + systemctl --no-reload disable google-ip-forwarding-daemon.service + rm /lib/systemd/system/google-ip-forwarding-daemon.service + fi + + if [ -f /lib/systemd/system/google-network-setup.service ]; then + deb-systemd-invoke stop google-network-setup.service + systemctl --no-reload disable google-network-setup.service + rm /lib/systemd/system/google-network-setup.service + fi fi diff --git a/google_compute_engine/instance_setup/instance_config.py b/google_compute_engine/instance_setup/instance_config.py index 9827137..c71a3ac 100644 --- a/google_compute_engine/instance_setup/instance_config.py +++ b/google_compute_engine/instance_setup/instance_config.py @@ -66,7 +66,8 @@ class InstanceConfig(config_manager.ConfigManager): 'Daemons': { 'accounts_daemon': 'true', 'clock_skew_daemon': 'true', - 'ip_forwarding_daemon': 'true', + 'ip_forwarding_daemon': 'true', # Deprecated. + 'network_daemon': 'true', }, 'Instance': { 'instance_id': '0', @@ -91,8 +92,9 @@ class InstanceConfig(config_manager.ConfigManager): }, 'NetworkInterfaces': { 'setup': 'true', - 'dhclient_script': '/sbin/google-dhclient-script', + 'ip_forwarding': 'true', 'dhcp_command': '', + 'dhclient_script': '/sbin/google-dhclient-script', }, } diff --git a/google_compute_engine/ip_forwarding/tests/ip_forwarding_daemon_test.py b/google_compute_engine/ip_forwarding/tests/ip_forwarding_daemon_test.py deleted file mode 100644 index 85e5b57..0000000 --- a/google_compute_engine/ip_forwarding/tests/ip_forwarding_daemon_test.py +++ /dev/null @@ -1,280 +0,0 @@ -#!/usr/bin/python -# Copyright 2016 Google Inc. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Unittest for ip_forwarding_daemon.py module.""" - -from google_compute_engine.ip_forwarding import ip_forwarding_daemon -from google_compute_engine.test_compat import mock -from google_compute_engine.test_compat import unittest - - -class IpForwardingDaemonTest(unittest.TestCase): - - def setUp(self): - self.mock_logger = mock.Mock() - self.mock_watcher = mock.Mock() - self.mock_ip_forwarding_utils = mock.Mock() - self.mock_network_utils = mock.Mock() - - self.mock_setup = mock.create_autospec( - ip_forwarding_daemon.IpForwardingDaemon) - self.mock_setup.logger = self.mock_logger - self.mock_setup.watcher = self.mock_watcher - self.mock_setup.ip_forwarding_utils = self.mock_ip_forwarding_utils - self.mock_setup.network_utils = self.mock_network_utils - - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.ip_forwarding_utils') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.network_utils') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.metadata_watcher') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.logger') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.file_utils') - def testIpForwardingDaemon( - self, mock_lock, mock_logger, mock_watcher, mock_network_utils, - mock_ip_forwarding_utils): - mock_logger_instance = mock.Mock() - mock_logger.Logger.return_value = mock_logger_instance - mocks = mock.Mock() - mocks.attach_mock(mock_lock, 'lock') - mocks.attach_mock(mock_logger, 'logger') - mocks.attach_mock(mock_network_utils, 'network') - mocks.attach_mock(mock_ip_forwarding_utils, 'forwarding') - mocks.attach_mock(mock_watcher, 'watcher') - metadata_key = ip_forwarding_daemon.IpForwardingDaemon.network_interfaces - with mock.patch.object( - ip_forwarding_daemon.IpForwardingDaemon, - 'HandleNetworkInterfaces') as mock_handle: - ip_forwarding_daemon.IpForwardingDaemon(proto_id='66', debug=True) - expected_calls = [ - mock.call.logger.Logger(name=mock.ANY, debug=True, facility=mock.ANY), - mock.call.watcher.MetadataWatcher(logger=mock_logger_instance), - mock.call.network.NetworkUtils(logger=mock_logger_instance), - mock.call.forwarding.IpForwardingUtils( - logger=mock_logger_instance, proto_id='66'), - mock.call.lock.LockFile(ip_forwarding_daemon.LOCKFILE), - mock.call.lock.LockFile().__enter__(), - mock.call.logger.Logger().info(mock.ANY), - mock.call.watcher.MetadataWatcher().WatchMetadata( - mock_handle, metadata_key=metadata_key, recursive=True, - timeout=mock.ANY), - mock.call.lock.LockFile().__exit__(None, None, None), - ] - self.assertEqual(mocks.mock_calls, expected_calls) - - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.ip_forwarding_utils') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.network_utils') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.metadata_watcher') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.logger') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_daemon.file_utils') - def testIpForwardingDaemonError( - self, mock_lock, mock_logger, mock_watcher, mock_network_utils, - mock_ip_forwarding_utils): - mock_logger_instance = mock.Mock() - mock_logger.Logger.return_value = mock_logger_instance - mocks = mock.Mock() - mocks.attach_mock(mock_lock, 'lock') - mocks.attach_mock(mock_logger, 'logger') - mocks.attach_mock(mock_network_utils, 'network') - mocks.attach_mock(mock_ip_forwarding_utils, 'forwarding') - mocks.attach_mock(mock_watcher, 'watcher') - mock_lock.LockFile.side_effect = IOError('Test Error') - with mock.patch.object( - ip_forwarding_daemon.IpForwardingDaemon, 'HandleNetworkInterfaces'): - ip_forwarding_daemon.IpForwardingDaemon() - expected_calls = [ - mock.call.logger.Logger( - name=mock.ANY, debug=False, facility=mock.ANY), - mock.call.watcher.MetadataWatcher(logger=mock_logger_instance), - mock.call.network.NetworkUtils(logger=mock_logger_instance), - mock.call.forwarding.IpForwardingUtils( - logger=mock_logger_instance, proto_id=None), - mock.call.lock.LockFile(ip_forwarding_daemon.LOCKFILE), - mock.call.logger.Logger().warning('Test Error'), - ] - self.assertEqual(mocks.mock_calls, expected_calls) - - def testLogForwardedIpChanges(self): - ip_forwarding_daemon.IpForwardingDaemon._LogForwardedIpChanges( - self.mock_setup, [], [], [], [], '1') - ip_forwarding_daemon.IpForwardingDaemon._LogForwardedIpChanges( - self.mock_setup, ['a'], ['a'], [], [], '2') - ip_forwarding_daemon.IpForwardingDaemon._LogForwardedIpChanges( - self.mock_setup, ['a'], [], [], ['a'], '3') - ip_forwarding_daemon.IpForwardingDaemon._LogForwardedIpChanges( - self.mock_setup, ['a', 'b'], ['b'], [], ['a'], '4') - ip_forwarding_daemon.IpForwardingDaemon._LogForwardedIpChanges( - self.mock_setup, ['a'], ['b'], ['b'], ['a'], '5') - expected_calls = [ - mock.call.info(mock.ANY, '3', ['a'], None, None, ['a']), - mock.call.info(mock.ANY, '4', ['a', 'b'], ['b'], None, ['a']), - mock.call.info(mock.ANY, '5', ['a'], ['b'], ['b'], ['a']), - ] - self.assertEqual(self.mock_logger.mock_calls, expected_calls) - - def testAddForwardedIp(self): - ip_forwarding_daemon.IpForwardingDaemon._AddForwardedIps( - self.mock_setup, [], 'interface') - self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, []) - - ip_forwarding_daemon.IpForwardingDaemon._AddForwardedIps( - self.mock_setup, ['a', 'b', 'c'], 'interface') - expected_calls = [ - mock.call.AddForwardedIp('a', 'interface'), - mock.call.AddForwardedIp('b', 'interface'), - mock.call.AddForwardedIp('c', 'interface'), - ] - self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, expected_calls) - - def testRemoveForwardedIp(self): - ip_forwarding_daemon.IpForwardingDaemon._RemoveForwardedIps( - self.mock_setup, [], 'interface') - self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, []) - - ip_forwarding_daemon.IpForwardingDaemon._RemoveForwardedIps( - self.mock_setup, ['a', 'b', 'c'], 'interface') - expected_calls = [ - mock.call.RemoveForwardedIp('a', 'interface'), - mock.call.RemoveForwardedIp('b', 'interface'), - mock.call.RemoveForwardedIp('c', 'interface'), - ] - self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, expected_calls) - - def testHandleForwardedIps(self): - configured = ['c', 'c', 'b', 'b', 'a', 'a'] - desired = ['d', 'd', 'c'] - mocks = mock.Mock() - mocks.attach_mock(self.mock_ip_forwarding_utils, 'forwarding') - mocks.attach_mock(self.mock_setup, 'setup') - self.mock_ip_forwarding_utils.ParseForwardedIps.return_value = desired - self.mock_ip_forwarding_utils.GetForwardedIps.return_value = configured - forwarded_ips = 'forwarded ips' - interface = 'interface' - expected_add = ['d'] - expected_remove = ['a', 'b'] - - ip_forwarding_daemon.IpForwardingDaemon._HandleForwardedIps( - self.mock_setup, forwarded_ips, interface) - expected_calls = [ - mock.call.forwarding.ParseForwardedIps(forwarded_ips), - mock.call.forwarding.GetForwardedIps(interface), - mock.call.setup._LogForwardedIpChanges( - configured, desired, expected_add, expected_remove, interface), - mock.call.setup._AddForwardedIps(expected_add, interface), - mock.call.setup._RemoveForwardedIps(expected_remove, interface), - ] - self.assertEqual(mocks.mock_calls, expected_calls) - - def testHandleNetworkInterfaces(self): - self.mock_setup.ip_aliases = False - self.mock_setup.target_instance_ips = False - mocks = mock.Mock() - mocks.attach_mock(self.mock_network_utils, 'network') - mocks.attach_mock(self.mock_setup, 'setup') - self.mock_network_utils.GetNetworkInterface.side_effect = [ - 'eth0', 'eth1', 'eth2', 'eth3', None] - result = [ - { - 'mac': '1', - 'forwardedIps': ['a'] - }, - { - 'mac': '2', - 'forwardedIps': ['b'], - 'ipAliases': ['banana'], - 'targetInstanceIps': ['baklava'], - }, - { - 'mac': '3', - 'ipAliases': ['cherry'], - 'targetInstanceIps': ['cake'], - }, - { - 'mac': '4' - }, - { - 'forwardedIps': ['d'], - 'ipAliases': ['date'], - 'targetInstanceIps': ['doughnuts'], - }, - ] - - ip_forwarding_daemon.IpForwardingDaemon.HandleNetworkInterfaces( - self.mock_setup, result) - expected_calls = [ - mock.call.network.GetNetworkInterface('1'), - mock.call.setup._HandleForwardedIps(['a'], 'eth0'), - mock.call.network.GetNetworkInterface('2'), - mock.call.setup._HandleForwardedIps(['b'], 'eth1'), - mock.call.network.GetNetworkInterface('3'), - mock.call.setup._HandleForwardedIps([], 'eth2'), - mock.call.network.GetNetworkInterface('4'), - mock.call.setup._HandleForwardedIps([], 'eth3'), - mock.call.network.GetNetworkInterface(None), - mock.call.setup.logger.warning(mock.ANY, None), - ] - self.assertEqual(mocks.mock_calls, expected_calls) - - def testHandleNetworkInterfacesFeatures(self): - self.mock_setup.ip_aliases = True - self.mock_setup.target_instance_ips = True - mocks = mock.Mock() - mocks.attach_mock(self.mock_network_utils, 'network') - mocks.attach_mock(self.mock_setup, 'setup') - self.mock_network_utils.GetNetworkInterface.side_effect = [ - 'eth0', 'eth1', 'eth2', 'eth3', None] - result = [ - { - 'mac': '1', - 'forwardedIps': ['a'] - }, - { - 'mac': '2', - 'forwardedIps': ['b'], - 'ipAliases': ['banana'], - 'targetInstanceIps': ['baklava'], - }, - { - 'mac': '3', - 'ipAliases': ['cherry'], - 'targetInstanceIps': ['cake'], - }, - { - 'mac': '4' - }, - { - 'forwardedIps': ['d'], - 'ipAliases': ['date'], - 'targetInstanceIps': ['doughnuts'], - }, - ] - - ip_forwarding_daemon.IpForwardingDaemon.HandleNetworkInterfaces( - self.mock_setup, result) - expected_calls = [ - mock.call.network.GetNetworkInterface('1'), - mock.call.setup._HandleForwardedIps(['a'], 'eth0'), - mock.call.network.GetNetworkInterface('2'), - mock.call.setup._HandleForwardedIps(['b', 'banana', 'baklava'], 'eth1'), - mock.call.network.GetNetworkInterface('3'), - mock.call.setup._HandleForwardedIps(['cherry', 'cake'], 'eth2'), - mock.call.network.GetNetworkInterface('4'), - mock.call.setup._HandleForwardedIps([], 'eth3'), - mock.call.network.GetNetworkInterface(None), - mock.call.setup.logger.warning(mock.ANY, None), - ] - self.assertEqual(mocks.mock_calls, expected_calls) - - -if __name__ == '__main__': - unittest.main() diff --git a/google_compute_engine/network_setup/tests/network_setup_test.py b/google_compute_engine/network_setup/tests/network_setup_test.py deleted file mode 100644 index 072dc00..0000000 --- a/google_compute_engine/network_setup/tests/network_setup_test.py +++ /dev/null @@ -1,136 +0,0 @@ -#!/usr/bin/python -# Copyright 2016 Google Inc. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Unittest for network_setup.py module.""" - -import shutil -import subprocess -import tempfile - -from google_compute_engine.network_setup import network_setup -from google_compute_engine.test_compat import mock -from google_compute_engine.test_compat import unittest - - -class NetworkSetupTest(unittest.TestCase): - - def setUp(self): - # Create a temporary directory. - self.test_dir = tempfile.mkdtemp() - - self.mock_logger = mock.Mock() - self.mock_watcher = mock.Mock() - self.mock_ip_forwarding_utils = mock.Mock() - self.mock_network_utils = mock.Mock() - self.metadata_key = 'metadata_key' - self.mock_distro_utils = mock.Mock() - - self.mock_setup = mock.create_autospec(network_setup.NetworkSetup) - self.mock_setup.logger = self.mock_logger - self.mock_setup.watcher = self.mock_watcher - self.mock_setup.network_utils = self.mock_network_utils - self.mock_setup.network_interfaces = self.metadata_key - self.mock_setup.distro_utils = self.mock_distro_utils - self.mock_setup.network_path = '/etc/sysconfig/network-scripts' - self.mock_setup.dhclient_script = '/bin/script' - self.mock_setup.dhcp_command = '' - - def tearDown(self): - # Remove the directory after the test. - shutil.rmtree(self.test_dir) - - @mock.patch('google_compute_engine.network_setup.network_setup.network_utils') - @mock.patch('google_compute_engine.network_setup.network_setup.metadata_watcher') - @mock.patch('google_compute_engine.network_setup.network_setup.logger') - def testNetworkSetup(self, mock_logger, mock_watcher, mock_network_utils): - mock_logger_instance = mock.Mock() - mock_logger.Logger.return_value = mock_logger_instance - mocks = mock.Mock() - mocks.attach_mock(mock_logger, 'logger') - mocks.attach_mock(mock_watcher, 'watcher') - mocks.attach_mock(mock_network_utils, 'network') - with mock.patch.object( - network_setup.NetworkSetup, '_SetupNetworkInterfaces'): - - network_setup.NetworkSetup(debug=True) - expected_calls = [ - mock.call.logger.Logger(name=mock.ANY, debug=True, facility=mock.ANY), - mock.call.watcher.MetadataWatcher(logger=mock_logger_instance), - mock.call.network.NetworkUtils(logger=mock_logger_instance), - ] - self.assertEqual(mocks.mock_calls, expected_calls) - - @mock.patch('google_compute_engine.network_setup.network_setup.subprocess.check_call') - def testEnableNetworkInterfaces(self, mock_call): - mocks = mock.Mock() - mocks.attach_mock(mock_call, 'call') - mocks.attach_mock(self.mock_logger, 'logger') - mocks.attach_mock(self.mock_setup.distro_utils.EnableNetworkInterfaces, 'enable') - mock_call.side_effect = [None, subprocess.CalledProcessError(1, 'Test')] - - # Return immediately with fewer than two interfaces. - network_setup.NetworkSetup._EnableNetworkInterfaces(self.mock_setup, None) - network_setup.NetworkSetup._EnableNetworkInterfaces(self.mock_setup, []) - # Enable interfaces with network manager enabled. - network_setup.NetworkSetup._EnableNetworkInterfaces( - self.mock_setup, ['A', 'B']) - # Enable interfaces with network manager is not present. - network_setup.NetworkSetup._EnableNetworkInterfaces( - self.mock_setup, ['C', 'D']) - # Run a user supplied command successfully. - self.mock_setup.dhcp_command = 'success' - network_setup.NetworkSetup._EnableNetworkInterfaces( - self.mock_setup, ['E', 'F']) - # Run a user supplied command and logger error messages. - self.mock_setup.dhcp_command = 'failure' - network_setup.NetworkSetup._EnableNetworkInterfaces( - self.mock_setup, ['G', 'H']) - expected_calls = [ - # First calls with empty `interfaces` were no-ops. - mock.call.enable(['A', 'B'], mock.ANY, dhclient_script='/bin/script'), - mock.call.enable(['C', 'D'], mock.ANY, dhclient_script='/bin/script'), - mock.call.call(['success']), - mock.call.call(['failure']), - mock.call.logger.warning(mock.ANY), - ] - self.assertEqual(mocks.mock_calls, expected_calls) - - def testSetupNetworkInterfaces(self): - mocks = mock.Mock() - mocks.attach_mock(self.mock_logger, 'logger') - mocks.attach_mock(self.mock_watcher, 'watcher') - mocks.attach_mock(self.mock_network_utils, 'network') - mocks.attach_mock(self.mock_setup, 'setup') - self.mock_watcher.GetMetadata.return_value = [ - {'mac': '1'}, {'mac': '2'}, {'mac': '3'}, {}] - self.mock_network_utils.GetNetworkInterface.side_effect = [ - 'eth0', 'eth1', None, None] - with mock.patch.object( - network_setup.NetworkSetup, '_EnableNetworkInterfaces'): - self.mock_setup.dhcp_command = 'command' - - network_setup.NetworkSetup._SetupNetworkInterfaces(self.mock_setup) - expected_calls = [ - mock.call.watcher.GetMetadata( - metadata_key=self.metadata_key, recursive=True), - mock.call.network.GetNetworkInterface('1'), - mock.call.network.GetNetworkInterface('2'), - mock.call.network.GetNetworkInterface('3'), - mock.call.logger.warning(mock.ANY, '3'), - mock.call.network.GetNetworkInterface(None), - mock.call.logger.warning(mock.ANY, None), - mock.call.setup._EnableNetworkInterfaces(['eth0', 'eth1']), - ] - self.assertEqual(mocks.mock_calls, expected_calls) diff --git a/google_compute_engine/ip_forwarding/__init__.py b/google_compute_engine/networking/__init__.py index e69de29..e69de29 100644 --- a/google_compute_engine/ip_forwarding/__init__.py +++ b/google_compute_engine/networking/__init__.py diff --git a/google_compute_engine/network_setup/__init__.py b/google_compute_engine/networking/ip_forwarding/__init__.py index e69de29..e69de29 100644 --- a/google_compute_engine/network_setup/__init__.py +++ b/google_compute_engine/networking/ip_forwarding/__init__.py diff --git a/google_compute_engine/ip_forwarding/ip_forwarding_daemon.py b/google_compute_engine/networking/ip_forwarding/ip_forwarding.py index aa44098..3c92294 100755 --- a/google_compute_engine/ip_forwarding/ip_forwarding_daemon.py +++ b/google_compute_engine/networking/ip_forwarding/ip_forwarding.py @@ -15,10 +15,9 @@ """Manage IP forwarding on a Google Compute Engine instance. -Fetch a list of public endpoint IPs from the metadata server, compare it with -the IPs configured the associated interfaces, and add or remove addresses from -the interfaces to make them match. Only remove those which match our proto -code. +When given a list of public endpoint IPs, compare it with the IPs configured +for the associated interfaces, and add or remove addresses from the interfaces +to make them match. Only remove those which match our proto code. Command used to add IPs: ip route add to local $IP/32 dev eth0 proto 66 @@ -27,54 +26,26 @@ Command used to fetch list of configured IPs: """ import logging.handlers -import optparse -import random -from google_compute_engine import config_manager -from google_compute_engine import constants -from google_compute_engine import file_utils from google_compute_engine import logger -from google_compute_engine import metadata_watcher -from google_compute_engine import network_utils -from google_compute_engine.ip_forwarding import ip_forwarding_utils +from google_compute_engine.networking.ip_forwarding import ip_forwarding_utils -LOCKFILE = constants.LOCALSTATEDIR + '/lock/google_ip_forwarding.lock' - -class IpForwardingDaemon(object): +class IpForwarding(object): """Manage IP forwarding based on changes to forwarded IPs metadata.""" - network_interfaces = 'instance/network-interfaces' - - def __init__( - self, proto_id=None, ip_aliases=True, target_instance_ips=True, - debug=False): + def __init__(self, proto_id=None, debug=False): """Constructor. Args: proto_id: string, the routing protocol identifier for Google IP changes. - ip_aliases: bool, True if the guest should configure IP alias routes. - target_instance_ips: bool, True supports internal IP load balancing. debug: bool, True if debug output should write to the console. """ facility = logging.handlers.SysLogHandler.LOG_DAEMON self.logger = logger.Logger( name='google-ip-forwarding', debug=debug, facility=facility) - self.watcher = metadata_watcher.MetadataWatcher(logger=self.logger) - self.network_utils = network_utils.NetworkUtils(logger=self.logger) self.ip_forwarding_utils = ip_forwarding_utils.IpForwardingUtils( logger=self.logger, proto_id=proto_id) - self.ip_aliases = ip_aliases - self.target_instance_ips = target_instance_ips - try: - with file_utils.LockFile(LOCKFILE): - self.logger.info('Starting Google IP Forwarding daemon.') - timeout = 60 + random.randint(0, 30) - self.watcher.WatchMetadata( - self.HandleNetworkInterfaces, metadata_key=self.network_interfaces, - recursive=True, timeout=timeout) - except (IOError, OSError) as e: - self.logger.warning(str(e)) def _LogForwardedIpChanges( self, configured, desired, to_add, to_remove, interface): @@ -114,12 +85,12 @@ class IpForwardingDaemon(object): for address in forwarded_ips: self.ip_forwarding_utils.RemoveForwardedIp(address, interface) - def _HandleForwardedIps(self, forwarded_ips, interface): + def HandleForwardedIps(self, interface, forwarded_ips): """Handle changes to the forwarded IPs on a network interface. Args: - forwarded_ips: list, the forwarded IP address strings desired. interface: string, the output device to configure. + forwarded_ips: list, the forwarded IP address strings desired. """ desired = self.ip_forwarding_utils.ParseForwardedIps(forwarded_ips) configured = self.ip_forwarding_utils.GetForwardedIps(interface) @@ -129,46 +100,3 @@ class IpForwardingDaemon(object): configured, desired, to_add, to_remove, interface) self._AddForwardedIps(to_add, interface) self._RemoveForwardedIps(to_remove, interface) - - def HandleNetworkInterfaces(self, result): - """Called when network interface metadata changes. - - Args: - result: dict, the metadata response with the new network interfaces. - """ - for network_interface in result: - mac_address = network_interface.get('mac') - interface = self.network_utils.GetNetworkInterface(mac_address) - ip_addresses = [] - if interface: - ip_addresses.extend(network_interface.get('forwardedIps', [])) - if self.ip_aliases: - ip_addresses.extend(network_interface.get('ipAliases', [])) - if self.target_instance_ips: - ip_addresses.extend(network_interface.get('targetInstanceIps', [])) - self._HandleForwardedIps(ip_addresses, interface) - else: - message = 'Network interface not found for MAC address: %s.' - self.logger.warning(message, mac_address) - - -def main(): - parser = optparse.OptionParser() - parser.add_option( - '-d', '--debug', action='store_true', dest='debug', - help='print debug output to the console.') - (options, _) = parser.parse_args() - instance_config = config_manager.ConfigManager() - if instance_config.GetOptionBool('Daemons', 'ip_forwarding_daemon'): - IpForwardingDaemon( - proto_id=instance_config.GetOptionString( - 'IpForwarding', 'ethernet_proto_id'), - ip_aliases=instance_config.GetOptionBool( - 'IpForwarding', 'ip_aliases'), - target_instance_ips=instance_config.GetOptionBool( - 'IpForwarding', 'target_instance_ips'), - debug=bool(options.debug)) - - -if __name__ == '__main__': - main() diff --git a/google_compute_engine/ip_forwarding/ip_forwarding_utils.py b/google_compute_engine/networking/ip_forwarding/ip_forwarding_utils.py index c131144..c131144 100644 --- a/google_compute_engine/ip_forwarding/ip_forwarding_utils.py +++ b/google_compute_engine/networking/ip_forwarding/ip_forwarding_utils.py diff --git a/google_compute_engine/networking/ip_forwarding/tests/__init__.py b/google_compute_engine/networking/ip_forwarding/tests/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/google_compute_engine/networking/ip_forwarding/tests/__init__.py diff --git a/google_compute_engine/networking/ip_forwarding/tests/ip_forwarding_test.py b/google_compute_engine/networking/ip_forwarding/tests/ip_forwarding_test.py new file mode 100644 index 0000000..f5a2fc9 --- /dev/null +++ b/google_compute_engine/networking/ip_forwarding/tests/ip_forwarding_test.py @@ -0,0 +1,120 @@ +#!/usr/bin/python +# Copyright 2018 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Unittest for ip_forwarding.py module.""" + +from google_compute_engine.networking.ip_forwarding import ip_forwarding +from google_compute_engine.test_compat import mock +from google_compute_engine.test_compat import unittest + + +class IpForwardingTest(unittest.TestCase): + + def setUp(self): + self.mock_logger = mock.Mock() + self.mock_watcher = mock.Mock() + self.mock_ip_forwarding_utils = mock.Mock() + self.mock_setup = mock.create_autospec(ip_forwarding.IpForwarding) + self.mock_setup.logger = self.mock_logger + self.mock_setup.ip_forwarding_utils = self.mock_ip_forwarding_utils + + @mock.patch('google_compute_engine.networking.ip_forwarding.ip_forwarding.ip_forwarding_utils') + @mock.patch('google_compute_engine.networking.ip_forwarding.ip_forwarding.logger') + def testIpForwarding(self, mock_logger, mock_ip_forwarding_utils): + mock_logger_instance = mock.Mock() + mock_logger.Logger.return_value = mock_logger_instance + mocks = mock.Mock() + mocks.attach_mock(mock_logger, 'logger') + mocks.attach_mock(mock_ip_forwarding_utils, 'forwarding') + with mock.patch.object(ip_forwarding.IpForwarding, 'HandleForwardedIps'): + + ip_forwarding.IpForwarding(proto_id='66', debug=True) + expected_calls = [ + mock.call.logger.Logger(name=mock.ANY, debug=True, facility=mock.ANY), + mock.call.forwarding.IpForwardingUtils( + logger=mock_logger_instance, proto_id='66'), + ] + self.assertEqual(mocks.mock_calls, expected_calls) + + def testLogForwardedIpChanges(self): + ip_forwarding.IpForwarding._LogForwardedIpChanges( + self.mock_setup, [], [], [], [], '1') + ip_forwarding.IpForwarding._LogForwardedIpChanges( + self.mock_setup, ['a'], ['a'], [], [], '2') + ip_forwarding.IpForwarding._LogForwardedIpChanges( + self.mock_setup, ['a'], [], [], ['a'], '3') + ip_forwarding.IpForwarding._LogForwardedIpChanges( + self.mock_setup, ['a', 'b'], ['b'], [], ['a'], '4') + ip_forwarding.IpForwarding._LogForwardedIpChanges( + self.mock_setup, ['a'], ['b'], ['b'], ['a'], '5') + expected_calls = [ + mock.call.info(mock.ANY, '3', ['a'], None, None, ['a']), + mock.call.info(mock.ANY, '4', ['a', 'b'], ['b'], None, ['a']), + mock.call.info(mock.ANY, '5', ['a'], ['b'], ['b'], ['a']), + ] + self.assertEqual(self.mock_logger.mock_calls, expected_calls) + + def testAddForwardedIp(self): + ip_forwarding.IpForwarding._AddForwardedIps( + self.mock_setup, [], 'interface') + self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, []) + + ip_forwarding.IpForwarding._AddForwardedIps( + self.mock_setup, ['a', 'b', 'c'], 'interface') + expected_calls = [ + mock.call.AddForwardedIp('a', 'interface'), + mock.call.AddForwardedIp('b', 'interface'), + mock.call.AddForwardedIp('c', 'interface'), + ] + self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, expected_calls) + + def testRemoveForwardedIp(self): + ip_forwarding.IpForwarding._RemoveForwardedIps( + self.mock_setup, [], 'interface') + self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, []) + + ip_forwarding.IpForwarding._RemoveForwardedIps( + self.mock_setup, ['a', 'b', 'c'], 'interface') + expected_calls = [ + mock.call.RemoveForwardedIp('a', 'interface'), + mock.call.RemoveForwardedIp('b', 'interface'), + mock.call.RemoveForwardedIp('c', 'interface'), + ] + self.assertEqual(self.mock_ip_forwarding_utils.mock_calls, expected_calls) + + def testHandleForwardedIps(self): + configured = ['c', 'c', 'b', 'b', 'a', 'a'] + desired = ['d', 'd', 'c'] + mocks = mock.Mock() + mocks.attach_mock(self.mock_ip_forwarding_utils, 'forwarding') + mocks.attach_mock(self.mock_setup, 'setup') + self.mock_ip_forwarding_utils.ParseForwardedIps.return_value = desired + self.mock_ip_forwarding_utils.GetForwardedIps.return_value = configured + forwarded_ips = 'forwarded ips' + interface = 'interface' + expected_add = ['d'] + expected_remove = ['a', 'b'] + + ip_forwarding.IpForwarding.HandleForwardedIps( + self.mock_setup, interface, forwarded_ips) + expected_calls = [ + mock.call.forwarding.ParseForwardedIps(forwarded_ips), + mock.call.forwarding.GetForwardedIps(interface), + mock.call.setup._LogForwardedIpChanges( + configured, desired, expected_add, expected_remove, interface), + mock.call.setup._AddForwardedIps(expected_add, interface), + mock.call.setup._RemoveForwardedIps(expected_remove, interface), + ] + self.assertEqual(mocks.mock_calls, expected_calls) diff --git a/google_compute_engine/ip_forwarding/tests/ip_forwarding_utils_test.py b/google_compute_engine/networking/ip_forwarding/tests/ip_forwarding_utils_test.py index 8d7cf70..59b829c 100644 --- a/google_compute_engine/ip_forwarding/tests/ip_forwarding_utils_test.py +++ b/google_compute_engine/networking/ip_forwarding/tests/ip_forwarding_utils_test.py @@ -15,7 +15,7 @@ """Unittest for ip_forwarding_utils.py module.""" -from google_compute_engine.ip_forwarding import ip_forwarding_utils +from google_compute_engine.networking.ip_forwarding import ip_forwarding_utils from google_compute_engine.test_compat import mock from google_compute_engine.test_compat import unittest @@ -33,7 +33,7 @@ class IpForwardingUtilsTest(unittest.TestCase): self.mock_logger = mock.Mock() self.options = {'hello': 'world'} with mock.patch( - 'google_compute_engine.ip_forwarding.ip_forwarding_utils' + 'google_compute_engine.networking.ip_forwarding.ip_forwarding_utils' '.subprocess') as mock_subprocess: mock_subprocess.Popen.return_value = _CreateMockProcess( 0, b'out', b'') @@ -69,7 +69,7 @@ class IpForwardingUtilsTest(unittest.TestCase): self.mock_utils._CreateRouteOptions(proto='test 1', scope='test 2'), expected_options) - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_utils.subprocess') + @mock.patch('google_compute_engine.networking.ip_forwarding.ip_forwarding_utils.subprocess') def testRunIpRoute(self, mock_subprocess): mock_process = _CreateMockProcess(0, b'out', b'') mock_subprocess.Popen.return_value = mock_process @@ -84,7 +84,7 @@ class IpForwardingUtilsTest(unittest.TestCase): mock_process.communicate.assert_called_once_with() self.mock_logger.warning.assert_not_called() - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_utils.subprocess') + @mock.patch('google_compute_engine.networking.ip_forwarding.ip_forwarding_utils.subprocess') def testRunIpRouteReturnCode(self, mock_subprocess): mock_process = _CreateMockProcess(1, b'out', b'error\n') mock_subprocess.Popen.return_value = mock_process @@ -93,9 +93,10 @@ class IpForwardingUtilsTest(unittest.TestCase): self.mock_utils._RunIpRoute(args=['foo', 'bar'], options=self.options), '') command = ['ip', 'route', 'foo', 'bar', 'hello', 'world'] - self.mock_logger.warning.assert_called_once_with(mock.ANY, command, b'error') + self.mock_logger.warning.assert_called_once_with( + mock.ANY, command, b'error') - @mock.patch('google_compute_engine.ip_forwarding.ip_forwarding_utils.subprocess') + @mock.patch('google_compute_engine.networking.ip_forwarding.ip_forwarding_utils.subprocess') def testRunIpRouteException(self, mock_subprocess): mock_subprocess.Popen.side_effect = OSError('Test Error') diff --git a/google_compute_engine/networking/network_daemon.py b/google_compute_engine/networking/network_daemon.py new file mode 100644 index 0000000..2ce2c0e --- /dev/null +++ b/google_compute_engine/networking/network_daemon.py @@ -0,0 +1,181 @@ +#!/usr/bin/python +# Copyright 2018 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Manage networking on a Google Compute Engine instance. + +Run network setup to enable multiple network interfaces on startup. +Update IP forwarding when metadata changes. +""" + +import logging.handlers +import optparse +import random + +from google_compute_engine import config_manager +from google_compute_engine import constants +from google_compute_engine import file_utils +from google_compute_engine import logger +from google_compute_engine import metadata_watcher +from google_compute_engine import network_utils +from google_compute_engine.networking.ip_forwarding import ip_forwarding +from google_compute_engine.networking.network_setup import network_setup + +LOCKFILE = constants.LOCALSTATEDIR + '/lock/google_networking.lock' + + +class NetworkDaemon(object): + """Manage networking based on changes to network metadata.""" + + network_interface_metadata_key = 'instance/network-interfaces' + + def __init__( + self, ip_forwarding_enabled, proto_id, ip_aliases, target_instance_ips, + dhclient_script, dhcp_command, network_setup_enabled, debug=False): + """Constructor. + + Args: + ip_forwarding_enabled: bool, True if ip forwarding is enabled. + proto_id: string, the routing protocol identifier for Google IP changes. + ip_aliases: bool, True if the guest should configure IP alias routes. + target_instance_ips: bool, True supports internal IP load balancing. + dhclient_script: string, the path to a dhclient script used by dhclient. + dhcp_command: string, a command to enable Ethernet interfaces. + network_setup_enabled: bool, True if network setup is enabled. + debug: bool, True if debug output should write to the console. + """ + facility = logging.handlers.SysLogHandler.LOG_DAEMON + self.logger = logger.Logger( + name='google-networking', debug=debug, facility=facility) + self.network_utils = network_utils.NetworkUtils(logger=self.logger) + self.watcher = metadata_watcher.MetadataWatcher(logger=self.logger) + + self.ip_aliases = ip_aliases + self.target_instance_ips = target_instance_ips + + # Get initial metadata. + result = self.watcher.GetMetadata( + metadata_key=self.network_interface_metadata_key, recursive=True) + network_interfaces = self._ExtractInterfaceMetadata(result) + + if network_setup_enabled: + interfaces = [interface.name for interface in network_interfaces] + network_setup.NetworkSetup( + interfaces, dhclient_script=dhclient_script, + dhcp_command=dhcp_command, debug=debug) + + self.ip_forwarding_enabled = ip_forwarding_enabled + if ip_forwarding_enabled: + self.ip_forwarding = ip_forwarding.IpForwarding(proto_id, debug) + + try: + with file_utils.LockFile(LOCKFILE): + self.logger.info('Starting Google Networking daemon.') + timeout = 60 + random.randint(0, 30) + self.watcher.WatchMetadata( + self.HandleNetworkInterfaces, + metadata_key=self.network_interface_metadata_key, recursive=True, + timeout=timeout) + except (IOError, OSError) as e: + self.logger.warning(str(e)) + + def HandleNetworkInterfaces(self, result): + """Called when network interface metadata changes. + + Args: + result: dict, the metadata response with the network interfaces. + """ + network_interfaces = self._ExtractInterfaceMetadata(result) + + for interface in network_interfaces: + if self.ip_forwarding_enabled: + self.ip_forwarding.HandleForwardedIps( + interface.name, interface.forwarded_ips) + + def _ExtractInterfaceMetadata(self, metadata): + """Extracts network interface metadata. + + Args: + metadata: dict, the metadata response with the new network interfaces. + + Returns: + list, a list of NetworkInterface objects. + """ + interfaces = [] + for network_interface in metadata: + mac_address = network_interface.get('mac') + interface = self.network_utils.GetNetworkInterface(mac_address) + ip_addresses = [] + if interface: + ip_addresses.extend(network_interface.get('forwardedIps', [])) + if self.ip_aliases: + ip_addresses.extend(network_interface.get('ipAliases', [])) + if self.target_instance_ips: + ip_addresses.extend(network_interface.get('targetInstanceIps', [])) + interfaces.append(NetworkDaemon.NetworkInterface( + interface, ip_addresses)) + else: + message = 'Network interface not found for MAC address: %s.' + self.logger.warning(message, mac_address) + return interfaces + + class NetworkInterface(object): + """Network interface information extracted from metadata.""" + + def __init__(self, name, forwarded_ips=None): + self.name = name + self.forwarded_ips = forwarded_ips + + +def main(): + parser = optparse.OptionParser() + parser.add_option( + '-d', '--debug', action='store_true', dest='debug', + help='print debug output to the console.') + (options, _) = parser.parse_args() + debug = bool(options.debug) + instance_config = config_manager.ConfigManager() + ip_forwarding_daemon_enabled = instance_config.GetOptionBool( + 'Daemons', 'ip_forwarding_daemon') + ip_forwarding_enabled = instance_config.GetOptionBool( + 'NetworkInterfaces', 'ip_forwarding') or ip_forwarding_daemon_enabled + network_setup_enabled = instance_config.GetOptionBool( + 'NetworkInterfaces', 'setup') + network_daemon_enabled = instance_config.GetOptionBool( + 'Daemons', 'network_daemon') + proto_id = instance_config.GetOptionString( + 'IpForwarding', 'ethernet_proto_id') + ip_aliases = instance_config.GetOptionBool( + 'IpForwarding', 'ip_aliases') + target_instance_ips = instance_config.GetOptionBool( + 'IpForwarding', 'target_instance_ips') + dhclient_script = instance_config.GetOptionString( + 'NetworkInterfaces', 'dhclient_script') + dhcp_command = instance_config.GetOptionString( + 'NetworkInterfaces', 'dhcp_command') + + if network_daemon_enabled: + NetworkDaemon( + ip_forwarding_enabled=ip_forwarding_enabled, + proto_id=proto_id, + ip_aliases=ip_aliases, + target_instance_ips=target_instance_ips, + dhclient_script=dhclient_script, + dhcp_command=dhcp_command, + network_setup_enabled=network_setup_enabled, + debug=debug) + + +if __name__ == '__main__': + main() diff --git a/google_compute_engine/networking/network_setup/__init__.py b/google_compute_engine/networking/network_setup/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/google_compute_engine/networking/network_setup/__init__.py diff --git a/google_compute_engine/network_setup/network_setup.py b/google_compute_engine/networking/network_setup/network_setup.py index 954fda4..985e396 100755 --- a/google_compute_engine/network_setup/network_setup.py +++ b/google_compute_engine/networking/network_setup/network_setup.py @@ -13,30 +13,26 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Enables the network interfaces provided in metadata.""" +"""Enables the network interfaces for multi-nic support.""" import logging.handlers -import optparse import subprocess -from google_compute_engine import config_manager -from google_compute_engine import constants from google_compute_engine import logger -from google_compute_engine import metadata_watcher -from google_compute_engine import network_utils from google_compute_engine.compat import distro_utils class NetworkSetup(object): - """Enable network interfaces specified by metadata.""" + """Enable network interfaces.""" network_interfaces = 'instance/network-interfaces' - network_path = constants.LOCALBASE + '/etc/sysconfig/network-scripts' - def __init__(self, dhclient_script=None, dhcp_command=None, debug=False): + def __init__( + self, interfaces, dhclient_script=None, dhcp_command=None, debug=False): """Constructor. Args: + interfaces: list of string, the output device names to enable. dhclient_script: string, the path to a dhclient script used by dhclient. dhcp_command: string, a command to enable Ethernet interfaces. debug: bool, True if debug output should write to the console. @@ -46,10 +42,8 @@ class NetworkSetup(object): facility = logging.handlers.SysLogHandler.LOG_DAEMON self.logger = logger.Logger( name='network-setup', debug=debug, facility=facility) - self.watcher = metadata_watcher.MetadataWatcher(logger=self.logger) - self.network_utils = network_utils.NetworkUtils(logger=self.logger) self.distro_utils = distro_utils.Utils(debug=debug) - self._SetupNetworkInterfaces() + self._EnableNetworkInterfaces(interfaces) def _EnableNetworkInterfaces(self, interfaces): """Enable the list of network interfaces. @@ -72,40 +66,3 @@ class NetworkSetup(object): # Distro-specific setup for network interfaces. self.distro_utils.EnableNetworkInterfaces( interfaces, self.logger, dhclient_script=self.dhclient_script) - - def _SetupNetworkInterfaces(self): - """Get network interfaces metadata and enable each Ethernet interface.""" - result = self.watcher.GetMetadata( - metadata_key=self.network_interfaces, recursive=True) - interfaces = [] - - for network_interface in result: - mac_address = network_interface.get('mac') - interface = self.network_utils.GetNetworkInterface(mac_address) - if interface: - interfaces.append(interface) - else: - message = 'Network interface not found for MAC address: %s.' - self.logger.warning(message, mac_address) - - self._EnableNetworkInterfaces(interfaces) - - -def main(): - parser = optparse.OptionParser() - parser.add_option( - '-d', '--debug', action='store_true', dest='debug', - help='print debug output to the console.') - (options, _) = parser.parse_args() - instance_config = config_manager.ConfigManager() - if instance_config.GetOptionBool('NetworkInterfaces', 'setup'): - NetworkSetup( - dhclient_script=instance_config.GetOptionString( - 'NetworkInterfaces', 'dhclient_script'), - dhcp_command=instance_config.GetOptionString( - 'NetworkInterfaces', 'dhcp_command'), - debug=bool(options.debug)) - - -if __name__ == '__main__': - main() diff --git a/google_compute_engine/networking/network_setup/tests/__init__.py b/google_compute_engine/networking/network_setup/tests/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/google_compute_engine/networking/network_setup/tests/__init__.py diff --git a/google_compute_engine/networking/network_setup/tests/network_setup_test.py b/google_compute_engine/networking/network_setup/tests/network_setup_test.py new file mode 100644 index 0000000..1758dc1 --- /dev/null +++ b/google_compute_engine/networking/network_setup/tests/network_setup_test.py @@ -0,0 +1,81 @@ +#!/usr/bin/python +# Copyright 2016 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Unittest for network_setup.py module.""" + +import subprocess + +from google_compute_engine.networking.network_setup import network_setup +from google_compute_engine.test_compat import mock +from google_compute_engine.test_compat import unittest + + +class NetworkSetupTest(unittest.TestCase): + + def setUp(self): + self.mock_logger = mock.Mock() + self.mock_distro_utils = mock.Mock() + self.mock_setup = mock.create_autospec(network_setup.NetworkSetup) + self.mock_setup.logger = self.mock_logger + self.mock_setup.distro_utils = self.mock_distro_utils + self.mock_setup.dhclient_script = '/bin/script' + self.mock_setup.dhcp_command = '' + + @mock.patch('google_compute_engine.networking.network_setup.network_setup.logger') + def testNetworkSetup(self, mock_logger): + mock_logger_instance = mock.Mock() + mock_logger.Logger.return_value = mock_logger_instance + mocks = mock.Mock() + mocks.attach_mock(mock_logger, 'logger') + with mock.patch.object( + network_setup.NetworkSetup, '_EnableNetworkInterfaces'): + + network_setup.NetworkSetup(['A', 'B'], debug=True) + expected_calls = [ + mock.call.logger.Logger(name=mock.ANY, debug=True, facility=mock.ANY), + ] + self.assertEqual(mocks.mock_calls, expected_calls) + + @mock.patch('google_compute_engine.networking.network_setup.network_setup.subprocess.check_call') + def testEnableNetworkInterfaces(self, mock_call): + mocks = mock.Mock() + mocks.attach_mock(mock_call, 'call') + mocks.attach_mock(self.mock_logger, 'logger') + mocks.attach_mock( + self.mock_setup.distro_utils.EnableNetworkInterfaces, 'enable') + mock_call.side_effect = [None, subprocess.CalledProcessError(1, 'Test')] + + # Return immediately with fewer than two interfaces. + network_setup.NetworkSetup._EnableNetworkInterfaces(self.mock_setup, None) + network_setup.NetworkSetup._EnableNetworkInterfaces(self.mock_setup, []) + # Enable interfaces. + network_setup.NetworkSetup._EnableNetworkInterfaces( + self.mock_setup, ['A', 'B']) + # Run a user supplied command successfully. + self.mock_setup.dhcp_command = 'success' + network_setup.NetworkSetup._EnableNetworkInterfaces( + self.mock_setup, ['E', 'F']) + # Run a user supplied command and logger error messages. + self.mock_setup.dhcp_command = 'failure' + network_setup.NetworkSetup._EnableNetworkInterfaces( + self.mock_setup, ['G', 'H']) + expected_calls = [ + # First calls with empty `interfaces` were no-ops. + mock.call.enable(['A', 'B'], mock.ANY, dhclient_script='/bin/script'), + mock.call.call(['success']), + mock.call.call(['failure']), + mock.call.logger.warning(mock.ANY), + ] + self.assertEqual(mocks.mock_calls, expected_calls) diff --git a/google_compute_engine/networking/tests/__init__.py b/google_compute_engine/networking/tests/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/google_compute_engine/networking/tests/__init__.py diff --git a/google_compute_engine/networking/tests/network_daemon_test.py b/google_compute_engine/networking/tests/network_daemon_test.py new file mode 100644 index 0000000..51142b6 --- /dev/null +++ b/google_compute_engine/networking/tests/network_daemon_test.py @@ -0,0 +1,271 @@ +#!/usr/bin/python +# Copyright 2018 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Unittest for network_daemon.py module.""" + +from google_compute_engine import network_utils +from google_compute_engine.networking import network_daemon +from google_compute_engine.test_compat import mock +from google_compute_engine.test_compat import unittest + + +class NetworkDaemonTest(unittest.TestCase): + + def setUp(self): + self.mock_logger = mock.Mock() + self.mock_watcher = mock.Mock() + self.mock_setup = mock.create_autospec(network_daemon.NetworkDaemon) + self.mock_setup.logger = self.mock_logger + self.mock_setup.watcher = self.mock_watcher + self.mock_ip_forwarding = mock.Mock() + self.mock_network_utils = mock.Mock() + self.mock_setup.ip_forwarding = self.mock_ip_forwarding + self.mock_setup.network_utils = self.mock_network_utils + + @mock.patch('google_compute_engine.networking.network_daemon.ip_forwarding') + @mock.patch('google_compute_engine.networking.network_daemon.network_setup') + @mock.patch('google_compute_engine.networking.network_daemon.network_utils') + @mock.patch('google_compute_engine.networking.network_daemon.metadata_watcher') + @mock.patch('google_compute_engine.networking.network_daemon.logger') + @mock.patch('google_compute_engine.networking.network_daemon.file_utils') + def testNetworkDaemon( + self, mock_lock, mock_logger, mock_watcher, mock_network_utils, + mock_network_setup, mock_ip_forwarding): + mock_logger_instance = mock.Mock() + mock_logger.Logger.return_value = mock_logger_instance + mocks = mock.Mock() + mocks.attach_mock(mock_lock, 'lock') + mocks.attach_mock(mock_logger, 'logger') + mocks.attach_mock(mock_network_utils, 'network') + mocks.attach_mock(mock_ip_forwarding, 'forwarding') + mocks.attach_mock(mock_network_setup, 'network_setup') + mocks.attach_mock(mock_watcher, 'watcher') + metadata_key = network_daemon.NetworkDaemon.network_interface_metadata_key + with mock.patch.object( + network_daemon.NetworkDaemon, 'HandleNetworkInterfaces' + ) as mock_handle, mock.patch.object( + network_daemon.NetworkDaemon, '_ExtractInterfaceMetadata' + ) as mock_extract: + mocks.attach_mock(mock_extract, 'extract') + mock_extract.return_value = [ + network_daemon.NetworkDaemon.NetworkInterface('a', []), + network_daemon.NetworkDaemon.NetworkInterface('b', []), + ] + + network_daemon.NetworkDaemon( + ip_forwarding_enabled=True, + proto_id='66', + ip_aliases=None, + target_instance_ips=None, + dhclient_script='x', + dhcp_command='y', + network_setup_enabled=True, + debug=True) + expected_calls = [ + mock.call.logger.Logger(name=mock.ANY, debug=True, facility=mock.ANY), + mock.call.network.NetworkUtils(logger=mock_logger_instance), + mock.call.watcher.MetadataWatcher(logger=mock_logger_instance), + mock.call.watcher.MetadataWatcher().GetMetadata( + metadata_key=metadata_key, recursive=True), + mock.call.extract(mock.ANY), + mock.call.network_setup.NetworkSetup( + mock.ANY, debug=True, dhclient_script='x', dhcp_command='y'), + mock.call.forwarding.IpForwarding('66', True), + mock.call.lock.LockFile(network_daemon.LOCKFILE), + mock.call.lock.LockFile().__enter__(), + mock.call.logger.Logger().info(mock.ANY), + mock.call.watcher.MetadataWatcher().WatchMetadata( + mock_handle, metadata_key=metadata_key, recursive=True, + timeout=mock.ANY), + mock.call.lock.LockFile().__exit__(None, None, None), + ] + self.assertEqual(mocks.mock_calls, expected_calls) + + @mock.patch('google_compute_engine.networking.network_daemon.ip_forwarding') + @mock.patch('google_compute_engine.networking.network_daemon.network_setup') + @mock.patch('google_compute_engine.networking.network_daemon.network_utils') + @mock.patch('google_compute_engine.networking.network_daemon.metadata_watcher') + @mock.patch('google_compute_engine.networking.network_daemon.logger') + @mock.patch('google_compute_engine.networking.network_daemon.file_utils') + def testNetworkDaemonError( + self, mock_lock, mock_logger, mock_watcher, mock_network_utils, + mock_network_setup, mock_ip_forwarding): + mock_logger_instance = mock.Mock() + mock_logger.Logger.return_value = mock_logger_instance + mocks = mock.Mock() + mocks.attach_mock(mock_lock, 'lock') + mocks.attach_mock(mock_logger, 'logger') + mocks.attach_mock(mock_network_utils, 'network') + mocks.attach_mock(mock_ip_forwarding, 'forwarding') + mocks.attach_mock(mock_network_setup, 'network_setup') + mocks.attach_mock(mock_watcher, 'watcher') + metadata_key = network_daemon.NetworkDaemon.network_interface_metadata_key + self.mock_setup._ExtractInterfaceMetadata.return_value = [] + mock_lock.LockFile.side_effect = IOError('Test Error') + with mock.patch.object( + network_daemon.NetworkDaemon, 'HandleNetworkInterfaces' + ), mock.patch.object( + network_daemon.NetworkDaemon, '_ExtractInterfaceMetadata' + ) as mock_extract: + mocks.attach_mock(mock_extract, 'extract') + mock_extract.return_value = [ + network_daemon.NetworkDaemon.NetworkInterface('a', []), + network_daemon.NetworkDaemon.NetworkInterface('b', []), + ] + + network_daemon.NetworkDaemon( + ip_forwarding_enabled=False, + proto_id='66', + ip_aliases=None, + target_instance_ips=None, + dhclient_script='x', + dhcp_command='y', + network_setup_enabled=False, + debug=True) + expected_calls = [ + mock.call.logger.Logger(name=mock.ANY, debug=True, facility=mock.ANY), + mock.call.network.NetworkUtils(logger=mock_logger_instance), + mock.call.watcher.MetadataWatcher(logger=mock_logger_instance), + mock.call.watcher.MetadataWatcher().GetMetadata( + metadata_key=metadata_key, recursive=True), + mock.call.extract(mock.ANY), + mock.call.lock.LockFile(network_daemon.LOCKFILE), + mock.call.logger.Logger().warning('Test Error'), + ] + self.assertEqual(mocks.mock_calls, expected_calls) + + def testHandleNetworkInterfaces(self): + mocks = mock.Mock() + mocks.attach_mock(self.mock_ip_forwarding, 'forwarding') + mocks.attach_mock(self.mock_setup, 'setup') + self.mock_setup.ip_aliases = None + self.mock_setup.target_instance_ips = None + self.mock_setup.ip_forwarding_enabled = True + self.mock_setup._ExtractInterfaceMetadata.return_value = [ + network_daemon.NetworkDaemon.NetworkInterface('a'), + network_daemon.NetworkDaemon.NetworkInterface('b'), + ] + result = mock.Mock() + + network_daemon.NetworkDaemon.HandleNetworkInterfaces( + self.mock_setup, result) + expected_calls = [ + mock.call.setup._ExtractInterfaceMetadata(result), + mock.call.forwarding.HandleForwardedIps('a', None), + mock.call.forwarding.HandleForwardedIps('b', None), + ] + self.assertEqual(mocks.mock_calls, expected_calls) + + def testHandleNetworkInterfacesDisabled(self): + mocks = mock.Mock() + mocks.attach_mock(self.mock_ip_forwarding, 'forwarding') + mocks.attach_mock(self.mock_setup, 'setup') + self.mock_setup.ip_aliases = None + self.mock_setup.target_instance_ips = None + self.mock_setup.ip_forwarding_enabled = False + self.mock_setup._ExtractInterfaceMetadata.return_value = [ + network_daemon.NetworkDaemon.NetworkInterface('a'), + network_daemon.NetworkDaemon.NetworkInterface('b'), + ] + result = mock.Mock() + + network_daemon.NetworkDaemon.HandleNetworkInterfaces( + self.mock_setup, result) + expected_calls = [ + mock.call.setup._ExtractInterfaceMetadata(result), + ] + self.assertEqual(mocks.mock_calls, expected_calls) + + def testExtractInterfaceMetadata(self): + self.mock_setup.ip_aliases = True + self.mock_setup.target_instance_ips = True + self.mock_setup.network_utils = network_utils.NetworkUtils() + self.mock_setup.network_utils.interfaces = { + '1': 'eth0', '2': 'eth1', '3': 'eth2', + } + metadata = [ + { + 'mac': '1', + 'forwardedIps': ['a'], + }, + { + 'mac': '2', + 'forwardedIps': ['b'], + 'ipAliases': ['banana'], + 'targetInstanceIps': ['baklava'], + }, + { + 'mac': '3', + 'ipAliases': ['cherry'], + 'targetInstanceIps': ['cake'], + }, + { + 'mac': '4', + }, + { + 'forwardedIps': ['d'], + 'ipAliases': ['date'], + 'targetInstanceIps': ['doughnuts'], + }, + ] + expected_interfaces = [ + network_daemon.NetworkDaemon.NetworkInterface('eth0', ['a']), + network_daemon.NetworkDaemon.NetworkInterface( + 'eth1', ['b', 'banana', 'baklava']), + network_daemon.NetworkDaemon.NetworkInterface( + 'eth2', ['cherry', 'cake']), + ] + + actual_interfaces = network_daemon.NetworkDaemon._ExtractInterfaceMetadata( + self.mock_setup, metadata) + for actual, expected in zip(actual_interfaces, expected_interfaces): + self.assertEqual(actual.name, expected.name) + self.assertEqual(actual.forwarded_ips, expected.forwarded_ips) + + def testExtractInterfaceMetadataWithoutOptions(self): + self.mock_setup.ip_aliases = None + self.mock_setup.target_instance_ips = None + self.mock_setup.network_utils = network_utils.NetworkUtils() + self.mock_setup.network_utils.interfaces = { + '1': 'eth0', '2': 'eth1', '3': 'eth2', + } + metadata = [ + { + 'mac': '1', + 'forwardedIps': ['a'], + }, + { + 'mac': '2', + 'forwardedIps': ['b'], + 'ipAliases': ['banana'], + 'targetInstanceIps': ['baklava'], + }, + { + 'mac': '3', + 'ipAliases': ['cherry'], + 'targetInstanceIps': ['cake'], + }, + ] + expected_interfaces = [ + network_daemon.NetworkDaemon.NetworkInterface('eth0', ['a']), + network_daemon.NetworkDaemon.NetworkInterface('eth1', ['b']), + network_daemon.NetworkDaemon.NetworkInterface('eth2', []), + ] + + actual_interfaces = network_daemon.NetworkDaemon._ExtractInterfaceMetadata( + self.mock_setup, metadata) + for actual, expected in zip(actual_interfaces, expected_interfaces): + self.assertEqual(actual.name, expected.name) + self.assertEqual(actual.forwarded_ips, expected.forwarded_ips) diff --git a/google_compute_engine_init/systemd/90-google-compute-engine.preset b/google_compute_engine_init/systemd/90-google-compute-engine.preset index e2674ac..b903cb7 100644 --- a/google_compute_engine_init/systemd/90-google-compute-engine.preset +++ b/google_compute_engine_init/systemd/90-google-compute-engine.preset @@ -1,7 +1,6 @@ enable google-accounts-daemon.service enable google-clock-skew-daemon.service enable google-instance-setup.service -enable google-ip-forwarding-daemon.service -enable google-network-setup.service +enable google-network-daemon.service enable google-shutdown-scripts.service enable google-startup-scripts.service diff --git a/google_compute_engine_init/systemd/google-accounts-daemon.service b/google_compute_engine_init/systemd/google-accounts-daemon.service index b5945ee..f1d7111 100644 --- a/google_compute_engine_init/systemd/google-accounts-daemon.service +++ b/google_compute_engine_init/systemd/google-accounts-daemon.service @@ -1,6 +1,6 @@ [Unit] Description=Google Compute Engine Accounts Daemon -After=network.target google-instance-setup.service google-network-setup.service +After=network.target google-instance-setup.service google-network-daemon.service Requires=network.target [Service] diff --git a/google_compute_engine_init/systemd/google-clock-skew-daemon.service b/google_compute_engine_init/systemd/google-clock-skew-daemon.service index 476abde..71369f5 100644 --- a/google_compute_engine_init/systemd/google-clock-skew-daemon.service +++ b/google_compute_engine_init/systemd/google-clock-skew-daemon.service @@ -1,6 +1,6 @@ [Unit] Description=Google Compute Engine Clock Skew Daemon -After=network.target google-instance-setup.service google-network-setup.service +After=network.target google-instance-setup.service google-network-daemon.service Requires=network.target [Service] diff --git a/google_compute_engine_init/systemd/google-ip-forwarding-daemon.service b/google_compute_engine_init/systemd/google-ip-forwarding-daemon.service deleted file mode 100644 index d3704c6..0000000 --- a/google_compute_engine_init/systemd/google-ip-forwarding-daemon.service +++ /dev/null @@ -1,11 +0,0 @@ -[Unit] -Description=Google Compute Engine IP Forwarding Daemon -After=network.target google-instance-setup.service google-network-setup.service -Requires=network.target - -[Service] -Type=simple -ExecStart=/usr/bin/google_ip_forwarding_daemon - -[Install] -WantedBy=multi-user.target diff --git a/google_compute_engine_init/systemd/google-network-setup.service b/google_compute_engine_init/systemd/google-network-daemon.service index 4400391..c5da336 100644 --- a/google_compute_engine_init/systemd/google-network-setup.service +++ b/google_compute_engine_init/systemd/google-network-daemon.service @@ -1,13 +1,13 @@ [Unit] -Description=Google Compute Engine Network Setup +Description=Google Compute Engine Network Daemon After=local-fs.target network-online.target network.target rsyslog.service After=google-instance-setup.service Wants=local-fs.target network-online.target network.target +Requires=network.target [Service] -ExecStart=/usr/bin/google_network_setup -KillMode=process -Type=oneshot +Type=simple +ExecStart=/usr/bin/google_network_daemon [Install] WantedBy=multi-user.target diff --git a/google_compute_engine_init/systemd/google-shutdown-scripts.service b/google_compute_engine_init/systemd/google-shutdown-scripts.service index 9191a9c..069405e 100644 --- a/google_compute_engine_init/systemd/google-shutdown-scripts.service +++ b/google_compute_engine_init/systemd/google-shutdown-scripts.service @@ -1,7 +1,7 @@ [Unit] Description=Google Compute Engine Shutdown Scripts After=local-fs.target network-online.target network.target rsyslog.service systemd-resolved.service -After=google-instance-setup.service google-network-setup.service +After=google-instance-setup.service google-network-daemon.service Wants=local-fs.target network-online.target network.target [Service] diff --git a/google_compute_engine_init/systemd/google-startup-scripts.service b/google_compute_engine_init/systemd/google-startup-scripts.service index 9c04d79..73ee8db 100644 --- a/google_compute_engine_init/systemd/google-startup-scripts.service +++ b/google_compute_engine_init/systemd/google-startup-scripts.service @@ -1,7 +1,7 @@ [Unit] Description=Google Compute Engine Startup Scripts After=local-fs.target network-online.target network.target rsyslog.service -After=google-instance-setup.service google-network-setup.service +After=google-instance-setup.service google-network-daemon.service Wants=local-fs.target network-online.target network.target [Service] diff --git a/google_compute_engine_init/systemd/postinst.sh b/google_compute_engine_init/systemd/postinst.sh index eae4a87..01b4f8a 100755 --- a/google_compute_engine_init/systemd/postinst.sh +++ b/google_compute_engine_init/systemd/postinst.sh @@ -16,24 +16,20 @@ # Stop existing daemons. systemctl stop --no-block google-accounts-daemon systemctl stop --no-block google-clock-skew-daemon -systemctl stop --no-block google-ip-forwarding-daemon +systemctl stop --no-block google-network-daemon # Enable systemd services. systemctl enable google-accounts-daemon.service systemctl enable google-clock-skew-daemon.service systemctl enable google-instance-setup.service -systemctl enable google-ip-forwarding-daemon.service -systemctl enable google-network-setup.service +systemctl enable google-network-daemon.service systemctl enable google-shutdown-scripts.service systemctl enable google-startup-scripts.service # Run instance setup manually to prevent startup script execution. /usr/bin/google_instance_setup -# Enable network interfaces. -/usr/bin/google_network_setup - # Start daemons. +systemctl start --no-block google-network-daemon systemctl start --no-block google-accounts-daemon systemctl start --no-block google-clock-skew-daemon -systemctl start --no-block google-ip-forwarding-daemon diff --git a/google_compute_engine_init/systemd/prerm.sh b/google_compute_engine_init/systemd/prerm.sh index d746b6f..e4f8bdd 100755 --- a/google_compute_engine_init/systemd/prerm.sh +++ b/google_compute_engine_init/systemd/prerm.sh @@ -16,13 +16,12 @@ if [ "$1" = purge ]; then systemctl stop --no-block google-accounts-daemon systemctl stop --no-block google-clock-skew-daemon - systemctl stop --no-block google-ip-forwarding-daemon + systemctl stop --no-block google-network-daemon systemctl --no-reload disable google-accounts-daemon.service systemctl --no-reload disable google-clock-skew-daemon.service systemctl --no-reload disable google-instance-setup.service - systemctl --no-reload disable google-ip-forwarding-daemon.service - systemctl --no-reload disable google-network-setup.service + systemctl --no-reload disable google-network-daemon.service systemctl --no-reload disable google-shutdown-scripts.service systemctl --no-reload disable google-startup-scripts.service fi diff --git a/google_compute_engine_init/systemd/rpm_replace b/google_compute_engine_init/systemd/rpm_replace index 35366e8..2224799 100755 --- a/google_compute_engine_init/systemd/rpm_replace +++ b/google_compute_engine_init/systemd/rpm_replace @@ -4,18 +4,14 @@ systemctl enable google-accounts-daemon.service systemctl enable google-clock-skew-daemon.service systemctl enable google-instance-setup.service -systemctl enable google-ip-forwarding-daemon.service -systemctl enable google-network-setup.service +systemctl enable google-network-daemon.service systemctl enable google-shutdown-scripts.service systemctl enable google-startup-scripts.service # Run instance setup manually. /usr/bin/google_instance_setup -# Enable network interfaces. -/usr/bin/google_network_setup - # Start daemons. systemctl start --no-block google-accounts-daemon systemctl start --no-block google-clock-skew-daemon -systemctl start --no-block google-ip-forwarding-daemon +systemctl start --no-block google-network-daemon diff --git a/google_compute_engine_init/sysvinit/google-accounts-daemon b/google_compute_engine_init/sysvinit/google-accounts-daemon index fe5ead5..efeff6b 100755 --- a/google_compute_engine_init/sysvinit/google-accounts-daemon +++ b/google_compute_engine_init/sysvinit/google-accounts-daemon @@ -16,7 +16,7 @@ ### BEGIN INIT INFO # Provides: google_accounts_daemon # Required-Start: $local_fs $network $named $syslog $google_instance_setup -# $google_network_setup +# $google_network_daemon # Required-Stop: # Default-Start: 2 3 4 5 # Default-Stop: diff --git a/google_compute_engine_init/sysvinit/google-clock-skew-daemon b/google_compute_engine_init/sysvinit/google-clock-skew-daemon index 92e1607..9f7cfc7 100755 --- a/google_compute_engine_init/sysvinit/google-clock-skew-daemon +++ b/google_compute_engine_init/sysvinit/google-clock-skew-daemon @@ -16,7 +16,7 @@ ### BEGIN INIT INFO # Provides: google_clock_skew_daemon # Required-Start: $network $syslog $google_instance_setup -# $google_network_setup +# $google_network_daemon # Required-Stop: $network # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 diff --git a/google_compute_engine_init/sysvinit/google-ip-forwarding-daemon b/google_compute_engine_init/sysvinit/google-network-daemon index 6aefc0b..84ef3e6 100755..100644 --- a/google_compute_engine_init/sysvinit/google-ip-forwarding-daemon +++ b/google_compute_engine_init/sysvinit/google-network-daemon @@ -14,20 +14,19 @@ # limitations under the License. # ### BEGIN INIT INFO -# Provides: google_ip_forwarding_daemon +# Provides: google_network_daemon # Required-Start: $network $syslog $google_instance_setup -# $google_network_setup # Required-Stop: $network # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 -# Short-Description: Google Compute Engine IP Forwarding Daemon -# Description: Manages IP forwarding. +# Short-Description: Google Compute Engine Network Setup and Daemon +# Description: Enables Networks on boot and manages Networking. ### END INIT INFO # Do NOT "set -e". -NAME=google-ip-forwarding-daemon -DAEMON=/usr/bin/google_ip_forwarding_daemon +NAME=google-network-daemon +DAEMON=/usr/bin/google_network_daemon PIDFILE=/var/run/$NAME.pid SCRIPTNAME=/etc/init.d/$NAME diff --git a/google_compute_engine_init/sysvinit/google-network-setup b/google_compute_engine_init/sysvinit/google-network-setup deleted file mode 100755 index 41b800c..0000000 --- a/google_compute_engine_init/sysvinit/google-network-setup +++ /dev/null @@ -1,50 +0,0 @@ -#!/bin/sh -# Copyright 2016 Google Inc. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -### BEGIN INIT INFO -# Provides: google_network_setup -# Required-Start: $all $google_instance_setup -# Required-Stop: -# Default-Start: 2 3 4 5 -# Default-Stop: -# Short-Description: Google Compute Engine Network Setup -# Description: Enables network interfaces on boot. -### END INIT INFO - -NAME=google-network-setup -SCRIPTNAME=/etc/init.d/$NAME - -# Load the rcS variables. -. /lib/init/vars.sh - -# -# Function that starts the daemon/service. -# -do_start() -{ - /usr/bin/google_network_setup > /dev/null -} - -case "$1" in - start) - do_start - ;; - *) - echo "Usage: $SCRIPTNAME start" >&2 - exit 1 - ;; -esac - -: diff --git a/google_compute_engine_init/sysvinit/google-startup-scripts b/google_compute_engine_init/sysvinit/google-startup-scripts index c9d61d1..e7c5be0 100755 --- a/google_compute_engine_init/sysvinit/google-startup-scripts +++ b/google_compute_engine_init/sysvinit/google-startup-scripts @@ -15,7 +15,7 @@ # ### BEGIN INIT INFO # Provides: google_startup_scripts -# Required-Start: $all $google_instance_setup $google_network_setup +# Required-Start: $all $google_instance_setup $google_network_daemon # Required-Stop: # Default-Start: 2 3 4 5 # Default-Stop: diff --git a/google_compute_engine_init/sysvinit/postinst.sh b/google_compute_engine_init/sysvinit/postinst.sh index 96676d9..02b5a7e 100755 --- a/google_compute_engine_init/sysvinit/postinst.sh +++ b/google_compute_engine_init/sysvinit/postinst.sh @@ -16,13 +16,9 @@ update-rc.d google-accounts-daemon defaults update-rc.d google-clock-skew-daemon defaults update-rc.d google-instance-setup defaults -update-rc.d google-ip-forwarding-daemon defaults -update-rc.d google-network-setup defaults +update-rc.d google-network-daemon defaults update-rc.d google-shutdown-scripts defaults update-rc.d google-startup-scripts defaults # Run instance setup. /etc/init.d/google-instance-setup start - -# Enable network interfaces. -/etc/init.d/google-network-setup start diff --git a/google_compute_engine_init/sysvinit/prerm.sh b/google_compute_engine_init/sysvinit/prerm.sh index 466a357..a501cb6 100755 --- a/google_compute_engine_init/sysvinit/prerm.sh +++ b/google_compute_engine_init/sysvinit/prerm.sh @@ -17,8 +17,8 @@ if [ "$1" = purge ]; then update-rc.d google-accounts-daemon remove update-rc.d google-clock-skew-daemon remove update-rc.d google-instance-setup remove + update-rc.d google-network-daemon remove update-rc.d google-ip-forwarding-daemon remove - update-rc.d google-network-setup remove update-rc.d google-shutdown-scripts defaults update-rc.d google-startup-scripts defaults fi diff --git a/google_compute_engine_init/sysvinit/rpm_replace b/google_compute_engine_init/sysvinit/rpm_replace index 782941b..a19fc2d 100755 --- a/google_compute_engine_init/sysvinit/rpm_replace +++ b/google_compute_engine_init/sysvinit/rpm_replace @@ -3,5 +3,3 @@ # Run instance setup. /usr/bin/google_instance_setup -# Enable network interfaces. -/usr/bin/google_network_setup diff --git a/google_compute_engine_init/upstart/google-accounts-daemon.conf b/google_compute_engine_init/upstart/google-accounts-daemon.conf index 189ac2d..446b92c 100644 --- a/google_compute_engine_init/upstart/google-accounts-daemon.conf +++ b/google_compute_engine_init/upstart/google-accounts-daemon.conf @@ -1,5 +1,5 @@ # Manages accounts from metadata SSH keys. -start on stopped google-network-setup +start on started google-network-daemon respawn exec /usr/bin/google_accounts_daemon diff --git a/google_compute_engine_init/upstart/google-clock-skew-daemon.conf b/google_compute_engine_init/upstart/google-clock-skew-daemon.conf index 9c17ad4..05c73ce 100644 --- a/google_compute_engine_init/upstart/google-clock-skew-daemon.conf +++ b/google_compute_engine_init/upstart/google-clock-skew-daemon.conf @@ -1,5 +1,5 @@ # Sync the system clock on migration. -start on stopped google-network-setup +start on started google-network-daemon respawn exec /usr/bin/google_clock_skew_daemon diff --git a/google_compute_engine_init/upstart/google-ip-forwarding-daemon.conf b/google_compute_engine_init/upstart/google-ip-forwarding-daemon.conf deleted file mode 100644 index 7446310..0000000 --- a/google_compute_engine_init/upstart/google-ip-forwarding-daemon.conf +++ /dev/null @@ -1,5 +0,0 @@ -# Manages IP forwarding. -start on stopped google-network-setup - -respawn -exec /usr/bin/google_ip_forwarding_daemon diff --git a/google_compute_engine_init/upstart/google-network-daemon.conf b/google_compute_engine_init/upstart/google-network-daemon.conf new file mode 100644 index 0000000..d2e985a --- /dev/null +++ b/google_compute_engine_init/upstart/google-network-daemon.conf @@ -0,0 +1,5 @@ +# Manages network interfaces. +start on stopped google-instance-setup + +respawn +exec /usr/bin/google_network_daemon diff --git a/google_compute_engine_init/upstart/google-network-setup.conf b/google_compute_engine_init/upstart/google-network-setup.conf deleted file mode 100644 index 69c25ac..0000000 --- a/google_compute_engine_init/upstart/google-network-setup.conf +++ /dev/null @@ -1,6 +0,0 @@ -# Enables network interfaces on boot. -start on stopped google-instance-setup - -task - -exec /usr/bin/google_network_setup diff --git a/google_compute_engine_init/upstart/google-startup-scripts.conf b/google_compute_engine_init/upstart/google-startup-scripts.conf index 10569ea..3bda504 100644 --- a/google_compute_engine_init/upstart/google-startup-scripts.conf +++ b/google_compute_engine_init/upstart/google-startup-scripts.conf @@ -1,4 +1,4 @@ # Runs a startup script from metadata. -start on stopped google-network-setup +start on started google-network-daemon exec /usr/bin/google_metadata_script_runner --script-type startup diff --git a/google_compute_engine_init/upstart/postinst.sh b/google_compute_engine_init/upstart/postinst.sh index 8214102..dfa1b45 100755 --- a/google_compute_engine_init/upstart/postinst.sh +++ b/google_compute_engine_init/upstart/postinst.sh @@ -16,15 +16,12 @@ # Stop existing daemons stop --no-wait google-accounts-daemon stop --no-wait google-clock-skew-daemon -stop --no-wait google-ip-forwarding-daemon +stop --no-wait google-network-daemon # Run instance setup manually to prevent startup script execution. /usr/bin/google_instance_setup -# Enable network interfaces. -/usr/bin/google_network_setup - # Start daemons start --no-wait google-accounts-daemon start --no-wait google-clock-skew-daemon -start --no-wait google-ip-forwarding-daemon +start --no-wait google-network-daemon diff --git a/google_compute_engine_init/upstart/prerm.sh b/google_compute_engine_init/upstart/prerm.sh index 0b161ab..e2f5611 100755 --- a/google_compute_engine_init/upstart/prerm.sh +++ b/google_compute_engine_init/upstart/prerm.sh @@ -16,5 +16,6 @@ if [ "$1" = purge ]; then stop --no-wait google-accounts-daemon stop --no-wait google-clock-skew-daemon + stop --no-wait google-network-daemon stop --no-wait google-ip-forwarding-daemon fi diff --git a/google_compute_engine_init/upstart/rpm_replace b/google_compute_engine_init/upstart/rpm_replace index 0663e78..6b4c42b 100755 --- a/google_compute_engine_init/upstart/rpm_replace +++ b/google_compute_engine_init/upstart/rpm_replace @@ -3,10 +3,7 @@ # Run instance setup. /usr/bin/google_instance_setup -# Enable network interfaces. -/usr/bin/google_network_setup - # Manually start daemons. start --no-wait google-accounts-daemon start --no-wait google-clock-skew-daemon -start --no-wait google-ip-forwarding-daemon +start --no-wait google-network-daemon @@ -42,9 +42,8 @@ setuptools.setup( 'console_scripts': [ 'google_accounts_daemon=google_compute_engine.accounts.accounts_daemon:main', 'google_clock_skew_daemon=google_compute_engine.clock_skew.clock_skew_daemon:main', - 'google_ip_forwarding_daemon=google_compute_engine.ip_forwarding.ip_forwarding_daemon:main', 'google_instance_setup=google_compute_engine.instance_setup.instance_setup:main', - 'google_network_setup=google_compute_engine.network_setup.network_setup:main', + 'google_network_daemon=google_compute_engine.networking.network_daemon:main', 'google_metadata_script_runner=google_compute_engine.metadata_scripts.script_manager:main', ], }, diff --git a/specs/google-compute-engine.spec b/specs/google-compute-engine.spec index 6ec3d57..78bbad9 100644 --- a/specs/google-compute-engine.spec +++ b/specs/google-compute-engine.spec @@ -108,18 +108,17 @@ cp google_config/dhcp/google_hostname.sh %{buildroot}/etc/dhcp/dhclient.d/google /etc/udev/rules.d/*.rules %attr(0755,root,root) %{_bindir}/* - %post %if 0%{?el6} # On upgrade run instance setup again to handle any new configs and restart daemons. if [ $1 -eq 2 ]; then stop -q -n google-accounts-daemon stop -q -n google-clock-skew-daemon - stop -q -n google-ip-forwarding-daemon + stop -q -n google-network-daemon /usr/bin/google_instance_setup start -q -n google-accounts-daemon start -q -n google-clock-skew-daemon - start -q -n google-ip-forwarding-daemon + start -q -n google-network-daemon fi # Install google-compute-engine from pypi into the SCL environment if it exists. @@ -132,12 +131,29 @@ if [ -d /opt/rh/python27/root/usr/lib/python2.7/site-packages/google_compute_eng fi %endif +# Remove old service. +if [ -f /lib/systemd/system/google-ip-forwarding-daemon.service ]; then + systemctl stop --no-block google-ip-forwarding-daemon + systemctl --no-reload disable google-ip-forwarding-daemon.service + rm /lib/systemd/system/google-ip-forwarding-daemon.service +fi + +if [ -f /lib/systemd/system/google-network-setup.service ]; then + systemctl stop --no-block google-network-setup + systemctl --no-reload disable google-network-setup.service + rm /lib/systemd/system/google-network-setup.service +fi + +if [ $1 -eq 2 ]; then + # New service might not be enabled during upgrade. + systemctl enable google-network-daemon.service +fi + %if 0%{?el7} %systemd_post google-accounts-daemon.service %systemd_post google-clock-skew-daemon.service %systemd_post google-instance-setup.service -%systemd_post google-ip-forwarding-daemon.service -%systemd_post google-network-setup.service +%systemd_post google-network-daemon.service %systemd_post google-shutdown-scripts.service %systemd_post google-startup-scripts.service # On upgrade run instance setup again to handle any new configs and restart daemons. @@ -145,7 +161,7 @@ if [ $1 -eq 2 ]; then /usr/bin/google_instance_setup systemctl reload-or-restart google-accounts-daemon.service systemctl reload-or-restart google-clock-skew-daemon.service - systemctl reload-or-restart google-ip-forwarding-daemon.service + systemctl reload-or-restart google-network-daemon.service fi %endif @@ -156,14 +172,13 @@ if [ $1 -eq 0 ]; then %if 0%{?el6} stop -q -n google-accounts-daemon stop -q -n google-clock-skew-daemon - stop -q -n google-ip-forwarding-daemon + stop -q -n google-network-daemon %endif %if 0%{?el7} %systemd_preun google-accounts-daemon.service %systemd_preun google-clock-skew-daemon.service %systemd_preun google-instance-setup.service - %systemd_preun google-ip-forwarding-daemon.service - %systemd_preun google-network-setup.service + %systemd_preun google-network-daemon.service %systemd_preun google-shutdown-scripts.service %systemd_preun google-startup-scripts.service %endif |