From 1190779a2d83a07e5229304535b6665b48a84c5b Mon Sep 17 00:00:00 2001 From: Max Illfelder Date: Wed, 8 Jun 2016 10:42:33 -0700 Subject: README file for the Linux guest environment. --- README.md | 270 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 268 insertions(+), 2 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 73a950f..5376587 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,270 @@ -## DEVELOPMENT +## Linux Guest Environment for Google Compute Engine + +This repository stores the collection of packages installed on Google supported +Compute Engine [images](https://cloud.google.com/compute/docs/images). + +## Background + +The Linux guest environment denotes the Google provided configuration and +tooling inside of a [Google Compute Engine](https://cloud.google.com/compute/) +(GCE) virtual machine. The +[metadata server](https://cloud.google.com/compute/docs/metadata) is a +communication channel for transferring information from a client into the guest. +The Linux guest environment includes a set of scripts and daemons (long running +processes) that read the content of the metadata server to make a virtual +machine run properly on our platform. + +## Guest Overview + +The guest environment is made up of the following components: + +* **Accounts** daemon to setup and manage user accounts, and to enable SSH key + based authentication. +* **Clock skew** daemon to keep the system clock in sync after VM start and + stop events. +* **Disk expand** scripts to expand the VM root partition for CentOS 6, + CentOS 7, RHEL 6, and RHEL 7 images. +* **Instance setup** scripts to execute VM configuration scripts during boot. +* **IP forwarding** daemon that integrates network load balancing with + forwarding rule changes into the guest. +* **Metadata scripts** running user provided scripts at VM startup and + shutdown. + +The Linux guest environment is written in Python, and is version agnostic +between Python 2.6 and 3.5. There is complete unittest coverage for every Python +library and script. The design of various guest libraries, daemons, and scripts, +are detailed in the sections below. + +## Common Libraries + +The Python libraries are shared with each of the daemons and the instance setup +tools. + +### Metadata Watcher + +The guest environment relies upon retrieving content from the metadata server to +configure the VM environment. A metadata watching library handles all +communication with the metadata server. + +The library exposes two functions: + +* **GetMetadata** immediately retrieves the contents of the metadata server + for a given metadata key. The function catches and logs any connection + related exceptions. The metadata server content is returned as a + deserialized JSON object. +* **WatchMetadata** continuously makes a hanging GET, watching for changes to + the specified contents of the metadata server. When the request closes, the + watcher verifies the etag was updated. In case of an update, the etag is + updated and a provided handler function is called with the deserialized JSON + metadata content. The WatchMetadata function should never terminate; it + catches and logs any connection related exceptions, and catches and logs any + exception generated from calling the handler. + +Metadata server requests have custom retry logic for metadata server +unavailability; by default, any request has one minute to complete before the +request is cancelled. In case of a brief network outage where the metadata +server is unavailable, there is a short delay between retries. + +### Logging + +The Google added daemons and scripts write to the serial port for added +transparency. A common logging library is a thin wrapper around the Python +logging module. The library configures appropriate SysLog handlers, sets the +logging formatter, and provides a debug options for added logging and console +output. + +### Configuration Management + +A configuration file allows users to disable daemons and modify instance setup +behaviors from a single location. Guest environment daemons and scripts need a +mechanism to integrate user settings into the guest. A configuration management +library retrieves and modifies these settings. + +The library exposes the following functions: + +* **GetOptionString** retrieves the value for a configuration option. The type + of the value is a string if set. +* **GetOptionBool** retrieves the value for a configuration option. The type + of the value is a boolean if set. +* **SetOption** sets the value of an option in the config file. An overwrite + flag specifies whether to replace an existing value. +* **WriteConfig** writes the configuration values to a file. The function is + responsible for locking the file, preventing concurrent writes, and writing + a file header if one is provided. + +### File Management + +Guest environment daemons and scripts use a common library for file management. +The library provides the following functions: + +* **SetPermissions** unifies the logic to set permissions and simplify file + creation across the various Linux distributions. The function sets the mode, + UID, and GID, of a provided path. On supported OS configurations that user + SELinux, the SELinux context is automatically set. +* **LockFile** is a context manager that simplifies the process of file + locking in Python. The function sets up an flock and releases the lock on + exit. + +## Daemons + +The guest environment daemons import and use the common libraries described +above. Each daemon reads the configuration file before execution. This allows a +user to easily disable undesired functionality. Additional daemon behaviors are +detailed below. + +### Accounts + +The accounts daemon is responsible for provisioning and deprovisioning user +accounts. The daemon grants permissions to user accounts, and updates the list +of authorized keys that has access to accounts based on metadata SSH key +updates. User account creation is based on +[adding and remove SSH Keys](https://cloud.google.com/compute/docs/instances/adding-removing-ssh-keys) +stored in metadata. + +The accounts management daemon has the following behaviors. + +* Administrator permissions are managed with a `google-sudoers` Linux group. +* All users provisioned by the account daemon are added to the + `google-sudoers` group. +* The daemon stores a file in the guest to preserve state for the user + accounts managed by Google. +* The authorized keys file for a Google managed user is delete when all SSH + keys for the user are removed from metadata. +* User accounts not managed by Google are not modified by the accounts daemon. + +### Clock Skew + +The clock skew daemon is responsible for syncing the software clock with the +hypervisor clock after a stop/start event or after a migration. Preventing clock +skew may result in `system time has changed` messages in VM logs. + +### IP Forwarding + +The IP forwarding daemon uses IP forwarding metadata to setup or remove IP +routes in the guest. + +* Only IPv4 IP addresses are currently supported. +* Routes are set on the default ethernet interface determined dynamically. +* Google routes are configured, by default, with the routing protocol ID `66`. + This ID is a namespace for daemon configured IP addresses. + +## Instance Setup + +Instance setup runs during VM boot. The script configures the Linux guest +environment by performing the following tasks. + +* Optimize for local SSD. +* Enable multi-queue on all the virtionet devices. +* Wait for network availability. +* Set SSH host keys the first time the instance is booted. +* Set the boto config for using Google Cloud Storage. +* Create the defaults configuration file. + +The defaults configuration file incorporates any user provided setting in +`/etc/default/instance_configs.cfg.template` and does not override other +conflicting settings. This allows package updates without overriding user +configuration. + +## Metadata Scripts + +Metadata scripts implement support for running user provided +[startup scripts](https://cloud.google.com/compute/docs/startupscript) and +[shutdown scripts](https://cloud.google.com/compute/docs/shutdownscript). The +guest support for metadata scripts is implemented in Python with the following +design details. + +* Metadata scripts are executed in a shell. +* If multiple metadata keys are specified (e.g. `startup-script` and + `startup-script-url`) both are executed. +* If multiple metadata keys are specified (e.g. `startup-script` and + `startup-script-url`) a URL is executed first. +* The exit status of a metadata script is logged after completed execution. + +## Configuration + +Users of Google provided images may configure the guest environment behaviors +using a configuration file. To make configuration changes, add settings to +`/etc/default/instance_configs.cfg.template`. Settings are not overridden in the +guest. + +The following are valid user configuration options. + +Section | Option | Value +--------------- | -------------------- | ----- +Accounts | deprovision_remove | `true` makes deprovisioning a user destructive. +Accounts | groups | Comma separated list of groups for newly provisioned users. +Daemons | accounts_daemon | `false` disables the accounts daemon. +Daemons | clock_skew_daemon | `false` disables the clock skew daemon. +Daemons | ip_forwarding_daemon | `false` disables the IP forwarding daemon. +InstanceSetup | optimize_local_ssd | `false` prevents optimizing for local SSD. +InstanceSetup | network_enabled | `false` skips all metadata related-functionality during instance setup. +InstanceSetup | set_boto_config | `false` skips setting up a boto config. +InstanceSetup | set_host_keys | `false` skips generating host keys on first boot. +InstanceSetup | set_multiqueue | `false` skips multiqueue driver support. +IpForwarding | ethernet_proto_id | Protocol ID string for daemon added routes. +MetadataScripts | startup | `false` disables startup script execution. +MetadataScripts | shutdown | `false` disables shutdown script execution. + +Setting `network_enabled` to `false` will skip setting up host keys and the +boto config in the guest. The setting may also prevent startup and shutdown +script execution. + +## Packaging + +The guest Python code is packaged as a +[compliant PyPI Python package](http://python-packaging-user-guide.readthedocs.io/en/latest/) +that can be used as a library or run independently. In addition to the Python +package, deb and rpm packages are created with appropriate init configuration +for supported GCE distros. The packages are targeted towards distribution +provided Python versions. + +Distro | Package Type | Python Version | Init System +------------ | ------------ | -------------- | ----------- +Debian 7 | deb | 2.7 | sysvinit +Debian 8 | deb | 2.7 | systemd +CentOS 6 | rpm | 2.6 | upstart +CentOS 7 | rpm | 2.7 | systemd +RHEL 6 | rpm | 2.6 | upstart +RHEL 7 | rpm | 2.7 | systemd +Ubuntu 12.04 | deb | 2.7 | upstart +Ubuntu 14.04 | deb | 2.7 | upstart +Ubuntu 16.04 | deb | 3.5 or 2.7 | systemd +SLES 11 | rpm | 2.6 | sysvinit +SLES 12 | rpm | 2.7 | systemd + +We build the following packages for the Linux guest environment. + +* `google_compute_engine` is a Python package for Linux daemons, scripts, and + libraries. + * The package is installed to its distro default Python package location + (e.g. `/usr/lib/python2.7/site-packages`). + * Includes appropriate init files for sysvinit, upstart, and systemd. + * Entry point scripts, created by the Python package, are located in + `/usr/bin`. +* `google_configs` is a package containing non-Python scripts and guest + configuration. + * Sets up udev rules and sysctl rules. + * Configures the SysLog output that gets sent to serial port output. + * Includes bash scripts needed by `instance_setup`. + +The package build tools are published. The tools are run on a GCE VM to build +release packages, and the output deb and rpm packages are included in our +GitHub releases. + +## Contributing + +Have a patch that will benefit this project? Awesome! Follow these steps to have +it accepted. + +1. Please sign our [Contributor License Agreement](CONTRIB.md). +1. Fork this Git repository and make your changes. +1. Create a Pull Request against the + [development](https://github.com/GoogleCloudPlatform/compute-image-packages/tree/development) + branch. +1. Incorporate review feedback to your changes. +1. Accepted! ## License -All files in this repository are under the [Apache License, Version 2.0](LICENSE) unless noted otherwise. + +All files in this repository are under the +[Apache License, Version 2.0](LICENSE) unless noted otherwise. -- cgit v1.2.1