From 795752a3cf5a598ebb6a6f7656c9ccf05c8175f4 Mon Sep 17 00:00:00 2001 From: Stephen Finucane Date: Thu, 8 Dec 2016 12:55:24 +0000 Subject: doc: Populate 'install', 'howto' sections This is a dumb move of all 'INSTALL*' docs, with very little refactoring (mostly updating links and making the titles a little more consistent. Additional refactoring will be done in subsequent changes. Signed-off-by: Stephen Finucane Signed-off-by: Ben Pfaff --- Documentation/automake.mk | 16 + Documentation/howto/docker.rst | 319 +++++++ Documentation/howto/index.rst | 6 + Documentation/howto/kvm.rst | 103 +++ Documentation/howto/libvirt.rst | 92 ++ Documentation/howto/selinux.rst | 174 ++++ Documentation/howto/ssl.rst | 338 ++++++++ Documentation/index.rst | 12 +- .../internals/contributing/documentation-style.rst | 4 +- Documentation/intro/install/bash-completion.rst | 111 +++ Documentation/intro/install/debian.rst | 123 +++ Documentation/intro/install/dpdk-advanced.rst | 938 ++++++++++++++++++++ Documentation/intro/install/dpdk.rst | 600 +++++++++++++ Documentation/intro/install/fedora.rst | 141 ++++ Documentation/intro/install/general.rst | 862 +++++++++++++++++++ Documentation/intro/install/index.rst | 33 + Documentation/intro/install/netbsd.rst | 61 ++ Documentation/intro/install/rhel.rst | 216 +++++ Documentation/intro/install/userspace.rst | 106 +++ Documentation/intro/install/windows.rst | 620 ++++++++++++++ Documentation/intro/install/xenserver.rst | 229 +++++ FAQ.rst | 23 +- INSTALL.DPDK-ADVANCED.rst | 939 --------------------- INSTALL.DPDK.rst | 604 ------------- INSTALL.Debian.rst | 123 --- INSTALL.Docker.rst | 320 ------- INSTALL.Fedora.rst | 144 ---- INSTALL.KVM.rst | 104 --- INSTALL.Libvirt.rst | 93 -- INSTALL.NetBSD.rst | 62 -- INSTALL.RHEL.rst | 208 ----- INSTALL.SELinux.rst | 174 ---- INSTALL.SSL.rst | 338 -------- INSTALL.Windows.rst | 621 -------------- INSTALL.XenServer.rst | 229 ----- INSTALL.rst | 871 ------------------- INSTALL.userspace.rst | 106 --- Makefile.am | 15 - PORTING.rst | 8 +- README.rst | 32 +- debian/openvswitch-common.docs | 2 - rhel/openvswitch-fedora.spec.in | 4 +- rhel/openvswitch.spec.in | 4 +- tutorial/ovn-tutorial.rst | 5 +- tutorial/tutorial.rst | 8 +- utilities/automake.mk | 2 - utilities/ovs-command-bashcomp.INSTALL.rst | 111 --- utilities/ovs-ctl.8 | 3 +- utilities/ovs-vsctl.8.in | 6 +- vswitchd/ovs-vswitchd.8.in | 10 +- vtep/README.ovs-vtep.rst | 15 +- vtep/vtep-ctl.8.in | 6 +- 52 files changed, 5155 insertions(+), 5139 deletions(-) create mode 100644 Documentation/howto/docker.rst create mode 100644 Documentation/howto/kvm.rst create mode 100644 Documentation/howto/libvirt.rst create mode 100644 Documentation/howto/selinux.rst create mode 100644 Documentation/howto/ssl.rst create mode 100644 Documentation/intro/install/bash-completion.rst create mode 100644 Documentation/intro/install/debian.rst create mode 100644 Documentation/intro/install/dpdk-advanced.rst create mode 100644 Documentation/intro/install/dpdk.rst create mode 100644 Documentation/intro/install/fedora.rst create mode 100644 Documentation/intro/install/general.rst create mode 100644 Documentation/intro/install/netbsd.rst create mode 100644 Documentation/intro/install/rhel.rst create mode 100644 Documentation/intro/install/userspace.rst create mode 100644 Documentation/intro/install/windows.rst create mode 100644 Documentation/intro/install/xenserver.rst delete mode 100644 INSTALL.DPDK-ADVANCED.rst delete mode 100644 INSTALL.DPDK.rst delete mode 100644 INSTALL.Debian.rst delete mode 100644 INSTALL.Docker.rst delete mode 100644 INSTALL.Fedora.rst delete mode 100644 INSTALL.KVM.rst delete mode 100644 INSTALL.Libvirt.rst delete mode 100644 INSTALL.NetBSD.rst delete mode 100644 INSTALL.RHEL.rst delete mode 100644 INSTALL.SELinux.rst delete mode 100644 INSTALL.SSL.rst delete mode 100644 INSTALL.Windows.rst delete mode 100644 INSTALL.XenServer.rst delete mode 100644 INSTALL.rst delete mode 100644 INSTALL.userspace.rst delete mode 100644 utilities/ovs-command-bashcomp.INSTALL.rst diff --git a/Documentation/automake.mk b/Documentation/automake.mk index 6f3df0770..3fade9492 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -9,9 +9,25 @@ EXTRA_DIST += \ Documentation/contents.rst \ Documentation/intro/index.rst \ Documentation/intro/install/index.rst \ + Documentation/intro/install/bash-completion.rst \ + Documentation/intro/install/debian.rst \ + Documentation/intro/install/dpdk-advanced.rst \ + Documentation/intro/install/dpdk.rst \ + Documentation/intro/install/fedora.rst \ + Documentation/intro/install/general.rst \ + Documentation/intro/install/netbsd.rst \ + Documentation/intro/install/rhel.rst \ + Documentation/intro/install/userspace.rst \ + Documentation/intro/install/windows.rst \ + Documentation/intro/install/xenserver.rst \ Documentation/tutorials/index.rst \ Documentation/topics/index.rst \ Documentation/howto/index.rst \ + Documentation/howto/docker.rst \ + Documentation/howto/kvm.rst \ + Documentation/howto/libvirt.rst \ + Documentation/howto/selinux.rst \ + Documentation/howto/ssl.rst \ Documentation/ref/index.rst \ Documentation/faq/index.rst \ Documentation/internals/index.rst \ diff --git a/Documentation/howto/docker.rst b/Documentation/howto/docker.rst new file mode 100644 index 000000000..e23ca7575 --- /dev/null +++ b/Documentation/howto/docker.rst @@ -0,0 +1,319 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=================================== +Open Virtual Networking With Docker +=================================== + +This document describes how to use Open Virtual Networking with Docker 1.9.0 +or later. + +.. important:: + + Requires Docker version 1.9.0 or later. Only Docker 1.9.0+ comes with support + for multi-host networking. Consult www.docker.com for instructions on how to + install Docker. + +.. note:: + + You must build and install Open vSwitch before proceeding with the below + guide. Refer to :doc:`/intro/install/index` for more information. + +Setup +----- + +For multi-host networking with OVN and Docker, Docker has to be started with a +destributed key-value store. For example, if you decide to use consul as your +distributed key-value store and your host IP address is ``$HOST_IP``, start +your Docker daemon with:: + + $ docker daemon --cluster-store=consul://127.0.0.1:8500 \ + --cluster-advertise=$HOST_IP:0 + +OVN provides network virtualization to containers. OVN's integration with +Docker currently works in two modes - the "underlay" mode or the "overlay" +mode. + +In the "underlay" mode, OVN requires a OpenStack setup to provide container +networking. In this mode, one can create logical networks and can have +containers running inside VMs, standalone VMs (without having any containers +running inside them) and physical machines connected to the same logical +network. This is a multi-tenant, multi-host solution. + +In the "overlay" mode, OVN can create a logical network amongst containers +running on multiple hosts. This is a single-tenant (extendable to multi-tenants +depending on the security characteristics of the workloads), multi-host +solution. In this mode, you do not need a pre-created OpenStack setup. + +For both the modes to work, a user has to install and start Open vSwitch in +each VM/host that they plan to run their containers on. + +.. _docker-overlay: + +The "overlay" mode +------------------ + +.. note:: + + OVN in "overlay" mode needs a minimum Open vSwitch version of 2.5. + +1. Start the central components. + + OVN architecture has a central component which stores your networking intent + in a database. On one of your machines, with an IP Address of + ``$CENTRAL_IP``, where you have installed and started Open vSwitch, you will + need to start some central components. + + Start ovn-northd daemon. This daemon translates networking intent from Docker + stored in the OVN\_Northbound database to logical flows in ``OVN_Southbound`` + database. For example:: + + $ /usr/share/openvswitch/scripts/ovn-ctl start_northd + +2. One time setup + + On each host, where you plan to spawn your containers, you will need to run + the below command once. You may need to run it again if your OVS database + gets cleared. It is harmless to run it again in any case:: + + $ ovs-vsctl set Open_vSwitch . \ + external_ids:ovn-remote="tcp:$CENTRAL_IP:6642" \ + external_ids:ovn-nb="tcp:$CENTRAL_IP:6641" \ + external_ids:ovn-encap-ip=$LOCAL_IP \ + external_ids:ovn-encap-type="$ENCAP_TYPE" + + where: + + ``$LOCAL_IP`` + is the IP address via which other hosts can reach this host. This acts as + your local tunnel endpoint. + + ``$ENCAP_TYPE`` + is the type of tunnel that you would like to use for overlay networking. + The options are ``geneve`` or ``stt``. Your kernel must have support for + your chosen ``$ENCAP_TYPE``. Both ``geneve`` and ``stt`` are part of the + Open vSwitch kernel module that is compiled from this repo. If you use the + Open vSwitch kernel module from upstream Linux, you will need a minumum + kernel version of 3.18 for ``geneve``. There is no ``stt`` support in + upstream Linux. You can verify whether you have the support in your kernel + as follows:: + + $ lsmod | grep $ENCAP_TYPE + + In addition, each Open vSwitch instance in an OVN deployment needs a unique, + persistent identifier, called the ``system-id``. If you install OVS from + distribution packaging for Open vSwitch (e.g. .deb or .rpm packages), or if + you use the ovs-ctl utility included with Open vSwitch, it automatically + configures a system-id. If you start Open vSwitch manually, you should set + one up yourself. For example:: + + $ id_file=/etc/openvswitch/system-id.conf + $ test -e $id_file || uuidgen > $id_file + $ ovs-vsctl set Open_vSwitch . external_ids:system-id=$(cat $id_file) + +3. Start the ``ovn-controller``. + + You need to run the below command on every boot:: + + $ /usr/share/openvswitch/scripts/ovn-ctl start_controller + +4. Start the Open vSwitch network driver. + + By default Docker uses Linux bridge for networking. But it has support for + external drivers. To use Open vSwitch instead of the Linux bridge, you will + need to start the Open vSwitch driver. + + The Open vSwitch driver uses the Python's flask module to listen to Docker's + networking api calls. So, if your host does not have Python's flask module, + install it:: + + $ sudo pip install Flask + + Start the Open vSwitch driver on every host where you plan to create your + containers. Refer to the note on ``$OVS_PYTHON_LIBS_PATH`` that is used below + at the end of this document:: + + $ PYTHONPATH=$OVS_PYTHON_LIBS_PATH ovn-docker-overlay-driver --detach + + .. note:: + + The ``$OVS_PYTHON_LIBS_PATH`` variable should point to the directory where + Open vSwitch Python modules are installed. If you installed Open vSwitch + Python modules via the Debian package of ``python-openvswitch`` or via pip + by running ``pip install ovs``, you do not need to specify the PATH. If + you installed it by following the instructions in + :doc:`/intro/install/general`, then you should specify the PATH. In this + case, the PATH depends on the options passed to ``./configure``. It is + usually either ``/usr/share/openvswitch/python`` or + ``/usr/local/share/openvswitch/python`` + +Docker has inbuilt primitives that closely match OVN's logical switches and +logical port concepts. Consult Docker's documentation for all the possible +commands. Here are some examples. + +Create a logical switch +~~~~~~~~~~~~~~~~~~~~~~~ + +To create a logical switch with name 'foo', on subnet '192.168.1.0/24', run:: + + $ NID=`docker network create -d openvswitch --subnet=192.168.1.0/24 foo` + +List all logical switches +~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + $ docker network ls + +You can also look at this logical switch in OVN's northbound database by +running the following command:: + + $ ovn-nbctl --db=tcp:$CENTRAL_IP:6640 ls-list + +Delete a logical switch +~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + $ docker network rm bar + + +Create a logical port +~~~~~~~~~~~~~~~~~~~~~ + +Docker creates your logical port and attaches it to the logical network in a +single step. For example, to attach a logical port to network ``foo`` inside +container busybox, run:: + + $ docker run -itd --net=foo --name=busybox busybox + +List all logical ports +~~~~~~~~~~~~~~~~~~~~~~ + +Docker does not currently have a CLI command to list all logical ports but you +can look at them in the OVN database by running:: + + $ ovn-nbctl --db=tcp:$CENTRAL_IP:6640 lsp-list $NID + +Create and attach a logical port to a running container +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + $ docker network create -d openvswitch --subnet=192.168.2.0/24 bar + $ docker network connect bar busybox + +Detach and delete a logical port from a running container +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can delete your logical port and detach it from a running container +by running: + +:: + + $ docker network disconnect bar busybox + +.. _docker-underlay: + +The "underlay" mode +------------------- + +.. note:: + + This mode requires that you have a OpenStack setup pre-installed with + OVN providing the underlay networking. + +1. One time setup + + A OpenStack tenant creates a VM with a single network interface (or multiple) + that belongs to management logical networks. The tenant needs to fetch the + port-id associated with the interface via which he plans to send the container + traffic inside the spawned VM. This can be obtained by running the below + command to fetch the 'id' associated with the VM:: + + $ nova list + + and then by running:: + + $ neutron port-list --device_id=$id + + Inside the VM, download the OpenStack RC file that contains the tenant + information (henceforth referred to as ``openrc.sh``). Edit the file and add the + previously obtained port-id information to the file by appending the following + line:: + + $ export OS_VIF_ID=$port_id + + After this edit, the file will look something like:: + + #!/bin/bash + export OS_AUTH_URL=http://10.33.75.122:5000/v2.0 + export OS_TENANT_ID=fab106b215d943c3bad519492278443d + export OS_TENANT_NAME="demo" + export OS_USERNAME="demo" + export OS_VIF_ID=e798c371-85f4-4f2d-ad65-d09dd1d3c1c9 + +2. Create the Open vSwitch bridge + + If your VM has one ethernet interface (e.g.: 'eth0'), you will need to add + that device as a port to an Open vSwitch bridge 'breth0' and move its IP + address and route related information to that bridge. (If it has multiple + network interfaces, you will need to create and attach an Open vSwitch + bridge for the interface via which you plan to send your container + traffic.) + + If you use DHCP to obtain an IP address, then you should kill the DHCP + client that was listening on the physical Ethernet interface (e.g. eth0) and + start one listening on the Open vSwitch bridge (e.g. breth0). + + Depending on your VM, you can make the above step persistent across reboots. + For example, if your VM is Debian/Ubuntu-based, read + `openvswitch-switch.README.Debian` found in `debian` folder. If your VM is + RHEL-based, refer to :doc:`/intro/install/rhel`. + +3. Start the Open vSwitch network driver + + The Open vSwitch driver uses the Python's flask module to listen to Docker's + networking api calls. The driver also uses OpenStack's + ``python-neutronclient`` libraries. If your host does not have Python's + ``flask`` module or ``python-neutronclient`` you must install them. For + example:: + + $ pip install python-neutronclient + $ pip install Flask + + Once installed, source the ``openrc`` file:: + + $ . ./openrc.sh + + Start the network driver and provide your OpenStack tenant password when + prompted:: + + $ PYTHONPATH=$OVS_PYTHON_LIBS_PATH ovn-docker-underlay-driver \ + --bridge breth0 --detach + +From here-on you can use the same Docker commands as described in +`docker-overlay`_. + +Refer the the ovs-architecture man pages (``man ovn-architecture``) to +understand OVN's architecture in detail. diff --git a/Documentation/howto/index.rst b/Documentation/howto/index.rst index 1c4d9d2e3..81b2931d7 100644 --- a/Documentation/howto/index.rst +++ b/Documentation/howto/index.rst @@ -32,3 +32,9 @@ topics covered herein, refer to :doc:`/topics/index`. .. toctree:: :maxdepth: 2 + + kvm + selinux + libvirt + ssl + docker diff --git a/Documentation/howto/kvm.rst b/Documentation/howto/kvm.rst new file mode 100644 index 000000000..4b43af334 --- /dev/null +++ b/Documentation/howto/kvm.rst @@ -0,0 +1,103 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +===================== +Open vSwitch with KVM +===================== + +This document describes how to use Open vSwitch with the Kernel-based Virtual +Machine (KVM). + +.. note:: + + This document assumes that you have Open vSwitch set up on a Linux system. + +Setup +----- + +KVM uses tunctl to handle various bridging modes, which you can install with +the Debian/Ubuntu package ``uml-utilities``:: + + $ apt-get install uml-utilities + +Next, you will need to modify or create custom versions of the ``qemu-ifup`` +and ``qemu-ifdown`` scripts. In this guide, we'll create custom versions that +make use of example Open vSwitch bridges that we'll describe in this guide. + +Create the following two files and store them in known locations. For example:: + + echo << EOF > /etc/ovs-ifup + #!/bin/sh + + switch='br0' + /sbin/ifconfig $1 0.0.0.0 up + ovs-vsctl add-port ${switch} $1 + EOF + +:: + + echo << EOF > /etc/ovs-ifdown + #!/bin/sh + + switch='br0' + /sbin/ifconfig $1 0.0.0.0 down + ovs-vsctl del-port ${switch} $1 + +The basic usage of Open vSwitch is described at the end of +:doc:`/intro/install/general`. If you haven't already, create a bridge named +``br0`` with the following command:: + + $ ovs-vsctl add-br br0 + +Then, add a port to the bridge for the NIC that you want your guests to +communicate over (e.g. ``eth0``):: + + $ ovs-vsctl add-port br0 eth0 + +Refer to ovs-vsctl(8) for more details. + +Next, we'll start a guest that will use our ifup and ifdown scripts:: + + $ kvm -m 512 -net nic,macaddr=00:11:22:EE:EE:EE -net \ + tap,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown -drive \ + file=/path/to/disk-image,boot=on + +This will start the guest and associate a tap device with it. The ``ovs-ifup`` +script will add a port on the br0 bridge so that the guest will be able to +communicate over that bridge. + +To get some more information and for debugging you can use Open vSwitch +utilities such as ovs-dpctl and ovs-ofctl, For example:: + + $ ovs-dpctl show + $ ovs-ofctl show br0 + +You should see tap devices for each KVM guest added as ports to the bridge +(e.g. tap0) + +Refer to ovs-dpctl(8) and ovs-ofctl(8) for more details. + +Bug Reporting +------------- + +Please report problems to bugs@openvswitch.org. diff --git a/Documentation/howto/libvirt.rst b/Documentation/howto/libvirt.rst new file mode 100644 index 000000000..0b303d54c --- /dev/null +++ b/Documentation/howto/libvirt.rst @@ -0,0 +1,92 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +========================= +Open vSwitch with Libvirt +========================= + +This document describes how to use Open vSwitch with Libvirt 0.9.11 or later. +This document assumes that you followed :doc:`/intro/install/general` or +installed Open vSwitch from distribution packaging such as a .deb or .rpm. The +Open vSwitch support is included by default in Libvirt 0.9.11. Consult +www.libvirt.org for instructions on how to build the latest Libvirt, if your +Linux distribution by default comes with an older Libvirt release. + +Limitations +----------- + +Currently there is no Open vSwitch support for networks that are managed by +libvirt (e.g. NAT). As of now, only bridged networks are supported (those where +the user has to manually create the bridge). + +Setup +----- + +First, create the Open vSwitch bridge by using the ovs-vsctl utility (this must +be done with administrative privileges):: + + $ ovs-vsctl add-br ovsbr + +Once that is done, create a VM, if necessary, and edit its Domain XML file:: + + $ virsh edit + +Lookup in the Domain XML file the ```` section. There should be one +such XML section for each interface the VM has:: + + + + +
+ + +And change it to something like this:: + + + + + +
+ + +The interface type must be set to ``bridge``. The ```` XML element +specifies to which bridge this interface will be attached to. The +```` element indicates that the bridge in ```` element is +an Open vSwitch bridge. + +Then (re)start the VM and verify if the guest's vnet interface is attached to +the ovsbr bridge:: + + $ ovs-vsctl show + +Troubleshooting +--------------- + +If the VM does not want to start, then try to run the libvirtd process either +from the terminal, so that all errors are printed in console, or inspect +Libvirt/Open vSwitch log files for possible root cause. + +Bug Reporting +------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/howto/selinux.rst b/Documentation/howto/selinux.rst new file mode 100644 index 000000000..ad556da58 --- /dev/null +++ b/Documentation/howto/selinux.rst @@ -0,0 +1,174 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +========================= +Open vSwitch with SELinux +========================= + +Security-Enhanced Linux (SELinux) is a Linux kernel security module that limits +"the malicious things" that certain processes, including OVS, can do to the +system in case they get compromised. In our case SELinux basically serves as +the "second line of defense" that limits the things that OVS processes are +allowed to do. The "first line of defense" is proper input validation that +eliminates code paths that could be used by attacker to do any sort of "escape +attacks", such as file name escape, shell escape, command line argument escape, +buffer escape. Since developers don't always implement proper input validation, +then SELinux Access Control's goal is to confine damage of such attacks, if +they turned out to be possible. + +Besides Type Enforcement there are other SELinux features, but they are out of +scope for this document. + +Currently there are two SELinux policies for Open vSwitch: + +- the one that ships with your Linux distribution (i.e. + selinux-policy-targeted package) + +- the one that ships with OVS (i.e. openvswitch-selinux-policy package) + +Limitations +----------- + +If Open vSwitch is directly started from command line, then it will run under +``unconfined_t`` SELinux domain that basically lets daemon to do whatever it +likes. This is very important for developers to understand, because they might +introduced code in OVS that invokes new system calls that SELinux policy did +not anticipate. This means that their feature may have worked out just fine +for them. However, if someone else would try to run the same code when Open +vSwitch is started through systemctl, then Open vSwitch would get Permission +Denied errors. + +Currently the only distributions that enforce SELinux on OVS by default are +RHEL, CentOS and Fedora. While Ubuntu and Debian also have some SELinux +support, they run Open vSwitch under the unrestricted ``unconfined`` domain. +Also, it seems that Ubuntu is leaning towards Apparmor that works slightly +differently than SELinux. + +SELinux and Open vSwitch are moving targets. What this means is that, if you +solely rely on your Linux distribution's SELinux policy, then this policy might +not have correctly anticipated that a newer Open vSwitch version needs extra +white list rules. However, if you solely rely on SELinux policy that ships +with Open vSwitch, then Open vSwitch developers might not have correctly +anticipated the feature set that your SELinux implementation supports. + +Installation +------------ + +Refer to :doc:`/intro/install/fedora` for instructions on how to build all Open +vSwitch rpm packages. + +Once the package is built, install it on your Linux distribution:: + + $ dnf install openvswitch-selinux-policy-2.4.1-1.el7.centos.noarch.rpm + +Restart Open vSwitch:: + + $ systemctl restart openvswitch + +Troubleshooting +--------------- + +When SELinux was implemented some of the standard system utilities acquired +``-Z`` flag (e.g. ``ps -Z``, ``ls -Z``). For example, to find out under which +SELinux security domain process runs, use:: + + $ ps -AZ | grep ovs-vswitchd + system_u:system_r:openvswitch_t:s0 854 ? ovs-vswitchd + +To find out the SELinux label of file or directory, use:: + + $ ls -Z /etc/openvswitch/conf.db + system_u:object_r:openvswitch_rw_t:s0 /etc/openvswitch/conf.db + +If, for example, SELinux policy for Open vSwitch is too strict, then you might +see in Open vSwitch log files "Permission Denied" errors:: + + $ cat /var/log/openvswitch/ovs-vswitchd.log + vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log + ovs_numa|INFO|Discovered 2 CPU cores on NUMA node 0 + ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU cores + reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting... + reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected + netlink_socket|ERR|fcntl: Permission denied + dpif_netlink|ERR|Generic Netlink family 'ovs_datapath' does not exist. + The Open vSwitch kernel module is probably not loaded. + dpif|WARN|failed to enumerate system datapaths: Permission denied + dpif|WARN|failed to create datapath ovs-system: Permission denied + +However, not all "Permission denied" errors are caused by SELinux. So, before +blaming too strict SELinux policy, make sure that indeed SELinux was the one +that denied OVS access to certain resources, for example, run: + + $ grep "openvswitch_t" /var/log/audit/audit.log | tail + type=AVC msg=audit(1453235431.640:114671): avc: denied { getopt } for pid=4583 comm="ovs-vswitchd" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_generic_socket permissive=0 + +If SELinux denied OVS access to certain resources, then make sure that you have +installed our SELinux policy package that "loosens" up distribution's SELinux +policy:: + + $ rpm -qa | grep openvswitch-selinux + openvswitch-selinux-policy-2.4.1-1.el7.centos.noarch + +Then verify that this module was indeed loaded:: + + # semodule -l | grep openvswitch + openvswitch-custom 1.0 + openvswitch 1.1.1 + +If you still see Permission denied errors, then take a look into +``selinux/openvswitch.te`` file in the OVS source tree and try to add white +list rules. This is really simple, just run SELinux audit2allow tool:: + + $ grep "openvswitch_t" /var/log/audit/audit.log | audit2allow -M ovslocal + +Contributing SELinux policy patches +----------------------------------- + +Here are few things to consider before proposing SELinux policy patches to Open +vSwitch developer mailing list: + +1. The SELinux policy that resides in Open vSwitch source tree amends SELinux + policy that ships with your distributions. + + Implications of this are that it is assumed that the distribution's Open + vSwitch SELinux module must be already loaded to satisfy dependencies. + +2. The SELinux policy that resides in Open vSwitch source tree must work on all + currently relevant Linux distributions. + + Implications of this are that you should use only those SELinux policy + features that are supported by the lowest SELinux version out there. + Typically this means that you should test your SELinux policy changes on the + oldest RHEL or CentOS version that this OVS version supports. Refer to + :doc:`/intro/install/fedora` to find out this. + +3. The SELinux policy is enforced only when state transition to + ``openvswitch_t`` domain happens. + + Implications of this are that perhaps instead of loosening SELinux policy + you can do certain things at the time rpm package is installed. + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/howto/ssl.rst b/Documentation/howto/ssl.rst new file mode 100644 index 000000000..3085206fb --- /dev/null +++ b/Documentation/howto/ssl.rst @@ -0,0 +1,338 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +===================== +Open vSwitch with SSL +===================== + +If you plan to configure Open vSwitch to connect across the network to an +OpenFlow controller, then we recommend that you build Open vSwitch with +OpenSSL. SSL support ensures integrity and confidentiality of the OpenFlow +connections, increasing network security. + +This document describes how to configure an Open vSwitch to connect to an +OpenFlow controller over SSL. Refer to :doc:`/intro/install/general`. for +instructions on building Open vSwitch with SSL support. + +Open vSwitch uses TLS version 1.0 or later (TLSv1), as specified by RFC 2246, +which is very similar to SSL version 3.0. TLSv1 was released in January 1999, +so all current software and hardware should implement it. + +This document assumes basic familiarity with public-key cryptography and +public-key infrastructure. + +SSL Concepts for OpenFlow +------------------------- + +This section is an introduction to the public-key infrastructure architectures +that Open vSwitch supports for SSL authentication. + +To connect over SSL, every Open vSwitch must have a unique private/public key +pair and a certificate that signs that public key. Typically, the Open vSwitch +generates its own public/private key pair. There are two common ways to obtain +a certificate for a switch: + +* Self-signed certificates: The Open vSwitch signs its certificate with its own + private key. In this case, each switch must be individually approved by the + OpenFlow controller(s), since there is no central authority. + + This is the only switch PKI model currently supported by NOX + (http://noxrepo.org). + +* Switch certificate authority: A certificate authority (the "switch CA") signs + each Open vSwitch's public key. The OpenFlow controllers then check that any + connecting switches' certificates are signed by that certificate authority. + + This is the only switch PKI model supported by the simple OpenFlow controller + included with Open vSwitch. + +Each Open vSwitch must also have a copy of the CA certificate for the +certificate authority that signs OpenFlow controllers' keys (the "controller +CA" certificate). Typically, the same controller CA certificate is installed +on all of the switches within a given administrative unit. There are two +common ways for a switch to obtain the controller CA certificate: + +* Manually copy the certificate to the switch through some secure means, e.g. + using a USB flash drive, or over the network with "scp", or even FTP or HTTP + followed by manual verification. + +* Open vSwitch "bootstrap" mode, in which Open vSwitch accepts and saves the + controller CA certificate that it obtains from the OpenFlow controller on its + first connection. Thereafter the switch will only connect to controllers + signed by the same CA certificate. + +Establishing a Public Key Infrastructure +---------------------------------------- + +Open vSwitch can make use of your existing public key infrastructure. If you +already have a PKI, you may skip forward to the next section. Otherwise, if +you do not have a PKI, the ovs-pki script included with Open vSwitch can help. +To create an initial PKI structure, invoke it as: + +:: + + $ ovs-pki init + +This will create and populate a new PKI directory. The default location for +the PKI directory depends on how the Open vSwitch tree was configured (to see +the configured default, look for the ``--dir`` option description in the output +of ``ovs-pki --help``). + +The pki directory contains two important subdirectories. The `controllerca` +subdirectory contains controller CA files, including the following: + +`cacert.pem` + Root certificate for the controller certificate authority. Each Open vSwitch + must have a copy of this file to allow it to authenticate valid controllers. + +`private/cakey.pem` + Private signing key for the controller certificate authority. This file must + be kept secret. There is no need for switches or controllers to have a copy + of it. + +The `switchca` subdirectory contains switch CA files, analogous to those in the +`controllerca` subdirectory: + +`cacert.pem` + Root certificate for the switch certificate authority. The OpenFlow + controller must have this file to enable it to authenticate valid switches. + +`private/cakey.pem` + Private signing key for the switch certificate authority. This file must be + kept secret. There is no need for switches or controllers to have a copy of + it. + +After you create the initial structure, you can create keys and certificates +for switches and controllers with ovs-pki. Refer to the ovs-pki(8) manage for +complete details. A few examples of its use follow: + +Controller Key Generation +~~~~~~~~~~~~~~~~~~~~~~~~~ + +To create a controller private key and certificate in files named +ctl-privkey.pem and ctl-cert.pem, run the following on the machine that +contains the PKI structure: + +:: + + $ ovs-pki req+sign ctl controller + +ctl-privkey.pem and ctl-cert.pem would need to be copied to the controller for +its use at runtime. If, for testing purposes, you were to use +ovs-testcontroller, the simple OpenFlow controller included with Open vSwitch, +then the --private-key and --certificate options, respectively, would point to +these files. + +It is very important to make sure that no stray copies of ctl-privkey.pem are +created, because they could be used to impersonate the controller. + +Switch Key Generation with Self-Signed Certificates +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you are using self-signed certificates (see "SSL Concepts for OpenFlow"), +this is one way to create an acceptable certificate for your controller to +approve. + +1. Run the following command on the Open vSwitch itself:: + + $ ovs-pki self-sign sc + + .. note:: + This command does not require a copy of any of the PKI files generated by + ``ovs-pki init``, and you should not copy them to the switch because some + of them have contents that must remain secret for security.) + + The ``ovs-pki self-sign`` command has the following output: + + sc-privkey.pem + the switch private key file. For security, the contents of this file must + remain secret. There is ordinarily no need to copy this file off the Open + vSwitch. + + sc-cert.pem + the switch certificate, signed by the switch's own private key. Its + contents are not a secret. + +2. Optionally, copy `controllerca/cacert.pem` from the machine that has the + OpenFlow PKI structure and verify that it is correct. (Otherwise, you will + have to use CA certificate bootstrapping when you configure Open vSwitch in + the next step.) + +3. Configure Open vSwitch to use the keys and certificates (see "Configuring + SSL Support", below). + +Switch Key Generation with a Switch PKI (Easy Method) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you are using a switch PKI (see "SSL Concepts for OpenFlow", above), this +method of switch key generation is a little easier than the alternate method +described below, but it is also a little less secure because it requires +copying a sensitive private key from file from the machine hosting the PKI to +the switch. + +1. Run the following on the machine that contains the PKI structure:: + + $ ovs-pki req+sign sc switch + + This command has the following output: + + sc-privkey.pem + the switch private key file. For security, the contents of this file must + remain secret. + + sc-cert.pem + the switch certificate. Its contents are not a secret. + +2. Copy sc-privkey.pem and sc-cert.pem, plus controllerca/cacert.pem, to the + Open vSwitch. + +3. Delete the copies of sc-privkey.pem and sc-cert.pem on the PKI machine and + any other copies that may have been made in transit. It is very important + to make sure that there are no stray copies of sc-privkey.pem, because they + could be used to impersonate the switch. + + .. warning:: + Don't delete controllerca/cacert.pem! It is not security-sensitive and + you will need it to configure additional switches. + +4. Configure Open vSwitch to use the keys and certificates (see "Configuring + SSL Support", below). + +Switch Key Generation with a Switch PKI (More Secure) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you are using a switch PKI (see "SSL Concepts for OpenFlow", above), then, +compared to the previous method, the method described here takes a little more +work, but it does not involve copying the private key from one machine to +another, so it may also be a little more secure. + +1. Run the following command on the Open vSwitch itself:: + + $ ovs-pki req sc + + .. note:: + This command does not require a copy of any of the PKI files generated by + "ovs-pki init", and you should not copy them to the switch because some of + them have contents that must remain secret for security. + + The "ovs-pki req" command has the following output: + + sc-privkey.pem + the switch private key file. For security, the contents of this file must + remain secret. There is ordinarily no need to copy this file off the Open + vSwitch. + + sc-req.pem + the switch "certificate request", which is essentially the switch's public + key. Its contents are not a secret. + + a fingerprint + this is output on stdout. + +2. Write the fingerprint down on a slip of paper and copy `sc-req.pem` to the + machine that contains the PKI structure. + +3. On the machine that contains the PKI structure, run:: + + $ ovs-pki sign sc switch + + This command will output a fingerprint to stdout and request that you verify + it. Check that it is the same as the fingerprint that you wrote down on the + slip of paper before you answer "yes". + + ``ovs-pki sign`` creates a file named `sc-cert.pem`, which is the switch + certificate. Its contents are not a secret. + +4. Copy the generated `sc-cert.pem`, plus `controllerca/cacert.pem` from the + PKI structure, to the Open vSwitch, and verify that they were copied + correctly. + + You may delete `sc-cert.pem` from the machine that hosts the PKI + structure now, although it is not important that you do so. + + .. warning:: + Don't delete `controllerca/cacert.pem`! It is not security-sensitive and + you will need it to configure additional switches. + +5. Configure Open vSwitch to use the keys and certificates (see "Configuring + SSL Support", below). + +Configuring SSL Support +----------------------- + +SSL configuration requires three additional configuration files. The first two +of these are unique to each Open vSwitch. If you used the instructions above +to build your PKI, then these files will be named `sc-privkey.pem` and +`sc-cert.pem`, respectively: + +- A private key file, which contains the private half of an RSA or DSA key. + + This file can be generated on the Open vSwitch itself, for the greatest + security, or it can be generated elsewhere and copied to the Open vSwitch. + + The contents of the private key file are secret and must not be exposed. + +- A certificate file, which certifies that the private key is that of a + trustworthy Open vSwitch. + + This file has to be generated on a machine that has the private key for the + switch certification authority, which should not be an Open vSwitch; ideally, + it should be a machine that is not networked at all. + + The certificate file itself is not a secret. + +The third configuration file is typically the same across all the switches in a +given administrative unit. If you used the instructions above to build your +PKI, then this file will be named `cacert.pem`: + +- The root certificate for the controller certificate authority. The Open + vSwitch verifies it that is authorized to connect to an OpenFlow controller + by verifying a signature against this CA certificate. + +Once you have these files, configure ovs-vswitchd to use them using the +``ovs-vsctl set-ssl`` command, e.g.:: + + $ ovs-vsctl set-ssl /etc/openvswitch/sc-privkey.pem \ + /etc/openvswitch/sc-cert.pem /etc/openvswitch/cacert.pem + +Substitute the correct file names, of course, if they differ from the ones used +above. You should use absolute file names (ones that begin with ``/``), +because ovs-vswitchd's current directory is unrelated to the one from which you +run ovs-vsctl. + +If you are using self-signed certificates (see "SSL Concepts for OpenFlow") and +you did not copy controllerca/cacert.pem from the PKI machine to the Open +vSwitch, then add the ``--bootstrap`` option, e.g.:: + + $ ovs-vsctl -- --bootstrap set-ssl /etc/openvswitch/sc-privkey.pem \ + /etc/openvswitch/sc-cert.pem /etc/openvswitch/cacert.pem + +After you have added all of these configuration keys, you may specify ``ssl:`` +connection methods elsewhere in the configuration database. ``tcp:`` connection +methods are still allowed even after SSL has been configured, so for security +you should use only ``ssl:`` connections. + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/index.rst b/Documentation/index.rst index a46407a06..be794f169 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -45,7 +45,17 @@ The Open vSwitch documentation is organised into multiple sections: First Steps ----------- -**TODO** +Getting started with Open vSwitch (OVS) or Open Virtual Network (OVN) for Open +vSwitch? Start here. + +- **Install:** :doc:`intro/install/general` | + :doc:`intro/install/userspace` | + :doc:`intro/install/netbsd` | + :doc:`intro/install/windows` | + :doc:`intro/install/xenserver` | + :doc:`intro/install/dpdk` + +- **Tutorials:** **TODO** Deeper Dive ----------- diff --git a/Documentation/internals/contributing/documentation-style.rst b/Documentation/internals/contributing/documentation-style.rst index 756f1678d..c32921e70 100644 --- a/Documentation/internals/contributing/documentation-style.rst +++ b/Documentation/internals/contributing/documentation-style.rst @@ -29,8 +29,8 @@ Open vSwitch Documentation Style This file describes the documentation style used in all documentation found in Open vSwitch. Documentation includes any documents found in ``Documentation`` -along with any ``README``, ``INSTALL``, or generally ``rst`` suffixed documents -found in the project tree. +along with any ``README``, ``MAINTAINERS``, or generally ``rst`` suffixed +documents found in the project tree. reStructuredText vs. Sphinx --------------------------- diff --git a/Documentation/intro/install/bash-completion.rst b/Documentation/intro/install/bash-completion.rst new file mode 100644 index 000000000..6df60c65c --- /dev/null +++ b/Documentation/intro/install/bash-completion.rst @@ -0,0 +1,111 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +==================================== +Bash command-line completion scripts +==================================== + +There are two completion scripts available, ovs-appctl-bashcomp.bash and +ovs-vsctl-bashcomp.bash respectively. + +ovs-appctl-bashcomp +------------------- + +ovs-appctl-bashcomp.bash adds bash command-line completion support for +ovs-appctl, ovs-dpctl, ovs-ofctl and ovsdb-tool commands. + +Features +~~~~~~~~ + +- Display available completion or complete on unfinished user input (long + option, subcommand, and argument). + +- Subcommand hints + +- Convert between keywords like ``bridge``, ``port``, ``interface``, or ``dp`` + and the available record in ovsdb. + +Limitations +~~~~~~~~~~~ + +- Only supports a small set of important keywords (``dp``, ``datapath``, ``bridge``, ``switch``, + ``port``, ``interface``, ``iface``). + +- Does not support parsing of nested options. For example: + + + :: + + $ ovsdb-tool create [db [schema]] + +- Does not support expansion on repeated argument. For example: + + :: + + $ ovs-dpctl show [dp...]). + +- Only supports matching on long options, and only in the format ``--option + [arg]``. Do not use ``--option=[arg]``. + +ovs-vsctl-bashcomp +------------------- + +ovs-vsctl-bashcomp.bash adds bash command-line completion support for ovs-vsctl +command. + +Features +~~~~~~~~ + +- Display available completion and complete on user input for global/local + options, command, and argument. + +- Query database and expand keywords like ``table``, ``record``, ``column``, or + ``key``, to available completions. + +- Deal with argument relations like 'one and more', 'zero or one'. + +- Complete multiple ovs-vsctl commands cascaded via ``--``. + +Limitations +~~~~~~~~~~~ + +Completion of very long ``ovs-vsctl`` commands can take up to several seconds. + +Usage +----- + +The bashcomp scripts should be placed at ``/etc/bash_completion.d/`` to be +available for all bash sessions. Running ``make install`` will place the +scripts to ``$(sysconfdir)/bash_completion.d/``, thus, the user should specify +``--sysconfdir=/etc`` at configuration. If OVS is installed from packages, the +scripts will automatically be placed inside ``/etc/bash_completion.d/``. + +If you just want to run the scripts in one bash, you can remove them from +``/etc/bash_completion.d/`` and run the scripts via ``. +ovs-appctl-bashcomp.bash`` or ``. ovs-vsctl-bashcomp.bash``. + +Tests +----- + +Unit tests are added in ``tests/completion.at`` and integrated into autotest +framework. To run the tests, just run ``make check``. diff --git a/Documentation/intro/install/debian.rst b/Documentation/intro/install/debian.rst new file mode 100644 index 000000000..648433144 --- /dev/null +++ b/Documentation/intro/install/debian.rst @@ -0,0 +1,123 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +================================= +Debian Packaging for Open vSwitch +================================= + +This document describes how to build Debian packages for Open vSwitch. To +install Open vSwitch on Debian without building Debian packages, refer to +:doc:`general` instead. + +.. note:: + These instructions should also work on Ubuntu and other Debian derivative + distributions. + +Before You Begin +---------------- + +Before you begin, consider whether you really need to build packages yourself. +Debian "wheezy" and "sid", as well as recent versions of Ubuntu, contain +pre-built Debian packages for Open vSwitch. It is easier to install these than +to build your own. To use packages from your distribution, skip ahead to +"Installing .deb Packages", below. + +Building Open vSwitch Debian packages +------------------------------------- + +You may build from an Open vSwitch distribution tarball or from an Open vSwitch +Git tree with these instructions. + +You do not need to be the superuser to build the Debian packages. + +1. Install the "build-essential" and "fakeroot" packages. For example:: + + $ apt-get install build-essential fakeroot + +2. Obtain and unpack an Open vSwitch source distribution and ``cd`` into its + top level directory. + +3. Install the build dependencies listed under "Build-Depends:" near the top of + ``debian/control``. You can install these any way you like, e.g. with + ``apt-get install``. + +Check your work by running ``dpkg-checkbuilddeps`` in the top level of your ovs +directory. If you've installed all the dependencies properly, +``dpkg-checkbuilddeps`` will exit without printing anything. If you forgot to +install some dependencies, it will tell you which ones. + +4. Build the package:: + + $ fakeroot debian/rules binary + + This will do a serial build that runs the unit tests. This will take + approximately 8 to 10 minutes. If you prefer, you can run a faster parallel + build:: + + $ DEB_BUILD_OPTIONS='parallel=8' fakeroot debian/rules binary + + If you are in a big hurry, you can even skip the unit tests:: + + $ DEB_BUILD_OPTIONS='parallel=8 nocheck' fakeroot debian/rules binary + +.. note:: + + There are a few pitfalls in the Debian packaging building system so that, + occasionally, you may find that in a tree that you have using for a while, + the build command above exits immediately without actually building anything. + To fix the problem, run:: + + $ fakeroot debian/rules clean + + or start over from a fresh copy of the source tree. + +5. The generated .deb files will be in the parent directory of the Open vSwitch + source distribution. + +Installing .deb Packages +------------------------ + +These instructions apply to installing from Debian packages that you built +yourself, as described in the previous section, or from packages provided by +Debian or a Debian derivative distribution such as Ubuntu. In the former case, +use a command such as ``dpkg -i`` to install the .deb files that you build, and +in the latter case use a program such as ``apt-get`` or ``aptitude`` to +download and install the provided packages. + +.. important:: + You must be superuser to install Debian packages. + +1. Start by installing an Open vSwitch kernel module. See + ``debian/openvswitch-switch.README.Debian`` for the available options. + +2. Install the ``openvswitch-switch`` and ``openvswitch-common`` packages. + These packages include the core userspace components of the switch. + +Open vSwitch ``.deb`` packages not mentioned above are rarely useful. Refer to +their individual package descriptions to find out whether any of them are +useful to you. + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/intro/install/dpdk-advanced.rst b/Documentation/intro/install/dpdk-advanced.rst new file mode 100644 index 000000000..44d1cd78c --- /dev/null +++ b/Documentation/intro/install/dpdk-advanced.rst @@ -0,0 +1,938 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +================================= +Open vSwitch with DPDK (Advanced) +================================= + +The Advanced Install Guide explains how to improve OVS performance when using +DPDK datapath. This guide provides information on tuning, system configuration, +troubleshooting, static code analysis and testcases. + +Building as a Shared Library +---------------------------- + +DPDK can be built as a static or a shared library and shall be linked by +applications using DPDK datapath. When building OVS with DPDK, you can link +Open vSwitch against the shared DPDK library. + +.. note:: + Minor performance loss is seen with OVS when using shared DPDK library as + compared to static library. + +To build Open vSwitch using DPDK as a shared library, first refer to +:doc:`/intro/install/dpdk` for download instructions for DPDK and OVS. + +Once DPDK and OVS have been downloaded, you must configure the DPDK library +accordingly. Simply set ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in +``config/common_base``, then build and install DPDK. Once done, DPDK can be +built as usual. For example:: + + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET + $ make install T=$DPDK_TARGET DESTDIR=install + +Once DPDK is built, export the DPDK shared library location and setup OVS as +detailed in :doc:`/intro/install/dpdk`:: + + $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib + +System Configuration +-------------------- + +To achieve optimal OVS performance, the system can be configured and that +includes BIOS tweaks, Grub cmdline additions, better understanding of NUMA +nodes and apt selection of PCIe slots for NIC placement. + +Recommended BIOS Settings +~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: Recommended BIOS Settings + :header-rows: 1 + + * - Setting + - Value + * - C3 Power State + - Disabled + * - C6 Power State + - Disabled + * - MLC Streamer + - Enabled + * - MLC Spacial Prefetcher + - Enabled + * - DCU Data Prefetcher + - Enabled + * - DCA + - Enabled + * - CPU Power and Performance + - Performance + * - Memeory RAS and Performance Config -> NUMA optimized + - Enabled + +PCIe Slot Selection +~~~~~~~~~~~~~~~~~~~ + +The fastpath performance can be affected by factors related to the placement of +the NIC, such as channel speeds between PCIe slot and CPU or the proximity of +PCIe slot to the CPU cores running the DPDK application. Listed below are the +steps to identify right PCIe slot. + +#. Retrieve host details using ``dmidecode``. For example:: + + $ dmidecode -t baseboard | grep "Product Name" + +#. Download the technical specification for product listed, e.g: S2600WT2 + +#. Check the Product Architecture Overview on the Riser slot placement, CPU + sharing info and also PCIe channel speeds + + For example: On S2600WT, CPU1 and CPU2 share Riser Slot 1 with Channel speed + between CPU1 and Riser Slot1 at 32GB/s, CPU2 and Riser Slot1 at 16GB/s. + Running DPDK app on CPU1 cores and NIC inserted in to Riser card Slots will + optimize OVS performance in this case. + +#. Check the Riser Card #1 - Root Port mapping information, on the available + slots and individual bus speeds. In S2600WT slot 1, slot 2 has high bus + speeds and are potential slots for NIC placement. + +Advanced Hugepage Setup +~~~~~~~~~~~~~~~~~~~~~~~ + +Allocate and mount 1 GB hugepages. + +- For persistent allocation of huge pages, add the following options to the + kernel bootline:: + + default_hugepagesz=1GB hugepagesz=1G hugepages=N + + For platforms supporting multiple huge page sizes, add multiple options:: + + default_hugepagesz= hugepagesz= hugepages=N + + where: + + ``N`` + number of huge pages requested + ``size`` + huge page size with an optional suffix ``[kKmMgG]`` + +- For run-time allocation of huge pages:: + + $ echo N > /sys/devices/system/node/nodeX/hugepages/hugepages-1048576kB/nr_hugepages + + where: + + ``N`` + number of huge pages requested + ``X`` + NUMA Node + + .. note:: + For run-time allocation of 1G huge pages, Contiguous Memory Allocator + (``CONFIG_CMA``) has to be supported by kernel, check your Linux distro. + +Now mount the huge pages, if not already done so:: + + $ mount -t hugetlbfs -o pagesize=1G none /dev/hugepages + +Enable HyperThreading +~~~~~~~~~~~~~~~~~~~~~ + +With HyperThreading, or SMT, enabled, a physical core appears as two logical +cores. SMT can be utilized to spawn worker threads on logical cores of the same +physical core there by saving additional cores. + +With DPDK, when pinning pmd threads to logical cores, care must be taken to set +the correct bits of the ``pmd-cpu-mask`` to ensure that the pmd threads are +pinned to SMT siblings. + +Take a sample system configuration, with 2 sockets, 2 * 10 core processors, HT +enabled. This gives us a total of 40 logical cores. To identify the physical +core shared by two logical cores, run:: + + $ cat /sys/devices/system/cpu/cpuN/topology/thread_siblings_list + +where ``N`` is the logical core number. + +In this example, it would show that cores ``1`` and ``21`` share the same +physical core. As cores are counted from 0, the ``pmd-cpu-mask`` can be used +to enable these two pmd threads running on these two logical cores (one +physical core) is:: + + $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x200002 + +Isolate Cores +~~~~~~~~~~~~~ + +The ``isolcpus`` option can be used to isolate cores from the Linux scheduler. +The isolated cores can then be used to dedicatedly run HPC applications or +threads. This helps in better application performance due to zero context +switching and minimal cache thrashing. To run platform logic on core 0 and +isolate cores between 1 and 19 from scheduler, add ``isolcpus=1-19`` to GRUB +cmdline. + +.. note:: + It has been verified that core isolation has minimal advantage due to mature + Linux scheduler in some circumstances. + +NUMA/Cluster-on-Die +~~~~~~~~~~~~~~~~~~~ + +Ideally inter-NUMA datapaths should be avoided where possible as packets will +go across QPI and there may be a slight performance penalty when compared with +intra NUMA datapaths. On Intel Xeon Processor E5 v3, Cluster On Die is +introduced on models that have 10 cores or more. This makes it possible to +logically split a socket into two NUMA regions and again it is preferred where +possible to keep critical datapaths within the one cluster. + +It is good practice to ensure that threads that are in the datapath are pinned +to cores in the same NUMA area. e.g. pmd threads and QEMU vCPUs responsible for +forwarding. If DPDK is built with ``CONFIG_RTE_LIBRTE_VHOST_NUMA=y``, vHost +User ports automatically detect the NUMA socket of the QEMU vCPUs and will be +serviced by a PMD from the same node provided a core on this node is enabled in +the ``pmd-cpu-mask``. ``libnuma`` packages are required for this feature. + +Compiler Optimizations +~~~~~~~~~~~~~~~~~~~~~~ + +The default compiler optimization level is ``-O2``. Changing this to more +aggressive compiler optimization such as ``-O3 -march=native`` with +gcc (verified on 5.3.1) can produce performance gains though not siginificant. +``-march=native`` will produce optimized code on local machine and should be +used when software compilation is done on Testbed. + +Performance Tuning +------------------ + +Affinity +~~~~~~~~ + +For superior performance, DPDK pmd threads and Qemu vCPU threads needs to be +affinitized accordingly. + +- PMD thread Affinity + + A poll mode driver (pmd) thread handles the I/O of all DPDK interfaces + assigned to it. A pmd thread shall poll the ports for incoming packets, + switch the packets and send to tx port. pmd thread is CPU bound, and needs + to be affinitized to isolated cores for optimum performance. + + By setting a bit in the mask, a pmd thread is created and pinned to the + corresponding CPU core. e.g. to run a pmd thread on core 2:: + + $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x4 + + .. note:: + pmd thread on a NUMA node is only created if there is at least one DPDK + interface from that NUMA node added to OVS. + +- QEMU vCPU thread Affinity + + A VM performing simple packet forwarding or running complex packet pipelines + has to ensure that the vCPU threads performing the work has as much CPU + occupancy as possible. + + For example, on a multicore VM, multiple QEMU vCPU threads shall be spawned. + When the DPDK ``testpmd`` application that does packet forwarding is invoked, + the ``taskset`` command should be used to affinitize the vCPU threads to the + dedicated isolated cores on the host system. + +Multiple Poll-Mode Driver Threads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +With pmd multi-threading support, OVS creates one pmd thread for each NUMA node +by default. However, in cases where there are multiple ports/rxq's producing +traffic, performance can be improved by creating multiple pmd threads running +on separate cores. These pmd threads can share the workload by each being +responsible for different ports/rxq's. Assignment of ports/rxq's to pmd threads +is done automatically. + +A set bit in the mask means a pmd thread is created and pinned to the +corresponding CPU core. For example, to run pmd threads on core 1 and 2:: + + $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x6 + +When using dpdk and dpdkvhostuser ports in a bi-directional VM loopback as +shown below, spreading the workload over 2 or 4 pmd threads shows significant +improvements as there will be more total CPU occupancy available:: + + NIC port0 <-> OVS <-> VM <-> OVS <-> NIC port 1 + +DPDK Physical Port Rx Queues +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + $ ovs-vsctl set Interface options:n_rxq= + +The command above sets the number of rx queues for DPDK physical interface. +The rx queues are assigned to pmd threads on the same NUMA node in a +round-robin fashion. + +DPDK Physical Port Queue Sizes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + $ ovs-vsctl set Interface dpdk0 options:n_rxq_desc= + $ ovs-vsctl set Interface dpdk0 options:n_txq_desc= + +The command above sets the number of rx/tx descriptors that the NIC associated +with dpdk0 will be initialised with. + +Different ``n_rxq_desc`` and ``n_txq_desc`` configurations yield different +benefits in terms of throughput and latency for different scenarios. +Generally, smaller queue sizes can have a positive impact for latency at the +expense of throughput. The opposite is often true for larger queue sizes. +Note: increasing the number of rx descriptors eg. to 4096 may have a negative +impact on performance due to the fact that non-vectorised DPDK rx functions may +be used. This is dependant on the driver in use, but is true for the commonly +used i40e and ixgbe DPDK drivers. + +Exact Match Cache +~~~~~~~~~~~~~~~~~ + +Each pmd thread contains one Exact Match Cache (EMC). After initial flow setup +in the datapath, the EMC contains a single table and provides the lowest level +(fastest) switching for DPDK ports. If there is a miss in the EMC then the next +level where switching will occur is the datapath classifier. Missing in the +EMC and looking up in the datapath classifier incurs a significant performance +penalty. If lookup misses occur in the EMC because it is too small to handle +the number of flows, its size can be increased. The EMC size can be modified by +editing the define ``EM_FLOW_HASH_SHIFT`` in ``lib/dpif-netdev.c``. + +As mentioned above, an EMC is per pmd thread. An alternative way of increasing +the aggregate amount of possible flow entries in EMC and avoiding datapath +classifier lookups is to have multiple pmd threads running. + +Rx Mergeable Buffers +~~~~~~~~~~~~~~~~~~~~ + +Rx mergeable buffers is a virtio feature that allows chaining of multiple +virtio descriptors to handle large packet sizes. Large packets are handled by +reserving and chaining multiple free descriptors together. Mergeable buffer +support is negotiated between the virtio driver and virtio device and is +supported by the DPDK vhost library. This behavior is supported and enabled by +default, however in the case where the user knows that rx mergeable buffers are +not needed i.e. jumbo frames are not needed, it can be forced off by adding +``mrg_rxbuf=off`` to the QEMU command line options. By not reserving multiple +chains of descriptors it will make more individual virtio descriptors available +for rx to the guest using dpdkvhost ports and this can improve performance. + +OVS Testcases +------------- + +PHY-VM-PHY (vHost Loopback) +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:doc:`/intro/install/dpdk` details steps for PHY-VM-PHY loopback testcase and +packet forwarding using DPDK testpmd application in the Guest VM. For users +wishing to do packet forwarding using kernel stack below, you need to run the +below commands on the guest:: + + $ ifconfig eth1 1.1.1.2/24 + $ ifconfig eth2 1.1.2.2/24 + $ systemctl stop firewalld.service + $ systemctl stop iptables.service + $ sysctl -w net.ipv4.ip_forward=1 + $ sysctl -w net.ipv4.conf.all.rp_filter=0 + $ sysctl -w net.ipv4.conf.eth1.rp_filter=0 + $ sysctl -w net.ipv4.conf.eth2.rp_filter=0 + $ route add -net 1.1.2.0/24 eth2 + $ route add -net 1.1.1.0/24 eth1 + $ arp -s 1.1.2.99 DE:AD:BE:EF:CA:FE + $ arp -s 1.1.1.99 DE:AD:BE:EF:CA:EE + +PHY-VM-PHY (IVSHMEM) +~~~~~~~~~~~~~~~~~~~~ + +IVSHMEM can also be validated using the PHY-VM-PHY configuration. To begin, +follow the steps described in the :doc:`/intro/install/dpdk` to create and +initialize the database, start ovs-vswitchd and add ``dpdk``-type devices to +bridge ``br0``. Once complete, follow the below steps: + +1. Add DPDK ring port to the bridge:: + + $ ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr + +2. Build modified QEMU + + QEMU must be patched to enable IVSHMEM support:: + + $ cd /usr/src/ + $ wget http://wiki.qemu.org/download/qemu-2.2.1.tar.bz2 + $ tar -jxvf qemu-2.2.1.tar.bz2 + $ cd /usr/src/qemu-2.2.1 + $ wget https://raw.githubusercontent.com/netgroup-polito/un-orchestrator/master/orchestrator/compute_controller/plugins/kvm-libvirt/patches/ivshmem-qemu-2.2.1.patch + $ patch -p1 < ivshmem-qemu-2.2.1.patch + $ ./configure --target-list=x86_64-softmmu --enable-debug --extra-cflags='-g' + $ make -j 4 + +3. Generate QEMU commandline:: + + $ mkdir -p /usr/src/cmdline_generator + $ cd /usr/src/cmdline_generator + $ wget https://raw.githubusercontent.com/netgroup-polito/un-orchestrator/master/orchestrator/compute_controller/plugins/kvm-libvirt/cmdline_generator/cmdline_generator.c + $ wget https://raw.githubusercontent.com/netgroup-polito/un-orchestrator/master/orchestrator/compute_controller/plugins/kvm-libvirt/cmdline_generator/Makefile + $ export RTE_SDK=/usr/src/dpdk-16.11 + $ export RTE_TARGET=x86_64-ivshmem-linuxapp-gcc + $ make + $ ./build/cmdline_generator -m -p dpdkr0 XXX + $ cmdline=`cat OVSMEMPOOL` + +4. Start guest VM:: + + $ export VM_NAME=ivshmem-vm + $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 + $ export QEMU_BIN=/usr/src/qemu-2.2.1/x86_64-softmmu/qemu-system-x86_64 + $ taskset 0x20 $QEMU_BIN -cpu host -smp 2,cores=2 -hda $QCOW2_IMAGE \ + -m 4096 --enable-kvm -name $VM_NAME -nographic -vnc :2 \ + -pidfile /tmp/vm1.pid $cmdline + +5. Build and run the sample ``dpdkr`` app in VM:: + + $ echo 1024 > /proc/sys/vm/nr_hugepages + $ mount -t hugetlbfs nodev /dev/hugepages (if not already mounted) + + # Build the DPDK ring application in the VM + $ export RTE_SDK=/root/dpdk-16.11 + $ export RTE_TARGET=x86_64-ivshmem-linuxapp-gcc + $ make + + # Run dpdkring application + $ ./build/dpdkr -c 1 -n 4 -- -n 0 + # where "-n 0" refers to ring '0' i.e dpdkr0 + +PHY-VM-PHY (vHost Multiqueue) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +vHost Multique functionality can also be validated using the PHY-VM-PHY +configuration. To begin, follow the steps described in +:doc:`/intro/install/dpdk` to create and initialize the database, start +ovs-vswitchd and add ``dpdk``-type devices to bridge ``br0``. Once complete, +follow the below steps: + +1. Configure PMD and RXQs. + + For example, set the number of dpdk port rx queues to at least 2 The number + of rx queues at vhost-user interface gets automatically configured after + virtio device connection and doesn't need manual configuration:: + + $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0xC + $ ovs-vsctl set Interface dpdk0 options:n_rxq=2 + $ ovs-vsctl set Interface dpdk1 options:n_rxq=2 + +2. Instantiate Guest VM using QEMU cmdline + + We must configure with appropriate software versions to ensure this feature + is supported. + + .. list-table:: Recommended BIOS Settings + :header-rows: 1 + + * - Setting + - Value + * - QEMU version + - 2.5.0 + * - QEMU thread affinity + - 2 cores (taskset 0x30) + * - Memory + - 4 GB + * - Cores + - 2 + * - Distro + - Fedora 22 + * - Multiqueue + - Enabled + + To do this, instantiate the guest as follows:: + + $ export VM_NAME=vhost-vm + $ export GUEST_MEM=4096M + $ export QCOW2_IMAGE=/root/Fedora22_x86_64.qcow2 + $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch + $ taskset 0x30 qemu-system-x86_64 -cpu host -smp 2,cores=2 -m 4096M \ + -drive file=$QCOW2_IMAGE --enable-kvm -name $VM_NAME \ + -nographic -numa node,memdev=mem -mem-prealloc \ + -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ + -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ + -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce,queues=2 \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,vectors=6 \ + -chardev socket,id=char2,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ + -netdev type=vhost-user,id=mynet2,chardev=char2,vhostforce,queues=2 \ + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mq=on,vectors=6 + + .. note:: + Queue value above should match the queues configured in OVS, The vector + value should be set to "number of queues x 2 + 2" + +3. Configure the guest interface + + Assuming there are 2 interfaces in the guest named eth0, eth1 check the + channel configuration and set the number of combined channels to 2 for + virtio devices:: + + $ ethtool -l eth0 + $ ethtool -L eth0 combined 2 + $ ethtool -L eth1 combined 2 + + More information can be found in vHost walkthrough section. + +4. Configure kernel packet forwarding + + Configure IP and enable interfaces:: + + $ ifconfig eth0 5.5.5.1/24 up + $ ifconfig eth1 90.90.90.1/24 up + + Configure IP forwarding and add route entries:: + + $ sysctl -w net.ipv4.ip_forward=1 + $ sysctl -w net.ipv4.conf.all.rp_filter=0 + $ sysctl -w net.ipv4.conf.eth0.rp_filter=0 + $ sysctl -w net.ipv4.conf.eth1.rp_filter=0 + $ ip route add 2.1.1.0/24 dev eth1 + $ route add default gw 2.1.1.2 eth1 + $ route add default gw 90.90.90.90 eth1 + $ arp -s 90.90.90.90 DE:AD:BE:EF:CA:FE + $ arp -s 2.1.1.2 DE:AD:BE:EF:CA:FA + + Check traffic on multiple queues:: + + $ cat /proc/interrupts | grep virtio + +vHost Walkthrough +----------------- + +Two types of vHost User ports are available in OVS: + +- vhost-user (``dpdkvhostuser``) + +- vhost-user-client (``dpdkvhostuserclient``) + +vHost User uses a client-server model. The server creates/manages/destroys the +vHost User sockets, and the client connects to the server. Depending on which +port type you use, ``dpdkvhostuser`` or ``dpdkvhostuserclient``, a different +configuration of the client-server model is used. + +For vhost-user ports, Open vSwitch acts as the server and QEMU the client. For +vhost-user-client ports, Open vSwitch acts as the client and QEMU the server. + +vhost-user +~~~~~~~~~~ + +1. Install the prerequisites: + + - QEMU version >= 2.2 + +2. Add vhost-user ports to the switch. + + Unlike DPDK ring ports, DPDK vhost-user ports can have arbitrary names, + except that forward and backward slashes are prohibited in the names. + + For vhost-user, the name of the port type is ``dpdkvhostuser``:: + + $ ovs-vsctl add-port br0 vhost-user-1 -- set Interface vhost-user-1 \ + type=dpdkvhostuser + + This action creates a socket located at + ``/usr/local/var/run/openvswitch/vhost-user-1``, which you must provide to + your VM on the QEMU command line. More instructions on this can be found in + the next section "Adding vhost-user ports to VM" + + .. note:: + If you wish for the vhost-user sockets to be created in a sub-directory of + ``/usr/local/var/run/openvswitch``, you may specify this directory in the + ovsdb like so:: + + $ ovs-vsctl --no-wait \ + set Open_vSwitch . other_config:vhost-sock-dir=subdir` + +3. Add vhost-user ports to VM + + 1. Configure sockets + + Pass the following parameters to QEMU to attach a vhost-user device:: + + -chardev socket,id=char1,path=/usr/local/var/run/openvswitch/vhost-user-1 + -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1 + + where ``vhost-user-1`` is the name of the vhost-user port added to the + switch. + + Repeat the above parameters for multiple devices, changing the chardev + ``path`` and ``id`` as necessary. Note that a separate and different + chardev ``path`` needs to be specified for each vhost-user device. For + example you have a second vhost-user port named ``vhost-user-2``, you + append your QEMU command line with an additional set of parameters:: + + -chardev socket,id=char2,path=/usr/local/var/run/openvswitch/vhost-user-2 + -netdev type=vhost-user,id=mynet2,chardev=char2,vhostforce + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2 + + 2. Configure hugepages + + QEMU must allocate the VM's memory on hugetlbfs. vhost-user ports access + a virtio-net device's virtual rings and packet buffers mapping the VM's + physical memory on hugetlbfs. To enable vhost-user ports to map the VM's + memory into their process address space, pass the following parameters + to QEMU:: + + -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on + -numa node,memdev=mem -mem-prealloc + + 3. Enable multiqueue support (optional) + + QEMU needs to be configured to use multiqueue:: + + -chardev socket,id=char2,path=/usr/local/var/run/openvswitch/vhost-user-2 + -netdev type=vhost-user,id=mynet2,chardev=char2,vhostforce,queues=$q + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mq=on,vectors=$v + + where: + + ``$q`` + The number of queues + ``$v`` + The number of vectors, which is ``$q`` * 2 + 2 + + The vhost-user interface will be automatically reconfigured with + required number of rx and tx queues after connection of virtio device. + Manual configuration of ``n_rxq`` is not supported because OVS will work + properly only if ``n_rxq`` will match number of queues configured in + QEMU. + + A least 2 PMDs should be configured for the vswitch when using + multiqueue. Using a single PMD will cause traffic to be enqueued to the + same vhost queue rather than being distributed among different vhost + queues for a vhost-user interface. + + If traffic destined for a VM configured with multiqueue arrives to the + vswitch via a physical DPDK port, then the number of rxqs should also be + set to at least 2 for that physical DPDK port. This is required to + increase the probability that a different PMD will handle the multiqueue + transmission to the guest using a different vhost queue. + + If one wishes to use multiple queues for an interface in the guest, the + driver in the guest operating system must be configured to do so. It is + recommended that the number of queues configured be equal to ``$q``. + + For example, this can be done for the Linux kernel virtio-net driver + with:: + + $ ethtool -L combined <$q> + + where: + + ``-L`` + Changes the numbers of channels of the specified network device + ``combined`` + Changes the number of multi-purpose channels. + +Configure the VM using libvirt +++++++++++++++++++++++++++++++ + +You can also build and configure the VM using libvirt rather than QEMU by +itself. + +1. Change the user/group, access control policty and restart libvirtd. + + - In ``/etc/libvirt/qemu.conf`` add/edit the following lines:: + + user = "root" + group = "root" + + - Disable SELinux or set to permissive mode:: + + $ setenforce 0 + + - Restart the libvirtd process, For example, on Fedora:: + + $ systemctl restart libvirtd.service + +2. Instantiate the VM + + - Copy the XML configuration described in :doc:`/intro/install/dpdk` + + - Start the VM:: + + $ virsh create demovm.xml + + - Connect to the guest console:: + + $ virsh console demovm + +3. Configure the VM + + The demovm xml configuration is aimed at achieving out of box performance on + VM. + + - The vcpus are pinned to the cores of the CPU socket 0 using ``vcpupin``. + + - Configure NUMA cell and memory shared using ``memAccess='shared'``. + + - Disable ``mrg_rxbuf='off'`` + +Refer to the `libvirt documentation `__ +for more information. + +vhost-user-client +~~~~~~~~~~~~~~~~~ + +1. Install the prerequisites: + + - QEMU version >= 2.7 + +2. Add vhost-user-client ports to the switch. + + Unlike vhost-user ports, the name given to port does not govern the name of + the socket device. ``vhost-server-path`` reflects the full path of the + socket that has been or will be created by QEMU for the given vHost User + client port. + + For vhost-user-client, the name of the port type is + ``dpdkvhostuserclient``:: + + $ VHOST_USER_SOCKET_PATH=/path/to/socker + $ ovs-vsctl add-port br0 vhost-client-1 \ + -- set Interface vhost-client-1 type=dpdkvhostuserclient \ + options:vhost-server-path=$VHOST_USER_SOCKET_PATH + +3. Add vhost-user-client ports to VM + + 1. Configure sockets + + Pass the following parameters to QEMU to attach a vhost-user device:: + + -chardev socket,id=char1,path=$VHOST_USER_SOCKET_PATH,server + -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1 + + where ``vhost-user-1`` is the name of the vhost-user port added to the + switch. + + If the corresponding dpdkvhostuserclient port has not yet been configured + in OVS with ``vhost-server-path=/path/to/socket``, QEMU will print a log + similar to the following:: + + QEMU waiting for connection on: disconnected:unix:/path/to/socket,server + + QEMU will wait until the port is created sucessfully in OVS to boot the VM. + + One benefit of using this mode is the ability for vHost ports to + 'reconnect' in event of the switch crashing or being brought down. Once + it is brought back up, the vHost ports will reconnect automatically and + normal service will resume. + +DPDK Backend Inside VM +~~~~~~~~~~~~~~~~~~~~~~ + +Additional configuration is required if you want to run ovs-vswitchd with DPDK +backend inside a QEMU virtual machine. Ovs-vswitchd creates separate DPDK TX +queues for each CPU core available. This operation fails inside QEMU virtual +machine because, by default, VirtIO NIC provided to the guest is configured to +support only single TX queue and single RX queue. To change this behavior, you +need to turn on ``mq`` (multiqueue) property of all ``virtio-net-pci`` devices +emulated by QEMU and used by DPDK. You may do it manually (by changing QEMU +command line) or, if you use Libvirt, by adding the following string to +```` sections of all network devices used by DPDK:: + + + +Where: + +``N`` + determines how many queues can be used by the guest. + +This requires QEMU >= 2.2. + +QoS +--- + +Assuming you have a vhost-user port transmitting traffic consisting of packets +of size 64 bytes, the following command would limit the egress transmission +rate of the port to ~1,000,000 packets per second:: + + $ ovs-vsctl set port vhost-user0 qos=@newqos -- \ + --id=@newqos create qos type=egress-policer other-config:cir=46000000 \ + other-config:cbs=2048` + +To examine the QoS configuration of the port, run:: + + $ ovs-appctl -t ovs-vswitchd qos/show vhost-user0 + +To clear the QoS configuration from the port and ovsdb, run:: + + $ ovs-vsctl destroy QoS vhost-user0 -- clear Port vhost-user0 qos + +Refer to vswitch.xml for more details on egress-policer. + +Rate Limiting +-------------- + +Here is an example on Ingress Policing usage. Assuming you have a vhost-user +port receiving traffic consisting of packets of size 64 bytes, the following +command would limit the reception rate of the port to ~1,000,000 packets per +second:: + + $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=368000 \ + ingress_policing_burst=1000` + +To examine the ingress policer configuration of the port:: + + $ ovs-vsctl list interface vhost-user0 + +To clear the ingress policer configuration from the port:: + + $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 + +Refer to vswitch.xml for more details on ingress-policer. + +Flow Control +------------ + +Flow control can be enabled only on DPDK physical ports. To enable flow +control support at tx side while adding a port, run:: + + $ ovs-vsctl add-port br0 dpdk0 -- \ + set Interface dpdk0 type=dpdk options:tx-flow-ctrl=true + +Similarly, to enable rx flow control, run:: + + $ ovs-vsctl add-port br0 dpdk0 -- \ + set Interface dpdk0 type=dpdk options:rx-flow-ctrl=true + +To enable flow control auto-negotiation, run:: + + $ ovs-vsctl add-port br0 dpdk0 -- \ + set Interface dpdk0 type=dpdk options:flow-ctrl-autoneg=true + +To turn ON the tx flow control at run time(After the port is being added to +OVS):: + + $ ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=true + +The flow control parameters can be turned off by setting ``false`` to the +respective parameter. To disable the flow control at tx side, run:: + + $ ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=false + +pdump +----- + +Pdump allows you to listen on DPDK ports and view the traffic that is passing +on them. To use this utility, one must have libpcap installed on the system. +Furthermore, DPDK must be built with ``CONFIG_RTE_LIBRTE_PDUMP=y`` and +``CONFIG_RTE_LIBRTE_PMD_PCAP=y``. + +.. warning:: + A performance decrease is expected when using a monitoring application like + the DPDK pdump app. + +To use pdump, simply launch OVS as usual. Then, navigate to the ``app/pdump`` +directory in DPDK, ``make`` the application and run like so:: + + $ sudo ./build/app/dpdk-pdump -- \ + --pdump port=0,queue=0,rx-dev=/tmp/pkts.pcap \ + --server-socket-path=/usr/local/var/run/openvswitch + +The above command captures traffic received on queue 0 of port 0 and stores it +in ``/tmp/pkts.pcap``. Other combinations of port numbers, queues numbers and +pcap locations are of course also available to use. For example, to capture all +packets that traverse port 0 in a single pcap file:: + + $ sudo ./build/app/dpdk-pdump -- \ + --pdump 'port=0,queue=*,rx-dev=/tmp/pkts.pcap,tx-dev=/tmp/pkts.pcap' \ + --server-socket-path=/usr/local/var/run/openvswitch + +``server-socket-path`` must be set to the value of ovs_rundir() which typically +resolves to ``/usr/local/var/run/openvswitch``. + +Many tools are available to view the contents of the pcap file. Once example is +tcpdump. Issue the following command to view the contents of ``pkts.pcap``:: + + $ tcpdump -r pkts.pcap + +More information on the pdump app and its usage can be found in the `DPDK docs +`__. + +Jumbo Frames +------------ + +By default, DPDK ports are configured with standard Ethernet MTU (1500B). To +enable Jumbo Frames support for a DPDK port, change the Interface's +``mtu_request`` attribute to a sufficiently large value. For example, to add a +DPDK Phy port with MTU of 9000:: + + $ ovs-vsctl add-port br0 dpdk0 \ + -- set Interface dpdk0 type=dpdk \ + -- set Interface dpdk0 mtu_request=9000` + +Similarly, to change the MTU of an existing port to 6200:: + + $ ovs-vsctl set Interface dpdk0 mtu_request=6200 + +Some additional configuration is needed to take advantage of jumbo frames with +vHost ports: + +1. *mergeable buffers* must be enabled for vHost ports, as demonstrated in the + QEMU command line snippet below:: + + -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=on + +2. Where virtio devices are bound to the Linux kernel driver in a guest + environment (i.e. interfaces are not bound to an in-guest DPDK driver), the + MTU of those logical network interfaces must also be increased to a + sufficiently large value. This avoids segmentation of Jumbo Frames received + in the guest. Note that 'MTU' refers to the length of the IP packet only, + and not that of the entire frame. + + To calculate the exact MTU of a standard IPv4 frame, subtract the L2 header + and CRC lengths (i.e. 18B) from the max supported frame size. So, to set + the MTU for a 9018B Jumbo Frame:: + + $ ifconfig eth1 mtu 9000 + +When Jumbo Frames are enabled, the size of a DPDK port's mbuf segments are +increased, such that a full Jumbo Frame of a specific size may be accommodated +within a single mbuf segment. + +Jumbo frame support has been validated against 9728B frames, which is the +largest frame size supported by Fortville NIC using the DPDK i40e driver, but +larger frames and other DPDK NIC drivers may be supported. These cases are +common for use cases involving East-West traffic only. + +vsperf +------ + +The vsperf project aims to develop a vSwitch test framework that can be used to +validate the suitability of different vSwitch implementations in a telco +deployment environment. More information can be found on the `OPNFV wiki +`__. + +Bug Reporting +------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst new file mode 100644 index 000000000..7724c8a9f --- /dev/null +++ b/Documentation/intro/install/dpdk.rst @@ -0,0 +1,600 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +====================== +Open vSwitch with DPDK +====================== + +This document describes how to build and install Open vSwitch using a DPDK +datapath. Open vSwitch can use the DPDK library to operate entirely in +userspace. + +.. warning:: + The DPDK support of Open vSwitch is considered 'experimental'. + +Build requirements +------------------ + +In addition to the requirements described in :doc:`general`, building Open +vSwitch with DPDK will require the following: + +- DPDK 16.11 + +- A `DPDK supported NIC`_ + + Only required when physical ports are in use + +- A suitable kernel + + On Linux Distros running kernel version >= 3.0, only `IOMMU` needs to enabled + via the grub cmdline, assuming you are using **VFIO**. For older kernels, + ensure the kernel is built with ``UIO``, ``HUGETLBFS``, + ``PROC_PAGE_MONITOR``, ``HPET``, ``HPET_MMAP`` support. If these are not + present, it will be necessary to upgrade your kernel or build a custom kernel + with these flags enabled. + +.. TODO(stephenfin): drag the below information in from dpdk-advanced + +Detailed system requirements can be found at `DPDK requirements`_, while more +detailed install information can be found in :doc:`dpdk-advanced`. + +.. _DPDK supported NIC: http://dpdk.org/doc/nics +.. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html + +Installing +---------- + +DPDK +~~~~ + +1. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: + + $ cd /usr/src/ + $ wget http://fast.dpdk.org/rel/dpdk-16.11.tar.xz + $ tar xf dpdk-16.11.tar.xz + $ export DPDK_DIR=/usr/src/dpdk-16.11 + $ cd $DPDK_DIR + +2. Configure and install DPDK + + Build and install the DPDK library:: + + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET + $ make install T=$DPDK_TARGET DESTDIR=install + + If IVSHMEM support is required, use a different target:: + + $ export DPDK_TARGET=x86_64-ivshmem-linuxapp-gcc + +.. _DPDK sources: http://dpdk.org/rel + +Install OVS +~~~~~~~~~~~ + +OVS can be installed using different methods. For OVS to use DPDK datapath, it +has to be configured with DPDK support (``--with-dpdk``). + +.. note:: + This section focuses on generic recipe that suits most cases. For + distribution specific instructions, refer to one of the more relevant guides. + +.. _OVS sources: http://openvswitch.org/releases/ + +1. Ensure the standard OVS requirements, described in + :ref:`general-build-reqs`, are installed + +2. Bootstrap, if required, as described in :ref:`general-bootstrapping` + +3. Configure the package using the ``--with-dpdk`` flag:: + + $ ./configure --with-dpdk=$DPDK_BUILD + + where ``DPDK_BUILD`` is the path to the built DPDK library. This can be + skipped if DPDK library is installed in its default location + + .. note:: + While ``--with-dpdk`` is required, you can pass any other configuration + option described in :ref:`general-configuring`. + +4. Build and install OVS, as described in :ref:`general-building` + +Additional information can be found in :doc:`general`. + +Setup +----- + +Setup Hugepages +~~~~~~~~~~~~~~~ + +Allocate a number of 2M Huge pages: + +- For persistent allocation of huge pages, write to hugepages.conf file + in `/etc/sysctl.d`:: + + $ echo 'vm.nr_hugepages=2048' > /etc/sysctl.d/hugepages.conf + +- For run-time allocation of huge pages, use the ``sysctl`` utility:: + + $ sysctl -w vm.nr_hugepages=N # where N = No. of 2M huge pages + +To verify hugepage configuration:: + + $ grep HugePages_ /proc/meminfo + +Mount the hugepages, if not already mounted by default:: + + $ mount -t hugetlbfs none /dev/hugepages`` + +.. _dpdk-vfio: + +Setup DPDK devices using VFIO +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +VFIO is prefered to the UIO driver when using recent versions of DPDK. VFIO +support required support from both the kernel and BIOS. For the former, kernel +version > 3.6 must be used. For the latter, you must enable VT-d in the BIOS +and ensure this is configured via grub. To ensure VT-d is enabled via the BIOS, +run:: + + $ dmesg | grep -e DMAR -e IOMMU + +If VT-d is not enabled in the BIOS, enable it now. + +To ensure VT-d is enabled in the kernel, run:: + + $ cat /proc/cmdline | grep iommu=pt + $ cat /proc/cmdline | grep intel_iommu=on + +If VT-d is not enabled in the kernel, enable it now. + +Once VT-d is correctly configured, load the required modules and bind the NIC +to the VFIO driver:: + + $ modprobe vfio-pci + $ /usr/bin/chmod a+x /dev/vfio + $ /usr/bin/chmod 0666 /dev/vfio/* + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 + $ $DPDK_DIR/tools/dpdk-devbind.py --status + +Setup OVS +~~~~~~~~~ + +Open vSwitch should be started as described in :doc:`general` with the +exception of ovs-vswitchd, which requires some special configuration to enable +DPDK functionality. DPDK configuration arguments can be passed to ovs-vswitchd +via the ``other_config`` column of the ``Open_vSwitch`` table. At a minimum, +the ``dpdk-init`` option must be set to ``true``. For example:: + + $ export DB_SOCK=/usr/local/var/run/openvswitch/db.sock + $ ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true + $ ovs-vswitchd unix:$DB_SOCK --pidfile --detach + +There are many other configuration options, the most important of which are +listed below. Defaults will be provided for all values not explicitly set. + +``dpdk-init`` + Specifies whether OVS should initialize and support DPDK ports. This is a + boolean, and defaults to false. + +``dpdk-lcore-mask`` + Specifies the CPU cores on which dpdk lcore threads should be spawned and + expects hex string (eg '0x123'). + +``dpdk-socket-mem`` + Comma separated list of memory to pre-allocate from hugepages on specific + sockets. + +``dpdk-hugepage-dir`` + Directory where hugetlbfs is mounted + +``vhost-sock-dir`` + Option to set the path to the vhost-user unix socket files. + +If allocating more than one GB hugepage (as for IVSHMEM), you can configure the +amount of memory used from any given NUMA nodes. For example, to use 1GB from +NUMA node 0, run:: + + $ ovs-vsctl --no-wait set Open_vSwitch . \ + other_config:dpdk-socket-mem="1024,0" + +Similarly, if you wish to better scale the workloads across cores, then +multiple pmd threads can be created and pinned to CPU cores by explicity +specifying ``pmd-cpu-mask``. Cores are numbered from 0, so to spawn two pmd +threads and pin them to cores 1,2, run:: + + $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x6 + +For details on using ivshmem with DPDK, refer to :doc:`dpdk-advanced`. + +Refer to ovs-vswitchd.conf.db(5) for additional information on configuration +options. + +.. note:: + Changing any of these options requires restarting the ovs-vswitchd + application + +Validating +---------- + +Creating bridges and ports +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can now use ovs-vsctl to set up bridges and other Open vSwitch features. +Bridges should be created with a ``datapath_type=netdev``:: + + $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev + +Now you can add DPDK devices. OVS expects DPDK device names to start with +``dpdk`` and end with a portid. ovs-vswitchd should print the number of dpdk +devices found in the log file:: + + $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk + $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk + +After the DPDK ports get added to switch, a polling thread continuously polls +DPDK devices and consumes 100% of the core, as can be checked from 'top' and +'ps' cmds:: + + $ top -H + $ ps -eLo pid,psr,comm | grep pmd + +Creating bonds of DPDK interfaces is slightly different to creating bonds of +system interfaces. For DPDK, the interface type must be explicitly set. For +example:: + + $ ovs-vsctl add-bond br0 dpdkbond dpdk0 dpdk1 \ + -- set Interface dpdk0 type=dpdk \ + -- set Interface dpdk1 type=dpdk + +To stop ovs-vswitchd & delete bridge, run:: + + $ ovs-appctl -t ovs-vswitchd exit + $ ovs-appctl -t ovsdb-server exit + $ ovs-vsctl del-br br0 + +PMD thread statistics +~~~~~~~~~~~~~~~~~~~~~ + +To show current stats:: + + $ ovs-appctl dpif-netdev/pmd-stats-show + +To clear previous stats:: + + $ ovs-appctl dpif-netdev/pmd-stats-clear + +Port/rxq assigment to PMD threads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To show port/rxq assignment:: + + $ ovs-appctl dpif-netdev/pmd-rxq-show + +To change default rxq assignment to pmd threads, rxqs may be manually pinned to +desired cores using:: + + $ ovs-vsctl set Interface \ + other_config:pmd-rxq-affinity= + +where: + +- ```` ::= ``NULL`` | ```` +- ```` ::= ```` | + ```` , ```` +- ```` ::= ```` : ```` + +For example:: + + $ ovs-vsctl set interface dpdk0 options:n_rxq=4 \ + other_config:pmd-rxq-affinity="0:3,1:7,3:8" + +This will ensure: + +- Queue #0 pinned to core 3 +- Queue #1 pinned to core 7 +- Queue #2 not pinned +- Queue #3 pinned to core 8 + +After that PMD threads on cores where RX queues was pinned will become +``isolated``. This means that this thread will poll only pinned RX queues. + +.. warning:: + If there are no ``non-isolated`` PMD threads, ``non-pinned`` RX queues will + not be polled. Also, if provided ``core_id`` is not available (ex. this + ``core_id`` not in ``pmd-cpu-mask``), RX queue will not be polled by any PMD + thread. + +.. _dpdk-guest-setup: + +DPDK in the VM +-------------- + +DPDK 'testpmd' application can be run in the Guest VM for high speed packet +forwarding between vhostuser ports. DPDK and testpmd application has to be +compiled on the guest VM. Below are the steps for setting up the testpmd +application in the VM. More information on the vhostuser ports can be found in +:doc:`dpdk-advanced`. + +.. note:: + Support for DPDK in the guest requires QEMU >= 2.2.0. + +To being, instantiate the guest:: + + $ export VM_NAME=Centos-vm export GUEST_MEM=3072M + $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 + $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch + + $ qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ + -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ + -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ + -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ + -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ + -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ + -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ + -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off \ + +Download the DPDK sourcs to VM and build DPDK:: + + $ cd /root/dpdk/ + $ wget http://fast.dpdk.org/rel/dpdk-16.11.tar.xz + $ tar xf dpdk-16.11.tar.xz + $ export DPDK_DIR=/root/dpdk/dpdk-16.11 + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET + $ cd $DPDK_DIR + $ make install T=$DPDK_TARGET DESTDIR=install + +Build the test-pmd application:: + + $ cd app/test-pmd + $ export RTE_SDK=$DPDK_DIR + $ export RTE_TARGET=$DPDK_TARGET + $ make + +Setup huge pages and DPDK devices using UIO:: + + $ sysctl vm.nr_hugepages=1024 + $ mkdir -p /dev/hugepages + $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already mounted + $ modprobe uio + $ insmod $DPDK_BUILD/kmod/igb_uio.ko + $ $DPDK_DIR/tools/dpdk-devbind.py --status + $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 + +.. note:: + + vhost ports pci ids can be retrieved using:: + + lspci | grep Ethernet + +Testing +------- + +Below are few testcases and the list of steps to be followed. Before beginning, +ensure a userspace bridge has been created and two DPDK ports added:: + + $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev + $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk + $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk + +PHY-PHY +~~~~~~~ + +Add test flows to forward packets betwen DPDK port 0 and port 1:: + + # Clear current flows + $ ovs-ofctl del-flows br0 + + # Add flows between port 1 (dpdk0) to port 2 (dpdk1) + $ ovs-ofctl add-flow br0 in_port=1,action=output:2 + $ ovs-ofctl add-flow br0 in_port=2,action=output:1 + +Transmit traffic into either port. You should see it returned via the other. + +PHY-VM-PHY (vhost loopback) +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add two ``dpdkvhostuser`` ports to bridge ``br0``:: + + $ ovs-vsctl add-port br0 dpdkvhostuser0 \ + -- set Interface dpdkvhostuser0 type=dpdkvhostuser + $ ovs-vsctl add-port br0 dpdkvhostuser1 \ + -- set Interface dpdkvhostuser1 type=dpdkvhostuser + +Add test flows to forward packets betwen DPDK devices and VM ports:: + + # Clear current flows + $ ovs-ofctl del-flows br0 + + # Add flows + $ ovs-ofctl add-flow br0 in_port=1,action=output:3 + $ ovs-ofctl add-flow br0 in_port=3,action=output:1 + $ ovs-ofctl add-flow br0 in_port=4,action=output:2 + $ ovs-ofctl add-flow br0 in_port=2,action=output:4 + + # Dump flows + $ ovs-ofctl dump-flows br0 + +Create a VM using the following configuration: + ++----------------------+--------+-----------------+ +| configuration | values | comments | ++----------------------+--------+-----------------+ +| qemu version | 2.2.0 | n/a | +| qemu thread affinity | core 5 | taskset 0x20 | +| memory | 4GB | n/a | +| cores | 2 | n/a | +| Qcow2 image | CentOS7| n/a | +| mrg_rxbuf | off | n/a | ++----------------------+--------+-----------------+ + +You can do this directly with QEMU via the ``qemu-system-x86_64`` +application:: + + $ export VM_NAME=vhost-vm + $ export GUEST_MEM=3072M + $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 + $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch + + $ taskset 0x20 qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ + -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ + -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ + -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ + -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ + -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ + -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ + -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off + +Alternatively, you can configure the guest using libvirt. Below is an XML +configuration for a 'demovm' guest that can be instantiated using `virsh`:: + + + demovm + 4a9b3f53-fa2a-47f3-a757-dd87720d9d1d + 4194304 + 4194304 + + + + + + 2 + + 4096 + + + + + + hvm + + + + + + + + + + + + + + destroy + restart + destroy + + /usr/bin/qemu-kvm + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Once the guest is configured and booted, configure DPDK packet forwarding +within the guest. To accomplish this, DPDK and testpmd application have to +be first compiled on the VM as described in **Guest Setup**. Once compiled, run +the ``test-pmd`` application:: + + $ cd $DPDK_DIR/app/test-pmd; + $ ./testpmd -c 0x3 -n 4 --socket-mem 1024 -- \ + --burst=64 -i --txqflags=0xf00 --disable-hw-vlan + $ set fwd mac retry + $ start + +When you finish testing, bind the vNICs back to kernel:: + + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 + +.. note:: + Appropriate PCI IDs to be passed in above example. The PCI IDs can be + retrieved like so:: + + $ $DPDK_DIR/tools/dpdk-devbind.py --status + +.. note:: + More information on the dpdkvhostuser ports can be found in + :doc:`dpdk-advanced`. + +PHY-VM-PHY (IVSHMEM loopback) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to the :doc:`dpdk-advanced`. + +Limitations +------------ + +- Currently DPDK ports does not use HW offload functionality. +- Network Interface Firmware requirements: Each release of DPDK is + validated against a specific firmware version for a supported Network + Interface. New firmware versions introduce bug fixes, performance + improvements and new functionality that DPDK leverages. The validated + firmware versions are available as part of the release notes for + DPDK. It is recommended that users update Network Interface firmware + to match what has been validated for the DPDK release. + + The latest list of validated firmware versions can be found in the `DPDK + release notes`_. + +.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_16_11.html + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/intro/install/fedora.rst b/Documentation/intro/install/fedora.rst new file mode 100644 index 000000000..73635386a --- /dev/null +++ b/Documentation/intro/install/fedora.rst @@ -0,0 +1,141 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=========================================== +Fedora, RHEL 7.x Packaging for Open vSwitch +=========================================== + +This document provides instructions for building and installing Open vSwitch +RPM packages on a Fedora Linux host. Instructions for the installation of Open +vSwitch on a Fedora Linux host without using RPM packages can be found in the +:doc:`general`. + +These instructions have been tested with Fedora 23, and are also applicable for +RHEL 7.x and its derivatives, including CentOS 7.x and Scientific Linux 7.x. + +Build Requirements +------------------ + +To build packages for a Fedora Linux host, you will need the packages described +in the :doc:`general`. Specific packages (by package name) include: + +- rpm-build +- autoconf automake libtool +- systemd-units openssl openssl-devel +- python python-twisted-core python-zope-interface python-six +- desktop-file-utils +- groff graphviz +- procps-ng + +And (optionally): + +- libcap-ng libcap-ng-devel +- dpdk-devel + +Bootstraping +------------ + +Refer to :ref:`general-bootstrapping`. + +Configuring +----------- + +Refer to :ref:`general-configuring`. + +Building +-------- + +User Space RPMs +~~~~~~~~~~~~~~~ + +To build Open vSwitch user-space RPMs, execute the following from the directory +in which `./configure` was executed: + +:: + + $ make rpm-fedora + +This will create the RPMs `openvswitch`, `python-openvswitch`, +`openvswitch-test`, `openvswitch-devel`, `openvswitch-ovn-common`, +`openvswitch-ovn-central`, `openvswitch-ovn-host`, `openvswitch-ovn-vtep`, +`openvswitch-ovn-docker`, and `openvswitch-debuginfo`. + +To enable DPDK support in the openvswitch package, the ``--with dpdk`` option +can be added: + +:: + + $ make rpm-fedora RPMBUILD_OPT="--with dpdk" + +The above commands automatically run the Open vSwitch unit tests, which can +take several minutes. To reduce the build time by disabling the execution of +these tests, the ``--without check`` option can be added: + +:: + + $ make rpm-fedora RPMBUILD_OPT="--without check" + +Kernel OVS Tree Datapath RPM +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To build the Open vSwitch kernel module for the currently running kernel +version, run: + +:: + + $ make rpm-fedora-kmod + +To build the Open vSwitch kernel module for another kernel version, the desired +kernel version can be specified via the `kversion` macro. For example: + +:: + + $ make rpm-fedora-kmod \ + RPMBUILD_OPT='-D "kversion 4.3.4-300.fc23.x86_64"' + +Installing +---------- + +RPM packages can be installed by using the command ``rpm -i``. Package +installation requires superuser privileges. + +The `openvswitch-kmod` RPM should be installed first if the Linux OVS tree +datapath module is to be used. The `openvswitch-kmod` RPM should not be +installed if only the in-tree Linux datapath or user-space datapath is needed. +See the `FAQ document `__ for more information about the various Open +vSwitch datapath options. + +In most cases only the `openvswitch` RPM will need to be installed. The +`python-openvswitch`, `openvswitch-test`, `openvswitch-devel`, and +`openvswitch-debuginfo` RPMs are optional unless required for a specific +purpose. + +The `openvswitch-ovn-*` packages are only needed when using OVN. + +Refer to the `RHEL README `__ for additional usage and +configuration information. + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/intro/install/general.rst b/Documentation/intro/install/general.rst new file mode 100644 index 000000000..631dc9be5 --- /dev/null +++ b/Documentation/intro/install/general.rst @@ -0,0 +1,862 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +========================================= +Open vSwitch on Linux, FreeBSD and NetBSD +========================================= + +This document describes how to build and install Open vSwitch on a generic +Linux, FreeBSD, or NetBSD host. For specifics around installation on a specific +platform, refer to one of the other installation guides listed in :doc:`index`. + +.. _general-build-reqs: + +Build Requirements +------------------ + +To compile the userspace programs in the Open vSwitch distribution, you will +need the following software: + +- GNU make + +- A C compiler, such as: + + - GCC 4.x. + + - Clang. Clang 3.4 and later provide useful static semantic analysis and + thread-safety checks. For Ubuntu, there are nightly built packages + available on clang's website. + + - MSVC 2013. Refer to :doc:`windows` for additional Windows build + instructions. + + While OVS may be compatible with other compilers, optimal support for atomic + operations may be missing, making OVS very slow (see ``lib/ovs-atomic.h``). + +- libssl, from OpenSSL, is optional but recommended if you plan to connect the + Open vSwitch to an OpenFlow controller. libssl is required to establish + confidentiality and authenticity in the connections from an Open vSwitch to + an OpenFlow controller. If libssl is installed, then Open vSwitch will + automatically build with support for it. + +- libcap-ng, written by Steve Grubb, is optional but recommended. It is + required to run OVS daemons as a non-root user with dropped root privileges. + If libcap-ng is installed, then Open vSwitch will automatically build with + support for it. + +- Python 2.7. You must also have the Python ``six`` library. + +On Linux, you may choose to compile the kernel module that comes with the Open +vSwitch distribution or to use the kernel module built into the Linux kernel +(version 3.3 or later). See the `FAQ `__ question "What features +are not available in the Open vSwitch kernel datapath that ships as part of the +upstream Linux kernel?" for more information on this trade-off. You may also +use the userspace-only implementation, at some cost in features and +performance. Refer to :doc:`userspace` for details. + +To compile the kernel module on Linux, you must also install the +following: + +- A supported Linux kernel version. + + For optional support of ingress policing, you must enable kernel + configuration options ``NET_CLS_BASIC``, ``NET_SCH_INGRESS``, and + ``NET_ACT_POLICE``, either built-in or as modules. ``NET_CLS_POLICE`` is + obsolete and not needed.) + + On kernels before 3.11, the ``ip_gre`` module, for GRE tunnels over IP + (``NET_IPGRE``), must not be loaded or compiled in. + + To configure HTB or HFSC quality of service with Open vSwitch, you must + enable the respective configuration options. + + To use Open vSwitch support for TAP devices, you must enable ``CONFIG_TUN``. + +- To build a kernel module, you need the same version of GCC that was used to + build that kernel. + +- A kernel build directory corresponding to the Linux kernel image the module + is to run on. Under Debian and Ubuntu, for example, each linux-image package + containing a kernel binary has a corresponding linux-headers package with + the required build infrastructure. + +If you are working from a Git tree or snapshot (instead of from a distribution +tarball), or if you modify the Open vSwitch build system or the database +schema, you will also need the following software: + +- Autoconf version 2.63 or later. + +- Automake version 1.10 or later. + +- libtool version 2.4 or later. (Older versions might work too.) + +To run the unit tests, you also need: + +- Perl. Version 5.10.1 is known to work. Earlier versions should also + work. + +The datapath tests for userspace and Linux datapaths also rely upon: + +- pyftpdlib. Version 1.2.0 is known to work. Earlier versions should + also work. + +- GNU wget. Version 1.16 is known to work. Earlier versions should also + work. + +The ovs-vswitchd.conf.db(5) manpage will include an E-R diagram, in formats +other than plain text, only if you have the following: + +- dot from graphviz (http://www.graphviz.org/). + +- Perl. Version 5.10.1 is known to work. Earlier versions should also + work. + +If you are going to extensively modify Open vSwitch, consider installing the +following to obtain better warnings: + +- "sparse" version 0.4.4 or later + (http://www.kernel.org/pub/software/devel/sparse/dist/). + +- GNU make. + +- clang, version 3.4 or later + +- flake8, version 2.X, along with the hacking flake8 plugin (for Python code). + The automatic flake8 check that runs against Python code has some warnings + enabled that come from the "hacking" flake8 plugin. If it's not installed, + the warnings just won't occur until it's run on a system with "hacking" + installed. Note that there are problems with flake8 3.0 and the "hacking" + plugin. To ensure you get flake8 2.X, you can use:: + + $ pip install 'flake8<3.0' + +You may find the ovs-dev script found in ``utilities/ovs-dev.py`` useful. + +.. _general-install-reqs: + +Installation Requirements +------------------------- + +The machine you build Open vSwitch on may not be the one you run it on. To +simply install and run Open vSwitch you require the following software: + +- libc compatible with the libc used for build. + +- libssl compatible with the libssl used for build, if OpenSSL was used + for the build. + +- On Linux, the same kernel version configured as part of the build. + +- For optional support of ingress policing on Linux, the "tc" program + from iproute2 (part of all major distributions and available at + http://www.linux-foundation.org/en/Net:Iproute2). + +- Python 2.7. You must also have the Python six library. + +On Linux you should ensure that ``/dev/urandom`` exists. To support TAP +devices, you must also ensure that ``/dev/net/tun`` exists. + +.. _general-bootstrapping: + +Bootstrapping +------------- + +This step is not needed if you have downloaded a released tarball. If +you pulled the sources directly from an Open vSwitch Git tree or got a +Git tree snapshot, then run boot.sh in the top source directory to build +the "configure" script:: + + $ ./boot.sh + +.. _general-configuring: + +Configuring +----------- + +Configure the package by running the configure script. You can usually +invoke configure without any arguments. For example:: + + $ ./configure + +By default all files are installed under ``/usr/local``. Open vSwitch also +expects to find its database in ``/usr/local/etc/openvswitch`` by default. If +you want to install all files into, e.g., ``/usr`` and ``/var`` instead of +``/usr/local`` and ``/usr/local/var`` and expect to use ``/etc/openvswitch`` as +the default database directory, add options as shown here:: + + $ ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc + +.. note:: + + Open vSwitch installed with packages like .rpm (e.g. via ``yum install`` or + ``rpm -ivh``) and .deb (e.g. via ``apt-get install`` or ``dpkg -i``) use the + above configure options. + +By default, static libraries are built and linked against. If you want to use +shared libraries instead:: + + $ ./configure --enable-shared + +To use a specific C compiler for compiling Open vSwitch user programs, also +specify it on the configure command line, like so:: + + $ ./configure CC=gcc-4.2 + +To use 'clang' compiler:: + + $ ./configure CC=clang + +To supply special flags to the C compiler, specify them as ``CFLAGS`` on the +configure command line. If you want the default CFLAGS, which include ``-g`` to +build debug symbols and ``-O2`` to enable optimizations, you must include them +yourself. For example, to build with the default CFLAGS plus ``-mssse3``, you +might run configure as follows:: + + $ ./configure CFLAGS="-g -O2 -mssse3" + +For efficient hash computation special flags can be passed to leverage built-in +intrinsics. For example on X86_64 with SSE4.2 instruction set support, CRC32 +intrinsics can be used by passing ``-msse4.2``:: + + $ ./configure CFLAGS="-g -O2 -msse4.2"` + +If you are on a different processor and don't know what flags to choose, it is +recommended to use ``-march=native`` settings:: + + $ ./configure CFLAGS="-g -O2 -march=native" + +With this, GCC will detect the processor and automatically set appropriate +flags for it. This should not be used if you are compiling OVS outside the +target machine. + +.. note:: + CFLAGS are not applied when building the Linux kernel module. Custom CFLAGS + for the kernel module are supplied using the ``EXTRA_CFLAGS`` variable when + running make. For example:: + + $ make EXTRA_CFLAGS="-Wno-error=date-time" + +To build the Linux kernel module, so that you can run the kernel-based switch, +pass the location of the kernel build directory on ``--with-linux``. For +example, to build for a running instance of Linux:: + + $ ./configure --with-linux=/lib/modules/$(uname -r)/build + +.. note:: + If ``--with-linux`` requests building for an unsupported version of Linux, + then ``configure`` will fail with an error message. Refer to the `FAQ + `__ for advice in that case. + +If you wish to build the kernel module for an architecture other than the +architecture of the machine used for the build, you may specify the kernel +architecture string using the KARCH variable when invoking the configure +script. For example, to build for MIPS with Linux:: + + $ ./configure --with-linux=/path/to/linux KARCH=mips + +If you plan to do much Open vSwitch development, you might want to add +``--enable-Werror``, which adds the ``-Werror`` option to the compiler command +line, turning warnings into errors. That makes it impossible to miss warnings +generated by the build. For example:: + + $ ./configure --enable-Werror + +To build with gcov code coverage support, add ``--enable-coverage``:: + + $ ./configure --enable-coverage + +The configure script accepts a number of other options and honors additional +environment variables. For a full list, invoke configure with the ``--help`` +option:: + + $ ./configure --help + +You can also run configure from a separate build directory. This is helpful if +you want to build Open vSwitch in more than one way from a single source +directory, e.g. to try out both GCC and Clang builds, or to build kernel +modules for more than one Linux version. For example:: + + $ mkdir _gcc && (cd _gcc && ./configure CC=gcc) + $ mkdir _clang && (cd _clang && ./configure CC=clang) + +Under certains loads the ovsdb-server and other components perform better when +using the jemalloc memory allocator, instead of the glibc memory allocator. If +you wish to link with jemalloc add it to LIBS:: + + $ ./configure LIBS=-ljemalloc + +.. _general-building: + +Building +-------- + +1. Run GNU make in the build directory, e.g.:: + + $ make + + or if GNU make is installed as "gmake":: + + $ gmake + + If you used a separate build directory, run make or gmake from that + directory, e.g.:: + + $ make -C _gcc + $ make -C _clang + + For improved warnings if you installed ``sparse`` (see "Prerequisites"), add + ``C=1`` to the command line. + + .. note:: + Some versions of Clang and ccache are not completely compatible. If you + see unusual warnings when you use both together, consider disabling + ccache. + +2. Consider running the testsuite. Refer to **Testing** for instructions. + +3. Run ``make install`` to install the executables and manpages into the + running system, by default under ``/usr/local``:: + + $ make install + +5. If you built kernel modules, you may install them, e.g.:: + + $ make modules_install + + It is possible that you already had a Open vSwitch kernel module installed + on your machine that came from upstream Linux (in a different directory). To + make sure that you load the Open vSwitch kernel module you built from this + repository, you should create a ``depmod.d`` file that prefers your newly + installed kernel modules over the kernel modules from upstream Linux. The + following snippet of code achieves the same:: + + $ config_file="/etc/depmod.d/openvswitch.conf" + $ for module in datapath/linux/*.ko; do + modname="$(basename ${module})" + echo "override ${modname%.ko} * extra" >> "$config_file" + echo "override ${modname%.ko} * weak-updates" >> "$config_file" + done + $ depmod -a + + Finally, load the kernel modules that you need. e.g.:: + + $ /sbin/modprobe openvswitch + + To verify that the modules have been loaded, run ``/sbin/lsmod`` and check + that openvswitch is listed:: + + $ /sbin/lsmod | grep openvswitch + + .. note:: + If the ``modprobe`` operation fails, look at the last few kernel log + messages (e.g. with ``dmesg | tail``). Generally, issues like this occur + when Open vSwitch is built for a kernel different from the one into which + you are trying to load it. Run ``modinfo`` on ``openvswitch.ko`` and on a + module built for the running kernel, e.g.:: + + $ /sbin/modinfo openvswitch.ko + $ /sbin/modinfo /lib/modules/$(uname -r)/kernel/net/bridge/bridge.ko + + Compare the "vermagic" lines output by the two commands. If they differ, + then Open vSwitch was built for the wrong kernel. + + If you decide to report a bug or ask a question related to module loading, + include the output from the ``dmesg`` and ``modinfo`` commands mentioned + above. + +.. _general-starting: + +Starting +-------- + +Before starting ovs-vswitchd itself, you need to start its configuration +database, ovsdb-server. Each machine on which Open vSwitch is installed should +run its own copy of ovsdb-server. Before ovsdb-server itself can be started, +configure a database that it can use:: + + $ mkdir -p /usr/local/etc/openvswitch + $ ovsdb-tool create /usr/local/etc/openvswitch/conf.db \ + vswitchd/vswitch.ovsschema + +Configure ovsdb-server to use database created above, to listen on a Unix +domain socket, to connect to any managers specified in the database itself, and +to use the SSL configuration in the database:: + + $ mkdir -p /usr/local/var/run/openvswitch + $ ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ + --remote=db:Open_vSwitch,Open_vSwitch,manager_options \ + --private-key=db:Open_vSwitch,SSL,private_key \ + --certificate=db:Open_vSwitch,SSL,certificate \ + --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \ + --pidfile --detach --log-file + +.. note:: + If you built Open vSwitch without SSL support, then omit ``--private-key``, + ``--certificate``, and ``--bootstrap-ca-cert``.) + +Initialize the database using ovs-vsctl. This is only necessary the first time +after you create the database with ovsdb-tool, though running it at any time is +harmless:: + + $ ovs-vsctl --no-wait init + +Start the main Open vSwitch daemon, telling it to connect to the same Unix +domain socket:: + + $ ovs-vswitchd --pidfile --detach --log-file + +Validating +---------- + +At this point you can use ovs-vsctl to set up bridges and other Open vSwitch +features. For example, to create a bridge named ``br0`` and add ports ``eth0`` +and ``vif1.0`` to it:: + + $ ovs-vsctl add-br br0 + $ ovs-vsctl add-port br0 eth0 + $ ovs-vsctl add-port br0 vif1.0 + +Refer to ovs-vsctl(8) for more details. + +Upgrading +--------- + +When you upgrade Open vSwitch from one version to another you should also +upgrade the database schema: + +1. Stop the Open vSwitch daemons, e.g.:: + + $ kill `cd /usr/local/var/run/openvswitch && cat ovsdb-server.pid ovs-vswitchd.pid` + +2. Install the new Open vSwitch release by using the same configure options as + was used for installing the previous version. If you do not use the same + configure options, you can end up with two different versions of Open + vSwitch executables installed in different locations. + +3. Upgrade the database, in one of the following two ways: + + - If there is no important data in your database, then you may delete the + database file and recreate it with ovsdb-tool, following the instructions + under "Building and Installing Open vSwitch for Linux, FreeBSD or NetBSD". + + - If you want to preserve the contents of your database, back it up first, + then use ``ovsdb-tool convert`` to upgrade it, e.g.:: + + $ ovsdb-tool convert /usr/local/etc/openvswitch/conf.db \ + vswitchd/vswitch.ovsschema + +4. Start the Open vSwitch daemons as described under **Starting** above. + +Hot Upgrading +------------- + +Upgrading Open vSwitch from one version to the next version with minimum +disruption of traffic going through the system that is using that Open vSwitch +needs some considerations: + +1. If the upgrade only involves upgrading the userspace utilities and daemons + of Open vSwitch, make sure that the new userspace version is compatible with + the previously loaded kernel module. + +2. An upgrade of userspace daemons means that they have to be restarted. + Restarting the daemons means that the OpenFlow flows in the ovs-vswitchd + daemon will be lost. One way to restore the flows is to let the controller + re-populate it. Another way is to save the previous flows using a utility + like ovs-ofctl and then re-add them after the restart. Restoring the old + flows is accurate only if the new Open vSwitch interfaces retain the old + 'ofport' values. + +3. When the new userspace daemons get restarted, they automatically flush the + old flows setup in the kernel. This can be expensive if there are hundreds + of new flows that are entering the kernel but userspace daemons are busy + setting up new userspace flows from either the controller or an utility like + ovs-ofctl. Open vSwitch database provides an option to solve this problem + through the ``other_config:flow-restore-wait`` column of the + ``Open_vSwitch`` table. Refer to the ovs-vswitchd.conf.db(5) manpage for + details. + +4. If the upgrade also involves upgrading the kernel module, the old kernel + module needs to be unloaded and the new kernel module should be loaded. This + means that the kernel network devices belonging to Open vSwitch is recreated + and the kernel flows are lost. The downtime of the traffic can be reduced if + the userspace daemons are restarted immediately and the userspace flows are + restored as soon as possible. + +The ovs-ctl utility's ``restart`` function only restarts the userspace daemons, +makes sure that the 'ofport' values remain consistent across restarts, restores +userspace flows using the ovs-ofctl utility and also uses the +``other_config:flow-restore-wait`` column to keep the traffic downtime to the +minimum. The ovs-ctl utility's ``force-reload-kmod`` function does all of the +above, but also replaces the old kernel module with the new one. Open vSwitch +startup scripts for Debian, XenServer and RHEL use ovs-ctl's functions and it +is recommended that these functions be used for other software platforms too. + +.. _general-testing: + +Testing +------- + +This section describe Open vSwitch's built-in support for various test +suites. You must bootstrap, configure and build Open vSwitch (steps are +in "Building and Installing Open vSwitch for Linux, FreeBSD or NetBSD" +above) before you run the tests described here. You do not need to +install Open vSwitch or to build or load the kernel module to run these +test suites. You do not need supervisor privilege to run these test +suites. + +Unit Tests +~~~~~~~~~~ + +Open vSwitch includes a suite of self-tests. Before you submit patches +upstream, we advise that you run the tests and ensure that they pass. If you +add new features to Open vSwitch, then adding tests for those features will +ensure your features don't break as developers modify other areas of Open +vSwitch. + +To run all the unit tests in Open vSwitch, one at a time, run:: + + $ make check + +This takes under 5 minutes on a modern desktop system. + +To run all the unit tests in Open vSwitch in parallel, run:: + + $ make check TESTSUITEFLAGS=-j8 + +You can run up to eight threads. This takes under a minute on a modern 4-core +desktop system. + +To see a list of all the available tests, run: + + $ make check TESTSUITEFLAGS=--list + +To run only a subset of tests, e.g. test 123 and tests 477 through 484, run:: + + $ make check TESTSUITEFLAGS='123 477-484' + +Tests do not have inter-dependencies, so you may run any subset. + +To run tests matching a keyword, e.g. ``ovsdb``, run:: + + $ make check TESTSUITEFLAGS='-k ovsdb' + +To see a complete list of test options, run:: + + $ make check TESTSUITEFLAGS=--help + +The results of a testing run are reported in ``tests/testsuite.log``. Report +report test failures as bugs and include the ``testsuite.log`` in your report. + +.. note:: + Sometimes a few tests may fail on some runs but not others. This is usually a + bug in the testsuite, not a bug in Open vSwitch itself. If you find that a + test fails intermittently, please report it, since the developers may not + have noticed. You can make the testsuite automatically rerun tests that fail, + by adding ``RECHECK=yes`` to the ``make`` command line, e.g.:: + + $ make check TESTSUITEFLAGS=-j8 RECHECK=yes + +Coverage +++++++++ + +If the build was configured with ``--enable-coverage`` and the ``lcov`` utility +is installed, you can run the testsuite and generate a code coverage report by +using the ``check-lcoc`` target:: + + $ make check-lcov + +All the same options are avaiable via TESTSUITEFLAGS. For example:: + + $ make check-lcov TESTSUITEFLAGS=-j8 -k ovn + +Valgrind +++++++++ + +If you have ``valgrind`` installed, you can run the testsuite under +valgrind by using the ``check-valgrind`` target:: + + $ make check-valgrind + +When you do this, the "valgrind" results for test ```` are reported in files +named ``tests/testsuite.dir//valgrind.*``. + +All the same options are available via TESTSUITEFLAGS. + +.. hint:: + You may find that the valgrind results are easier to interpret if you put + ``-q`` in ``~/.valgrindrc``, since that reduces the amount of output. + +.. _general-oftest: + +OFTest +~~~~~~ + +OFTest is an OpenFlow protocol testing suite. Open vSwitch includes a Makefile +target to run OFTest with Open vSwitch in "dummy mode". In this mode of +testing, no packets travel across physical or virtual networks. Instead, Unix +domain sockets stand in as simulated networks. This simulation is imperfect, +but it is much easier to set up, does not require extra physical or virtual +hardware, and does not require supervisor privileges. + +To run OFTest with Open vSwitch, first read and follow the instructions under +**Testing** above. Second, obtain a copy of OFTest and install its +prerequisites. You need a copy of OFTest that includes commit 406614846c5 (make +ovs-dummy platform work again). This commit was merged into the OFTest +repository on Feb 1, 2013, so any copy of OFTest more recent than that should +work. Testing OVS in dummy mode does not require root privilege, so you may +ignore that requirement. + +Optionally, add the top-level OFTest directory (containing the ``oft`` program) +to your ``$PATH``. This slightly simplifies running OFTest later. + +To run OFTest in dummy mode, run the following command from your Open vSwitch +build directory:: + + $ make check-oftest OFT= + +where ```` is the absolute path to the ``oft`` program in OFTest. +If you added "oft" to your $PATH, you may omit the OFT variable +assignment + +By default, ``check-oftest`` passes ``oft`` just enough options to enable dummy +mode. You can use ``OFTFLAGS`` to pass additional options. For example, to run +just the ``basic.Echo`` test instead of all tests (the default) and enable +verbose logging, run:: + + $ make check-oftest OFT= OFTFLAGS='--verbose -T basic.Echo' + +If you use OFTest that does not include commit 4d1f3eb2c792 (oft: change +default port to 6653), merged into the OFTest repository in October 2013, then +you need to add an option to use the IETF-assigned controller port:: + + $ make check-oftest OFT= OFTFLAGS='--port=6653' + +Interpret OFTest results cautiously. Open vSwitch can fail a given test in +OFTest for many reasons, including bugs in Open vSwitch, bugs in OFTest, bugs +in the "dummy mode" integration, and differing interpretations of the OpenFlow +standard and other standards. + +.. note:: + Open vSwitch has not been validated against OFTest. Report test failures that + you believe to represent bugs in Open vSwitch. Include the precise versions + of Open vSwitch and OFTest in your bug report, plus any other information + needed to reproduce the problem. + +Ryu +~~~ + +Ryu is an OpenFlow controller written in Python that includes an extensive +OpenFlow testsuite. Open vSwitch includes a Makefile target to run Ryu in +"dummy mode". See **OFTest** above for an explanation of dummy mode. + +To run Ryu tests with Open vSwitch, first read and follow the instructions +under **Testing** above. Second, obtain a copy of Ryu, install its +prerequisites, and build it. You do not need to install Ryu (some of the tests +do not get installed, so it does not help). + +To run Ryu tests, run the following command from your Open vSwitch build +directory:: + + $ make check-ryu RYUDIR=`` + +where ```` is the absolute path to the root of the Ryu source +distribution. The default ```` is ``$srcdir/../ryu`` +where ``$srcdir`` is your Open vSwitch source directory. If this is correct, +omit ``RYUDIR`` + +.. note:: + Open vSwitch has not been validated against Ryu. Report test failures that + you believe to represent bugs in Open vSwitch. Include the precise versions + of Open vSwitch and Ryu in your bug report, plus any other information + needed to reproduce the problem. + +Datapath testing +~~~~~~~~~~~~~~~~ + +Open vSwitch includes a suite of tests specifically for datapath functionality, +which can be run against the userspace or kernel datapaths. If you are +developing datapath features, it is recommended that you use these tests and +build upon them to verify your implementation. + +The datapath tests make some assumptions about the environment. They must be +run under root privileges on a Linux system with support for network +namespaces. For ease of use, the OVS source tree includes a vagrant box to +invoke these tests. Running the tests inside Vagrant provides kernel isolation, +protecting your development host from kernel panics or configuration conflicts +in the testsuite. If you wish to run the tests without using the vagrant box, +there are further instructions below. + +Vagrant ++++++++ + +.. important:: + + Requires Vagrant (version 1.7.0 or later) and a compatible hypervisor + +.. note:: + You must **Bootstrap** and **Configure** the sources before you run the steps + described here. + +A Vagrantfile is provided allowing to compile and provision the source tree as +found locally in a virtual machine using the following command:: + + $ vagrant up + +This will bring up a Fedora 23 VM by default. If you wish to use a different +box or a vagrant backend not supported by the default box, the ``Vagrantfile`` +can be modified to use a different box as base. + +The VM can be reprovisioned at any time:: + + $ vagrant provision + +OVS out-of-tree compilation environment can be set up with:: + + $ ./boot.sh + $ vagrant provision --provision-with configure_ovs,build_ovs + +This will set up an out-of-tree build environment inside the VM in +``/root/build``. The source code can be found in ``/vagrant``. + +To recompile and reinstall OVS in the VM using RPM:: + + $ ./boot.sh + $ vagrant provision --provision-with configure_ovs,install_rpm + +Two provisioners are included to run system tests with the OVS kernel module or +with a userspace datapath. This tests are different from the self-tests +mentioned above. To run them:: + + $ ./boot.sh + $ vagrant provision --provision-with \ + configure_ovs,test_ovs_kmod,test_ovs_system_userspace + +The results of the testsuite reside in the VM root user's home directory:: + + $ vagrant ssh + $ sudo -s + $ cd /root/build + $ ls tests/system* + +Native +++++++ + +The datapath testsuite as invoked by Vagrant above may also be run manually on +a Linux system with root privileges. These tests may take several minutes to +complete, and cannot be run in parallel. + +Userspace datapath +''''''''''''''''''' + +To invoke the datapath testsuite with the userspace datapath, run:: + + $ make check-system-userspace + +The results of the testsuite are in ``tests/system-userspace-traffic.dir``. + +Kernel datapath +''''''''''''''' + +Make targets are also provided for testing the Linux kernel module. Note that +these tests operate by inserting modules into the running Linux kernel, so if +the tests are able to trigger a bug in the OVS kernel module or in the upstream +kernel then the kernel may panic. + +To run the testsuite against the kernel module which is currently installed on +your system, run:: + + $ make check-kernel + +To install the kernel module from the current build directory and run the +testsuite against that kernel module:: + + $ make check-kmod + +The results of the testsuite are in ``tests/system-kmod-traffic.dir``. + +Continuous Integration with Travis-CI +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A .travis.yml file is provided to automatically build Open vSwitch with various +build configurations and run the testsuite using travis-ci. Builds will be +performed with gcc, sparse and clang with the -Werror compiler flag included, +therefore the build will fail if a new warning has been introduced. + +The CI build is triggered via git push (regardless of the specific branch) or +pull request against any Open vSwitch GitHub repository that is linked to +travis-ci. + +Instructions to setup travis-ci for your GitHub repository: + +1. Go to http://travis-ci.org/ and sign in using your GitHub ID. +2. Go to the "Repositories" tab and enable the ovs repository. You may disable + builds for pushes or pull requests. +3. In order to avoid forks sending build failures to the upstream mailing list, + the notification email recipient is encrypted. If you want to receive email + notification for build failures, replace the the encrypted string: + + 1. Install the travis-ci CLI (Requires ruby >=2.0): gem install travis + 2. In your Open vSwitch repository: travis encrypt mylist@mydomain.org + 3. Add/replace the notifications section in .travis.yml and fill in the + secure string as returned by travis encrypt:: + + notifications: + email: + recipients: + - secure: "....." + + .. note:: + You may remove/omit the notifications section to fall back to default + notification behaviour which is to send an email directly to the author and + committer of the failing commit. Note that the email is only sent if the + author/committer have commit rights for the particular GitHub repository. + +4. Pushing a commit to the repository which breaks the build or the + testsuite will now trigger a email sent to mylist@mydomain.org + +Static Code Analysis +~~~~~~~~~~~~~~~~~~~~ + +Static Analysis is a method of debugging Software by examining code rather than +actually executing it. This can be done through 'scan-build' commandline +utility which internally uses clang (or) gcc to compile the code and also +invokes a static analyzer to do the code analysis. At the end of the build, the +reports are aggregated in to a common folder and can later be analyzed using +'scan-view'. + +Open vSwitch includes a Makefile target to trigger static code analysis:: + + $ ./boot.sh + $ ./configure CC=clang # clang + # or + $ ./configure CC=gcc CFLAGS="-std=gnu99" # gcc + $ make clang-analyze + +You should invoke scan-view to view analysis results. The last line of output +from ``clang-analyze`` will list the command (containing results directory) +that you should invoke to view the results on a browser. + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/intro/install/index.rst b/Documentation/intro/install/index.rst index 35dc67cf5..2366388a2 100644 --- a/Documentation/intro/install/index.rst +++ b/Documentation/intro/install/index.rst @@ -30,5 +30,38 @@ Installing Open vSwitch A collection of guides detailing how to install Open vSwitch in a variety of different environments and using different configurations. +Installation from Source +------------------------ + +.. TODO(stephenfin): The DPDK-ADVANCED doc is mostly usage material. The + install related instructions should be moved to the main doc, while the + rest should be moved to howto and topic docs + +.. TODO(stephenfin): Based on the title alone, the NetBSD doc should probably + be merged into the general install doc + +.. toctree:: + :maxdepth: 2 + + general + netbsd + windows + xenserver + userspace + dpdk + dpdk-advanced + bash-completion + +Installation from Packages +-------------------------- + +Open vSwitch is packaged on a variety of distributions. The tooling required to +build these packages is included in the Open vSwitch tree. The instructions are +provided below. + .. toctree:: :maxdepth: 2 + + debian + fedora + rhel diff --git a/Documentation/intro/install/netbsd.rst b/Documentation/intro/install/netbsd.rst new file mode 100644 index 000000000..2b52eaf08 --- /dev/null +++ b/Documentation/intro/install/netbsd.rst @@ -0,0 +1,61 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +====================== +Open vSwitch on NetBSD +====================== + +On NetBSD, you might want to install requirements from pkgsrc. In that case, +you need at least the following packages. + +- automake +- libtool-base +- gmake +- python27 +- py27-six +- py27-xml +- pkg_alternatives + +Some components have additional requirements. Refer to :doc:`general` for more +information. + +Assuming you are running NetBSD/amd64 6.1.2, you can download and install +pre-built binary packages as the following:: + + $ PKG_PATH=http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/6.1.2/All/ + $ export PKG_PATH + $ pkg_add automake libtool-base gmake python27 py27-six py27-xml \ + pkg_alternatives + +.. note:: + You might get some warnings about minor version mismatch. These can be safely + ignored. + +NetBSD's ``/usr/bin/make`` is not GNU make. GNU make is installed as +``/usr/pkg/bin/gmake`` by the above mentioned ``gmake`` package. + +As all executables installed with pkgsrc are placed in ``/usr/pkg/bin/`` +directory, it might be a good idea to add it to your PATH. + +Open vSwitch on NetBSD is currently "userspace switch" implementation in the +sense described in :doc:`userspace` and the `porting guide`. diff --git a/Documentation/intro/install/rhel.rst b/Documentation/intro/install/rhel.rst new file mode 100644 index 000000000..498fd8f25 --- /dev/null +++ b/Documentation/intro/install/rhel.rst @@ -0,0 +1,216 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +======================================== +RHEL 5.6, 6.x Packaging for Open vSwitch +======================================== + +This document describes how to build and install Open vSwitch on a Red Hat +Enterprise Linux (RHEL) host. If you want to install Open vSwitch on a generic +Linux host, refer to :doc:`general` instead. + +We have tested these instructions with RHEL 5.6 and RHEL 6.0. + +For RHEL 7.x (or derivatives, such as CentOS 7.x), you should follow the +instructions in the :doc:`fedora`. The Fedora spec files are used for RHEL +7.x. + +.. _rhel-prerequisites: + +Prerequisites +------------- + +You may build from an Open vSwitch distribution tarball or from an Open vSwitch +Git tree. + +The default RPM build directory, ``_topdir``, has five directories in the +top-level. + +BUILD/ + where the software is unpacked and built +RPMS/ + where the newly created binary package files are written +SOURCES/ + contains the original sources, patches, and icon files +SPECS/ + contains the spec files for each package to be built +SRPMS/ + where the newly created source package files are written + +Before you begin, note the RPM sources directory on your version of RHEL. The +command ``rpmbuild --showrc`` will show the configuration for each of those +directories. Alternatively, the command ``rpm --eval '%{_topdir}'`` shows the +current configuration for the top level directory and the command ``rpm --eval +'%{_sourcedir}'`` does the same for the sources directory. On RHEL 5, the +default RPM ``_topdir`` is ``/usr/src/redhat`` and the default RPM sources +directory is ``/usr/src/redhat/SOURCES``. On RHEL 6, the default ``_topdir`` is +``$HOME/rpmbuild`` and the default RPM sources directory is +``$HOME/rpmbuild/SOURCES``. + +Build Requirements +------------------ + +To compile the RPMs, you will need to install the packages described in the +:doc:`general` along with some additional packages. These can be installed with +the below command:: + + $ yum install gcc make python-devel openssl-devel kernel-devel graphviz \ + kernel-debug-devel autoconf automake rpm-build redhat-rpm-config \ + libtool + +.. _rhel-bootstrapping: + +Bootstrapping and Configuring +----------------------------- + +If you are building from a distribution tarball, skip to :ref:`rhel-building`. +If not, you must be building from an Open vSwitch Git tree. Determine what +version of Autoconf is installed (e.g. run ``autoconf --version``). If it is +not at least version 2.63, then you must upgrade or use another machine to +build the packages. + +Assuming all requirements have been met, build the tarball by running:: + + $ ./boot.sh + $ ./configure + $ make dist + +You must run this on a machine that has the tools listed in +:ref:`general-build-reqs` as prerequisites for building from a Git tree. +Afterward, proceed with the rest of the instructions using the distribution +tarball. + +Now you have a distribution tarball, named something like +``openvswitch-x.y.z.tar.gz``. Copy this file into the RPM sources directory, +e.g.:: + + $ cp openvswitch-x.y.z.tar.gz $HOME/rpmbuild/SOURCES + +Broken ``build`` symlink +~~~~~~~~~~~~~~~~~~~~~~~~ + +Some versions of the RHEL 6 kernel-devel package contain a broken ``build`` +symlink. If you are using such a version, you must fix the problem before +continuing. + +To find out whether you are affected, run:: + + $ cd /lib/modules/ + $ ls -l build/ + +where ```` is the version number of the RHEL 6 kernel. + +.. note:: + The trailing slash in the final command is important. Be sure to include + it. + +If the ``ls`` command produces a directory listing, your kernel-devel package +is OK. If it produces a ``No such file or directory`` error, your kernel-devel +package is buggy. + +If your kernel-devel package is buggy, then you can fix it with:: + + $ cd /lib/modules/ + $ rm build + $ ln -s /usr/src/kernels/ build + +where ```` is the name of an existing directory under +``/usr/src/kernels``, whose name should be similar to ```` but may +contain some extra parts. Once you have done this, verify the fix with the +same procedure you used above to check for the problem. + +.. _rhel-building: + +Building +-------- + +You should have a distribution tarball named something like +openvswitch-x.y.z.tar.gz. Copy this file into the RPM sources directory:: + + $ cp openvswitch-x.y.z.tar.gz $HOME/rpmbuild/SOURCES + +Make another copy of the distribution tarball in a temporary directory. Then +unpack the tarball and ``cd`` into its root:: + + $ tar xzf openvswitch-x.y.z.tar.gz + $ cd openvswitch-x.y.z + +Userspace +~~~~~~~~~ + +To build Open vSwitch userspace, run:: + + $ rpmbuild -bb rhel/openvswitch.spec + +This produces two RPMs: "openvswitch" and "openvswitch-debuginfo". + +The above command automatically runs the Open vSwitch unit tests. To disable +the unit tests, run:: + + $ rpmbuild -bb --without check rhel/openvswitch.spec + +.. note:: + If the build fails with ``configure: error: source dir + /lib/modules/2.6.32-279.el6.x86_64/build doesn't exist`` or similar, then + the kernel-devel package is missing or buggy. + +Kernel Module +~~~~~~~~~~~~~ + +On RHEL 6, to build the Open vSwitch kernel module, copy +rhel/openvswitch-kmod.files into the RPM sources directory and run:: + + $ rpmbuild -bb rhel/openvswitch-kmod-rhel6.spec + +You might have to specify a kernel version and/or variants, e.g.: + + $ rpmbuild -bb \ + -D "kversion 2.6.32-131.6.1.el6.x86_64" \ + -D "kflavors default debug kdump" \ + rhel/openvswitch-kmod-rhel6.spec + +This produces an "kmod-openvswitch" RPM for each kernel variant, in this +example: "kmod-openvswitch", "kmod-openvswitch-debug", and +"kmod-openvswitch-kdump". + +.. _rhel-script-integrations: + +Red Hat Network Scripts Integration +----------------------------------- + +A RHEL host has default firewall rules that prevent any Open vSwitch tunnel +traffic from passing through. If a user configures Open vSwitch tunnels like +Geneve, GRE, VXLAN, LISP etc., they will either have to manually add iptables +firewall rules to allow the tunnel traffic or add it through a startup script +Refer to the "enable-protocol" command in the ovs-ctl(8) manpage for more +information. + +In addition, simple integration with Red Hat network scripts has been +implemented. Refer to `README.RHEL.rst `__ in the source +tree or /usr/share/doc/openvswitch/README.RHEL.rst in the installed openvswitch +package for details. + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/intro/install/userspace.rst b/Documentation/intro/install/userspace.rst new file mode 100644 index 000000000..036852759 --- /dev/null +++ b/Documentation/intro/install/userspace.rst @@ -0,0 +1,106 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=================================== +Open vSwitch without Kernel Support +=================================== + +Open vSwitch can operate, at a cost in performance, entirely in userspace, +without assistance from a kernel module. This file explains how to install +Open vSwitch in such a mode. + +This version of Open vSwitch should be built manually with ``configure`` and +``make``. Debian packaging for Open vSwitch is also included, but it has not +been recently tested, and so Debian packages are not a recommended way to use +this version of Open vSwitch. + +.. warning:: + The userspace-only mode of Open vSwitch is considered experimental. It has + not been thoroughly tested. + +Building and Installing +----------------------- + +The requirements and procedure for building, installing, and configuring Open +vSwitch are the same as those given in :doc:`general`. You may omit +configuring, building, and installing the kernel module, and the related +requirements. + +On Linux, the userspace switch additionally requires the kernel TUN/TAP driver +to be available, either built into the kernel or loaded as a module. If you +are not sure, check for a directory named ``/sys/class/misc/tun``. If it does +not exist, then attempt to load the module with ``modprobe tun``. + +The tun device must also exist as ``/dev/net/tun``. If it does not exist, then +create ``/dev/net`` (if necessary) with ``mkdir /dev/net``, then create +``/dev/net/tun`` with ``mknod /dev/net/tun c 10 200``. + +On FreeBSD and NetBSD, the userspace switch additionally requires the kernel +tap(4) driver to be available, either built into the kernel or loaded as a +module. + +Using the Userspace Datapath with ovs-vswitchd +---------------------------------------------- + +To use ovs-vswitchd in userspace mode, create a bridge with +``datapath_type=netdev`` in the configuration database. For example:: + + $ ovs-vsctl add-br br0 + $ ovs-vsctl set bridge br0 datapath_type=netdev + $ ovs-vsctl add-port br0 eth0 + $ ovs-vsctl add-port br0 eth1 + $ ovs-vsctl add-port br0 eth2 + +ovs-vswitchd will create a TAP device as the bridge's local interface, named +the same as the bridge, as well as for each configured internal interface. + +Currently, on FreeBSD, the functionality required for in-band control support +is not implemented. To avoid related errors, you can disable the in-band +support with the following command:: + + $ ovs-vsctl set bridge br0 other_config:disable-in-band=true + +Firewall Rules +-------------- + +On Linux, when a physical interface is in use by the userspace datapath, +packets received on the interface still also pass into the kernel TCP/IP stack. +This can cause surprising and incorrect behavior. You can use "iptables" to +avoid this behavior, by using it to drop received packets. For example, to +drop packets received on eth0:: + + $ iptables -A INPUT -i eth0 -j DROP + $ iptables -A FORWARD -i eth0 -j DROP + +Other Settings +-------------- + +On NetBSD, depending on your network topology and applications, the following +configuration might help. See sysctl(7).:: + + $ sysctl -w net.inet.ip.checkinterface=1 + +Reporting Bugs +-------------- + +Report problems to bugs@openvswitch.org. diff --git a/Documentation/intro/install/windows.rst b/Documentation/intro/install/windows.rst new file mode 100644 index 000000000..eb55720ed --- /dev/null +++ b/Documentation/intro/install/windows.rst @@ -0,0 +1,620 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +======================= +Open vSwitch on Windows +======================= + +.. _windows-build-reqs: + +Build Requirements +------------------ + +Open vSwitch on Linux uses autoconf and automake for generating Makefiles. It +will be useful to maintain the same build system while compiling on Windows +too. One approach is to compile Open vSwitch in a MinGW environment that +contains autoconf and automake utilities and then use Visual C++ as a compiler +and linker. + +The following explains the steps in some detail. + +- Mingw + + Install Mingw on a Windows machine by following the instructions on + `mingw.org `__. + + This should install mingw at ``C:\Mingw`` and msys at ``C:\Mingw\msys``. Add + ``C:\MinGW\bin`` and ``C:\Mingw\msys\1.0\bin`` to PATH environment variable + of Windows. + + You can either use the MinGW installer or the command line utility + ``mingw-get`` to install both the base packages and additional packages like + automake and autoconf(version 2.68). + + Also make sure that ``/mingw`` mount point exists. If its not, please + add/create the following entry in ``/etc/fstab``:: + + 'C:/MinGW /mingw'. + +- Python + + Install the latest Python 2.x from python.org and verify that its path is + part of Windows' PATH environment variable. + +- Visual Studio + + You will need at least Visual Studio 2013 (update 4) to compile userspace + binaries. In addition to that, if you want to compile the kernel module you + will also need to install Windows Driver Kit (WDK) 8.1 Update. + + It is important to get the Visual Studio related environment variables and to + have the $PATH inside the bash to point to the proper compiler and linker. + One easy way to achieve this for VS2013 is to get into the "VS2013 x86 Native + Tools Command Prompt" (in a default installation of Visual Studio 2013 this + can be found under the following location: ``C:\Program Files (x86)\Microsoft + Visual Studio 12.0\Common7\Tools\Shortcuts``) and through it enter into the + bash shell available from msys by typing ``bash --login``. + + There is support for generating 64 bit binaries too. To compile under x64, + open the "VS2013 x64 Native Tools Command Prompt" (if your current running OS + is 64 bit) or "VS2013 x64 Cross Tools Command Prompt" (if your current + running OS is not 64 bit) instead of opening its x86 variant. This will + point the compiler and the linker to their 64 bit equivalent. + + If after the above step, a ``which link`` inside MSYS's bash says, + ``/bin/link.exe``, rename ``/bin/link.exe`` to something else so that the + Visual studio's linker is used. You should also see a 'which sort' report + ``/bin/sort.exe``. + +- pthreads-win32 + + For pthread support, install the library, dll and includes of pthreads-win32 + project from `sourceware + `__ to a + directory (e.g.: ``C:/pthread``). You should add the pthread-win32's dll path + (e.g.: ``C:\pthread\dll\x86``) to the Windows' PATH environment variable. + +- OpenSSQL + + To get SSL support for Open vSwitch on Windows, you will need to install + `OpenSSL for Windows `__ + + Note down the directory where OpenSSL is installed (e.g.: + ``C:/OpenSSL-Win32``) for later use. + +Install Requirements +-------------------- + +* Share network adaptors + + We require that you don't disable the "Allow management operating system to + share this network adapter" under 'Virtual Switch Properties' > 'Connection + type: External network', in the HyperV virtual network switch configuration. + +* Checksum Offloads + + While there is some support for checksum/segmentation offloads in software, + this is still a work in progress. Till the support is complete we recommend + disabling TX/RX offloads for both the VM's as well as the HyperV. + +Bootstrapping +------------- + +This step is not needed if you have downloaded a released tarball. If +you pulled the sources directly from an Open vSwitch Git tree or got a +Git tree snapshot, then run boot.sh in the top source directory to build +the "configure" script:: + + > ./boot.sh + +.. _windows-configuring: + +Configuring +----------- + +Configure the package by running the configure script. You should provide some +configure options to choose the right compiler, linker, libraries, Open vSwitch +component installation directories, etc. For example:: + + > ./configure CC=./build-aux/cccl LD="$(which link)" \ + LIBS="-lws2_32 -liphlpapi" --prefix="C:/openvswitch/usr" \ + --localstatedir="C:/openvswitch/var" \ + --sysconfdir="C:/openvswitch/etc" \ + --with-pthread="C:/pthread" + +.. note:: + By default, the above enables compiler optimization for fast code. For + default compiler optimization, pass the ``--with-debug`` configure option. + +To configure with SSL support, add the requisite additional options:: + + > ./configure CC=./build-aux/cccl LD="`which link`" \ + LIBS="-lws2_32 -liphlpapi" --prefix="C:/openvswitch/usr" \ + --localstatedir="C:/openvswitch/var" + --sysconfdir="C:/openvswitch/etc" \ + --with-pthread="C:/pthread" \ + --enable-ssl --with-openssl="C:/OpenSSL-Win32" + +Finally, to the kernel module also:: + + > ./configure CC=./build-aux/cccl LD="`which link`" \ + LIBS="-lws2_32 -liphlpapi" --prefix="C:/openvswitch/usr" \ + --localstatedir="C:/openvswitch/var" \ + --sysconfdir="C:/openvswitch/etc" \ + --with-pthread="C:/pthread" \ + --enable-ssl --with-openssl="C:/OpenSSL-Win32" \ + --with-vstudiotarget="" + +Possible values for ```` are: ``Debug`` and ``Release`` + +.. note:: + You can directly use the Visual Studio 2013 IDE to compile the kernel + datapath. Open the ovsext.sln file in the IDE and build the solution. + +Refer to :doc:`general` for information on additional configuration options. + +.. _windows-building: + +Building +-------- + +Once correctly configured, building Open vSwitch on Windows is similar to +building on Linux, FreeBSD, or NetBSD. + +#. Run make for the ported executables in the top source directory, e.g.:: + + > make + + For faster compilation, you can pass the ``-j`` argument to make. For + example, to run 4 jobs simultaneously, run ``make -j4``. + + .. note:: + + MSYS 1.0.18 has a bug that causes parallel make to hang. You can overcome + this by downgrading to MSYS 1.0.17. A simple way to downgrade is to exit + all MinGW sessions and then run the below command from MSVC developers + command prompt.:: + + > mingw-get upgrade msys-core-bin=1.0.17-1 + +#. To run all the unit tests in Open vSwitch, one at a time:: + + > make check + + To run all the unit tests in Open vSwitch, up to 8 in parallel:: + + > make check TESTSUITEFLAGS="-j8" + +#. To install all the compiled executables on the local machine, run:: + + > make install + + .. note:: + + This will install the Open vSwitch executables in ``C:/openvswitch``. You + can add ``C:\openvswitch\usr\bin`` and ``C:\openvswitch\usr\sbin`` to + Windows' PATH environment variable for easy access. + +The Kernel Module +~~~~~~~~~~~~~~~~~ + +If you are building the kernel module, you will need to copy the below files to +the target Hyper-V machine. + +- ``./datapath-windows/x64/Win8.1Debug/package/ovsext.inf`` +- ``./datapath-windows/x64/Win8.1Debug/package/OVSExt.sys`` +- ``./datapath-windows/x64/Win8.1Debug/package/ovsext.cat`` +- ``./datapath-windows/misc/install.cmd`` +- ``./datapath-windows/misc/uninstall.cmd`` + +.. note:: + The above path assumes that the kernel module has been built using Windows + DDK 8.1 in Debug mode. Change the path appropriately, if a different WDK + has been used. + +Now run ``./uninstall.cmd`` to remove the old extension. Once complete, run +``./install.cmd`` to insert the new one. For this to work you will have to +turn on ``TESTSIGNING`` boot option or 'Disable Driver Signature +Enforcement' during boot. The following commands can be used:: + + > bcdedit /set LOADOPTIONS DISABLE_INTEGRITY_CHECKS + > bcdedit /set TESTSIGNING ON + > bcdedit /set nointegritychecks ON + +.. note:: + You may have to restart the machine for the settings to take effect. + +In the Virtual Switch Manager configuration you can enable the Open vSwitch +Extension on an existing switch or create a new switch. If you are using an +existing switch, make sure to enable the "Allow Management OS" option for VXLAN +to work (covered later). + +The command to create a new switch named 'OVS-Extended-Switch' using a physical +NIC named 'Ethernet 1' is:: + + PS > New-VMSwitch "OVS-Extended-Switch" -AllowManagementOS $true \ + -NetAdapterName "Ethernet 1" + +.. note:: + You can obtain the list of physical NICs on the host using 'Get-NetAdapter' + command. + +In the properties of any switch, you should should now see "Open vSwitch +Extension" under 'Extensions'. Click the check box to enable the extension. +An alternative way to do the same is to run the following command:: + + PS > Enable-VMSwitchExtension "Open vSwitch Extension" OVS-Extended-Switch + +.. note:: + If you enabled the extension using the command line, a delay of a few seconds + has been observed for the change to be reflected in the UI. This is not a + bug in Open vSwitch. + +Starting +-------- + +.. important:: + The following steps assume that you have installed the Open vSwitch utilities + in the local machine via 'make install'. + +Before starting ovs-vswitchd itself, you need to start its configuration +database, ovsdb-server. Each machine on which Open vSwitch is installed should +run its own copy of ovsdb-server. Before ovsdb-server itself can be started, +configure a database that it can use:: + + > ovsdb-tool create C:\openvswitch\etc\openvswitch\conf.db \ + C:\openvswitch\usr\share\openvswitch\vswitch.ovsschema + +Configure ovsdb-server to use database created above and to listen on a Unix +domain socket:: + + > ovsdb-server -vfile:info --remote=punix:db.sock --log-file \ + --pidfile --detach + +.. note:: + The logfile is created at ``C:/openvswitch/var/log/openvswitch/`` + +Initialize the database using ovs-vsctl. This is only necessary the first time +after you create the database with ovsdb-tool, though running it at any time is +harmless:: + + > ovs-vsctl --no-wait init + +.. tip:: + If you would later like to terminate the started ovsdb-server, run:: + + > ovs-appctl -t ovsdb-server exit + +Start the main Open vSwitch daemon, telling it to connect to the same Unix +domain socket:: + + > ovs-vswitchd -vfile:info --log-file --pidfile --detach + +.. tip:: + If you would like to terminate the started ovs-vswitchd, run:: + + > ovs-appctl exit + +.. note:: + The logfile is created at ``C:/openvswitch/var/log/openvswitch/`` + +Validating +---------- + +At this point you can use ovs-vsctl to set up bridges and other Open vSwitch +features. + +Add bridges +~~~~~~~~~~~ + +Let's start by creating an integration bridge, ``br-int`` and a PIF bridge, +``br-pif``:: + + > ovs-vsctl add-br br-int + > ovs-vsctl add-br br-pif + +.. note:: + There's a known bug that running the ovs-vsctl command does not terminate. + This is generally solved by having ovs-vswitchd running. If you face the + issue despite that, hit Ctrl-C to terminate ovs-vsctl and check the output to + see if your command succeeded. + +Validate that ports are added by dumping from both ovs-dpctl and ovs-vsctl:: + + > ovs-dpctl show + system@ovs-system: + lookups: hit:0 missed:0 lost:0 + flows: 0 + port 2: br-pif (internal) <<< internal port on 'br-pif' bridge + port 1: br-int (internal) <<< internal port on 'br-int' bridge + + > ovs-vsctl show + a56ec7b5-5b1f-49ec-a795-79f6eb63228b + Bridge br-pif + Port br-pif + Interface br-pif + type: internal + Bridge br-int + Port br-int + Interface br-int + type: internal + +.. note:: + There's a known bug that the ports added to OVSDB via ovs-vsctl don't get to + the kernel datapath immediately, ie. they don't show up in the output of + ``ovs-dpctl show`` even though they show up in output of ``ovs-vsctl show``. + In order to workaround this issue, restart ovs-vswitchd. (You can terminate + ovs-vswitchd by running ``ovs-appctl exit``.) + +Add physicals NICs (PIF) +~~~~~~~~~~~~~~~~~~~~~~~~ + +Now, let's add the physical NIC and the internal port to ``br-pif``. In OVS for +Hyper-V, we use the name of the adapter on top of which the Hyper-V virtual +switch was created, as a special name to refer to the physical NICs connected +to the Hyper-V switch, e.g. if we created the Hyper-V virtual switch on top of +the adapter named ``Ethernet0``, then in OVS we use that name (``Ethernet0``) +as a special name to refer to that adapter. + +.. note:: + we assume that the Hyper-V switch on which OVS extension is enabled has a + single physical NIC connected to it. + +An internal port is the virtual adapter created on the Hyper-V switch using the +``AllowManagementOS`` setting. This has already been setup while creating the +switch using the instructions above. In OVS for Hyper-V, we use a the name of +that specific adapter as a special name to refer to that adapter. By default it +is created under the following rule ``vEthernet ()``. + +As a whole example, if we issue the following in a powershell console:: + + PS C:\package\binaries> Get-NetAdapter | select Name,MacAddress,InterfaceDescription + Name MacAddress InterfaceDescription + ---- ---------- -------------------- + Ethernet1 00-0C-29-94-05-65 Intel(R) PRO/1000 MT Network Connection + vEthernet (external) 00-0C-29-94-05-5B Hyper-V Virtual Ethernet Adapter #2 + Ethernet0 00-0C-29-94-05-5B Intel(R) PRO/1000 MT Network Connection #2 + + PS C:\package\binaries> Get-VMSwitch + Name SwitchType NetAdapterInterfaceDescription + ---- ---------- ------------------------------ + external External Intel(R) PRO/1000 MT Network Connection #2 + +We can see that we have a switch(external) created upon adapter name +'Ethernet0' with an internal port under name ``vEthernet (external)``. Thus +resulting into the following ovs-vsctl commands:: + + > ovs-vsctl add-port br-pif Ethernet0 + > ovs-vsctl add-port br-pif "vEthernet (external)" + +Dumping the ports should show the additional ports that were just added:: + + > ovs-dpctl show + system@ovs-system: + lookups: hit:0 missed:0 lost:0 + flows: 0 + port 4: vEthernet (external) (internal) <<< 'AllowManagementOS' + adapter on + Hyper-V switch + port 2: br-pif (internal) + port 1: br-int (internal) + port 3: Ethernet0 <<< Physical NIC + + > ovs-vsctl show + a56ec7b5-5b1f-49ec-a795-79f6eb63228b + Bridge br-pif + Port "vEthernet (external)" + Interface "vEthernet (external)" + Port br-pif + Interface br-pif + type: internal + Port "Ethernet0" + Interface "Ethernet0" + Bridge br-int + Port br-int + Interface br-int + type: internal + +Add virtual interfaces (VIFs) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Adding VIFs to openvswitch is a two step procedure. The first step is to +assign a 'OVS port name' which is a unique name across all VIFs on this +Hyper-V. The next step is to add the VIF to the ovsdb using its 'OVS port +name' as key. + +First, assign a unique 'OVS port name' to the VIF. The VIF needs to have been +disconnected from the Hyper-V switch before assigning a 'OVS port name' to it. +In the example below, we assign a 'OVS port name' called ``ovs-port-a`` to a +VIF on a VM ``VM1``. By using index 0 for ``$vnic``, the first VIF of the VM +is being addressed. After assigning the name ``ovs-port-a``, the VIF is +connected back to the Hyper-V switch with name ``OVS-HV-Switch``, which is +assumed to be the Hyper-V switch with OVS extension enabled.:: + + PS> import-module .\datapath-windows\misc\OVS.psm1 + PS> $vnic = Get-VMNetworkAdapter + PS> Disconnect-VMNetworkAdapter -VMNetworkAdapter $vnic[0] + PS> $vnic[0] | Set-VMNetworkAdapterOVSPort -OVSPortName ovs-port-a + PS> Connect-VMNetworkAdapter -VMNetworkAdapter $vnic[0] \ + -SwitchName OVS-Extended-Switch + +Next, add the VIFs to ``br-int``:: + + > ovs-vsctl add-port br-int ovs-port-a + +Dumping the ports should show the additional ports that were just added:: + + > ovs-dpctl show + system@ovs-system: + lookups: hit:0 missed:0 lost:0 + flows: 0 + port 4: vEthernet (external) (internal) + port 5: ovs-port-a + port 2: br-pif (internal) + port 1: br-int (internal + port 3: Ethernet0 + + > ovs-vsctl show + 4cd86499-74df-48bd-a64d-8d115b12a9f2 + Bridge br-pif + Port "vEthernet (external)" + Interface "vEthernet (external)" + Port "Ethernet0" + Interface "Ethernet0" + Port br-pif + Interface br-pif + type: internal + Bridge br-int + Port br-int + Interface br-int + type: internal + Port "ovs-port-a" + Interface "ovs-port-a" + +Add patch ports and configure VLAN tagging +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Windows Open vSwitch implementation support VLAN tagging in the switch. +Switch VLAN tagging along with patch ports between ``br-int`` and ``br-pif`` is +used to configure VLAN tagging functionality between two VMs on different +Hyper-Vs. To start, add a patch port from ``br-int`` to ``br-pif``:: + + > ovs-vsctl add-port br-int patch-to-pif + > ovs-vsctl set interface patch-to-pif type=patch \ + options:peer=patch-to-int + +Add a patch port from ``br-pif`` to ``br-int``:: + + > ovs-vsctl add-port br-pif patch-to-int + > ovs-vsctl set interface patch-to-int type=patch \ + options:peer=patch-to-pif + +Re-Add the VIF ports with the VLAN tag:: + + > ovs-vsctl add-port br-int ovs-port-a tag=900 + > ovs-vsctl add-port br-int ovs-port-b tag=900 + +Add tunnels +~~~~~~~~~~~ + +The Windows Open vSwitch implementation support VXLAN and STT tunnels. To add +tunnels. For example, first add the tunnel port between 172.168.201.101 <-> +172.168.201.102:: + + > ovs-vsctl add-port br-int tun-1 + > ovs-vsctl set Interface tun-1 type= + > ovs-vsctl set Interface tun-1 options:local_ip=172.168.201.101 + > ovs-vsctl set Interface tun-1 options:remote_ip=172.168.201.102 + > ovs-vsctl set Interface tun-1 options:in_key=flow + > ovs-vsctl set Interface tun-1 options:out_key=flow + +...and the tunnel port between 172.168.201.101 <-> 172.168.201.105:: + + > ovs-vsctl add-port br-int tun-2 + > ovs-vsctl set Interface tun-2 type= + > ovs-vsctl set Interface tun-2 options:local_ip=172.168.201.102 + > ovs-vsctl set Interface tun-2 options:remote_ip=172.168.201.105 + > ovs-vsctl set Interface tun-2 options:in_key=flow + > ovs-vsctl set Interface tun-2 options:out_key=flow + +Where ```` is one of: ``stt`` or ``vxlan`` + +.. note:: + Any patch ports created between br-int and br-pif MUST be be deleted prior to + adding tunnels. + +Windows Services +---------------- + +Open vSwitch daemons come with support to run as a Windows service. The +instructions here assume that you have installed the Open vSwitch utilities and +daemons via ``make install``. + +.. note:: + The commands shown here can be run from MSYS bash or Windows command prompt. + +To start, create the database:: + + > ovsdb-tool create C:/openvswitch/etc/openvswitch/conf.db \ + "C:/openvswitch/usr/share/openvswitch/vswitch.ovsschema" + +Create the ovsdb-server service and start it:: + + > sc create ovsdb-server \ + binpath="C:/openvswitch/usr/sbin/ovsdb-server.exe \ + C:/openvswitch/etc/openvswitch/conf.db \ + -vfile:info --log-file --pidfile \ + --remote=punix:db.sock --service --service-monitor" + > sc start ovsdb-server + +.. tip:: + One of the common issues with creating a Windows service is with mungled + paths. You can make sure that the correct path has been registered with the + Windows services manager by running:: + + > sc qc ovsdb-server + +Check that the service is healthy by running:: + + > sc query ovsdb-server + +Initialize the database:: + + > ovs-vsctl --no-wait init + +Create the ovs-vswitchd service and start it:: + + > sc create ovs-vswitchd \ + binpath="C:/openvswitch/usr/sbin/ovs-vswitchd.exe \ + --pidfile -vfile:info --log-file --service --service-monitor" + > sc start ovs-vswitchd + +Check that the service is healthy by running:: + + > sc query ovs-vswitchd + +To stop and delete the services, run:: + + > sc stop ovs-vswitchd + > sc stop ovsdb-server + > sc delete ovs-vswitchd + > sc delete ovsdb-server + +Windows CI Service +------------------ + +`AppVeyor `__ provides a free Windows autobuild service for +opensource projects. Open vSwitch has integration with AppVeyor for continuous +build. A developer can build test his changes for Windows by logging into +appveyor.com using a github account, creating a new project by linking it to +his development repository in github and triggering a new build. + +TODO +---- + +* Investigate the working of sFlow on Windows and re-enable the unit tests. + +* Investigate and add the feature to provide QoS. + +* Sign the driver & create an MSI for installing the different OpenvSwitch + components on Windows. diff --git a/Documentation/intro/install/xenserver.rst b/Documentation/intro/install/xenserver.rst new file mode 100644 index 000000000..c0f5e3156 --- /dev/null +++ b/Documentation/intro/install/xenserver.rst @@ -0,0 +1,229 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +================================ +Open vSwitch on Citrix XenServer +================================ + +This document describes how to build and install Open vSwitch on a Citrix +XenServer host. If you want to install Open vSwitch on a generic Linux or BSD +host, refer to :doc:`general` instead. + +Open vSwitch should work with XenServer 5.6.100 and later. However, Open +vSwitch requires Python 2.7 or later, so using Open vSwitch with XenServer 6.5 +or earlier requires installing Python 2.7. + +Building +-------- + +You may build from an Open vSwitch distribution tarball or from an Open vSwitch +Git tree. The recommended build environment to build RPMs for Citrix XenServer +is the DDK VM available from Citrix. + +1. If you are building from an Open vSwitch Git tree, then you will need to + first create a distribution tarball by running:: + + $ ./boot.sh + $ ./configure + $ make dist + + You cannot run this in the DDK VM, because it lacks tools that are necessary + to bootstrap the Open vSwitch distribution. Instead, you must run this on a + machine that has the tools listed in :ref:`general-install-reqs` as + prerequisites for building from a Git tree. + +2. Copy the distribution tarball into ``/usr/src/redhat/SOURCES`` inside + the DDK VM. + +3. In the DDK VM, unpack the distribution tarball into a temporary directory + and "cd" into the root of the distribution tarball. + +4. To build Open vSwitch userspace, run:: + + $ rpmbuild -bb xenserver/openvswitch-xen.spec + + This produces three RPMs in ``/usr/src/redhat/RPMS/i386``: + + - ``openvswitch`` + - ``openvswitch-modules-xen`` + - ``openvswitch-debuginfo`` + + The above command automatically runs the Open vSwitch unit tests. To + disable the unit tests, run:: + + $ rpmbuild -bb --without check xenserver/openvswitch-xen.spec + +Build Parameters +---------------- + +``openvswitch-xen.spec`` needs to know a number of pieces of information about +the XenServer kernel. Usually, it can figure these out for itself, but if it +does not do it correctly then you can specify them yourself as parameters to +the build. Thus, the final ``rpmbuild`` step above can be elaborated as:: + + $ VERSION= + $ KERNEL_NAME= + $ KERNEL_VERSION= + $ KERNEL_FLAVOR= + $ rpmbuild \ + -D "openvswitch_version $VERSION" \ + -D "kernel_name $KERNEL_NAME" \ + -D "kernel_version $KERNEL_VERSION" \ + -D "kernel_flavor $KERNEL_FLAVOR" \ + -bb xenserver/openvswitch-xen.spec + +where: + +```` + is the version number that appears in the name of the Open vSwitch tarball, + e.g. 0.90.0. + +```` + is the name of the XenServer kernel package, e.g. ``kernel-xen`` or + ``kernel-NAME-xen``, without the ``kernel-`` prefix. + +```` + is the output of:: + + $ rpm -q --queryformat "%{Version}-%{Release}" , + + e.g. ``2.6.32.12-0.7.1.xs5.6.100.323.170596``, where + ```` is the name of the ``-devel`` package + corresponding to ````. + +```` + is either ``xen`` or ``kdump``, where ``xen`` flavor is the main running + kernel flavor and the ``kdump`` flavor is the crashdump kernel flavor. + Commonly, one would specify ``xen`` here. + +For XenServer 6.5 or above, the kernel version naming no longer contains +KERNEL_FLAVOR. In fact, only providing the ``uname -r`` output is enough. So, +the final ``rpmbuild`` step changes to:: + + $ KERNEL_UNAME=<`uname -r` output> + $ rpmbuild \ + -D "kenel_uname $KERNEL_UNAME" \ + -bb xenserver/openvswitch-xen.spec + +Installing Open vSwitch for XenServer +------------------------------------- + +To install Open vSwitch on a XenServer host, or to upgrade to a newer version, +copy the ``openvswitch`` and ``openvswitch-modules-xen`` RPMs to that host with +``scp``, then install them with ``rpm -U``, e.g.:: + + $ scp openvswitch-$VERSION-1.i386.rpm \ + openvswitch-modules-xen-$XEN_KERNEL_VERSION-$VERSION-1.i386.rpm \ + root@: + # Enter 's root password. + $ ssh root@ + # Enter 's root password again. + $ rpm -U openvswitch-$VERSION-1.i386.rpm \ + openvswitch-modules-xen-$XEN_KERNEL_VERSION-$VERSION-1.i386.rpm + +To uninstall Open vSwitch from a XenServer host, remove the packages:: + + $ ssh root@ + # Enter 's root password again. + $ rpm -e openvswitch openvswitch-modules-xen-$XEN_KERNEL_VERSION + +After installing or uninstalling Open vSwitch, the XenServer should be rebooted +as soon as possible. + +Open vSwitch Boot Sequence on XenServer +--------------------------------------- + +When Open vSwitch is installed on XenServer, its startup script +``/etc/init.d/openvswitch`` runs early in boot. It does roughly the following: + +* Loads the OVS kernel module, openvswitch. + +* Starts ovsdb-server, the OVS configuration database. + +* XenServer expects there to be no bridges configured at startup, but the OVS + configuration database likely still has bridges configured from before + reboot. To match XenServer expectations, the startup script deletes all + configured bridges from the database. + +* Starts ovs-vswitchd, the OVS switching daemon. + +At this point in the boot process, then, there are no Open vSwitch bridges, +even though all of the Open vSwitch daemons are running. Later on in boot, +``/etc/init.d/management-interface`` (part of XenServer, not Open vSwitch) +creates the bridge for the XAPI management interface by invoking +``/opt/xensource/libexec/interface-reconfigure``. Normally this program +consults XAPI's database to obtain information about how to configure the +bridge, but XAPI is not running yet(\*) so it instead consults +``/var/xapi/network.dbcache``, which is a cached copy of the most recent +network configuration. + +(\*) Even if XAPI were running, if this XenServer node is a pool slave then the + query would have to consult the master, which requires network access, + which begs the question of how to configure the management interface. + +XAPI starts later on in the boot process. XAPI can then create other bridges +on demand using ``/opt/xensource/libexec/interface-reconfigure``. Now that +XAPI is running, that program consults XAPI directly instead of reading the +cache. + +As part of its own startup, XAPI invokes the Open vSwitch XAPI plugin script +``/etc/xapi.d/openvswitch-cfg-update`` passing the ``update`` command. The +plugin script does roughly the following: + +* Calls ``/opt/xensource/libexec/interface-reconfigure`` with the ``rewrite`` + command, to ensure that the network cache is up-to-date. + +* Queries the Open vSwitch manager setting (named ``vswitch_controller``) from + the XAPI database for the XenServer pool. + +* If XAPI and OVS are configured for different managers, or if OVS is + configured for a manager but XAPI is not, runs ``ovs-vsctl emer-reset`` to + bring the Open vSwitch configuration to a known state. One effect of + emer-reset is to deconfigure any manager from the OVS database. + +* If XAPI is configured for a manager, configures the OVS manager to match with + ``ovs-vsctl set-manager``. + +Notes +----- + +* The Open vSwitch boot sequence only configures an OVS configuration database + manager. There is no way to directly configure an OpenFlow controller on + XenServer and, as a consequence of the step above that deletes all of the + bridges at boot time, controller configuration only persists until XenServer + reboot. The configuration database manager can, however, configure + controllers for bridges. See the BUGS section of ovs-testcontroller(8) for + more information on this topic. + +* The Open vSwitch startup script automatically adds a firewall rule to allow + GRE traffic. This rule is needed for the XenServer feature called "Cross-Host + Internal Networks" (CHIN) that uses GRE. If a user configures tunnels other + than GRE (ex: Geneve, VXLAN, LISP), they will have to either manually add a + iptables firewall rule to allow the tunnel traffic or add it through a + startup script (Please refer to the "enable-protocol" command in the + ovs-ctl(8) manpage). + +Reporting Bugs +-------------- + +Please report problems to bugs@openvswitch.org. diff --git a/FAQ.rst b/FAQ.rst index a933c78ae..c5ae62f4d 100644 --- a/FAQ.rst +++ b/FAQ.rst @@ -486,7 +486,7 @@ Q: How do I configure a DPDK port as an access port? Finally, it is required that DPDK port names begin with ``dpdk``. - See `INSTALL.DPDK `__ for more information on enabling + Refer to the `DPDK installation guide`_ for more information on enabling and using DPDK with Open vSwitch. Q: How do I configure a VLAN as an RSPAN VLAN, that is, enable mirroring of all @@ -794,7 +794,8 @@ very high. on the Port table in ovs-vswitchd.conf.db(5) for all the details. Configuration for DPDK-enabled interfaces is slightly less - straightforward: see `INSTALL.DPDK `__. + straightforward. Refer to the `DPDK installation guide`_ for more + information. - Perhaps you don't actually need eth0 and eth1 to be on the same bridge. For example, if you simply want to be able to connect each of them to @@ -862,8 +863,8 @@ port in the datapath? Linux GRE module is already loaded and blocking OVS (to confirm, check dmesg for errors regarding GRE registration). To fix this, unload all GRE modules that appear in lsmod as well as the OVS kernel module. You can then - reload the OVS module following the directions in the `installation guide - `__, which will ensure that dependencies are satisfied. + reload the OVS module following the directions in the `general installation + guide`_, which will ensure that dependencies are satisfied. Q: Open vSwitch does not seem to obey my packet filter rules. @@ -2048,8 +2049,7 @@ Q: How do I implement a new OpenFlow message? ``lib/ofp-msgs.h``, following the existing pattern. Then recompile and fix all of the new warnings, implementing new functionality for the new message as needed. (If you configure with ``--enable-Werror``, as described in the - `installation guide `__, then it is impossible to miss any - warnings.) + `general installation guide`_, then it is impossible to miss any warnings.) If you need to add an OpenFlow vendor extension message for a vendor that doesn't yet have any extension messages, then you will also need to edit @@ -2065,8 +2065,8 @@ Q: How do I add support for a new field or header? ``lib/nx-match.c`` to output your new field in OXM matches. Then recompile and fix all of the new warnings, implementing new functionality for the new field or header as needed. (If you configure with ``--enable-Werror``, as - described in the `installation guide `__, then it is impossible - to miss any warnings.) + described in the `general installation guide`_, then it is impossible to + miss any warnings.) If you want kernel datapath support for your new field, you also need to modify the kernel module for the operating systems you are interested in. @@ -2085,9 +2085,12 @@ Q: How do I add support for a new OpenFlow action? ``lib/ofp-actions.c``, following the existing pattern. Then recompile and fix all of the new warnings, implementing new functionality for the new action as needed. (If you configure with ``--enable-Werror``, as described - in the `installation guide `__, then it is impossible to miss - any warnings.) + in the `general installation guide`_, then it is impossible to miss any + warnings.) If you need to add an OpenFlow vendor extension action for a vendor that doesn't yet have any extension actions, then you will also need to edit ``build-aux/extract-ofp-actions``. + +.. _general installation guide: Documentation/intro/install/general.rst +.. _DPDK installation guide: Documentation/intro/install/dpdk.rst diff --git a/INSTALL.DPDK-ADVANCED.rst b/INSTALL.DPDK-ADVANCED.rst deleted file mode 100644 index 1c2437d89..000000000 --- a/INSTALL.DPDK-ADVANCED.rst +++ /dev/null @@ -1,939 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -================================= -Open vSwitch with DPDK (Advanced) -================================= - -The Advanced Install Guide explains how to improve OVS performance when using -DPDK datapath. This guide provides information on tuning, system configuration, -troubleshooting, static code analysis and testcases. - -Building as a Shared Library ----------------------------- - -DPDK can be built as a static or a shared library and shall be linked by -applications using DPDK datapath. When building OVS with DPDK, you can link -Open vSwitch against the shared DPDK library. - -.. note:: - Minor performance loss is seen with OVS when using shared DPDK library as - compared to static library. - -To build Open vSwitch using DPDK as a shared library, first refer to the `DPDK -installation guide`_ for download instructions for DPDK and OVS. - -Once DPDK and OVS have been downloaded, you must configure the DPDK library -accordingly. Simply set ``CONFIG_RTE_BUILD_SHARED_LIB=y`` in -``config/common_base``, then build and install DPDK. Once done, DPDK can be -built as usual. For example:: - - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET - $ make install T=$DPDK_TARGET DESTDIR=install - -Once DPDK is built, export the DPDK shared library location and setup OVS as -detailed in the `DPDK installation guide`_:: - - $ export LD_LIBRARY_PATH=$DPDK_DIR/x86_64-native-linuxapp-gcc/lib - -System Configuration --------------------- - -To achieve optimal OVS performance, the system can be configured and that -includes BIOS tweaks, Grub cmdline additions, better understanding of NUMA -nodes and apt selection of PCIe slots for NIC placement. - -Recommended BIOS Settings -~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. list-table:: Recommended BIOS Settings - :header-rows: 1 - - * - Setting - - Value - * - C3 Power State - - Disabled - * - C6 Power State - - Disabled - * - MLC Streamer - - Enabled - * - MLC Spacial Prefetcher - - Enabled - * - DCU Data Prefetcher - - Enabled - * - DCA - - Enabled - * - CPU Power and Performance - - Performance - * - Memeory RAS and Performance Config -> NUMA optimized - - Enabled - -PCIe Slot Selection -~~~~~~~~~~~~~~~~~~~ - -The fastpath performance can be affected by factors related to the placement of -the NIC, such as channel speeds between PCIe slot and CPU or the proximity of -PCIe slot to the CPU cores running the DPDK application. Listed below are the -steps to identify right PCIe slot. - -#. Retrieve host details using ``dmidecode``. For example:: - - $ dmidecode -t baseboard | grep "Product Name" - -#. Download the technical specification for product listed, e.g: S2600WT2 - -#. Check the Product Architecture Overview on the Riser slot placement, CPU - sharing info and also PCIe channel speeds - - For example: On S2600WT, CPU1 and CPU2 share Riser Slot 1 with Channel speed - between CPU1 and Riser Slot1 at 32GB/s, CPU2 and Riser Slot1 at 16GB/s. - Running DPDK app on CPU1 cores and NIC inserted in to Riser card Slots will - optimize OVS performance in this case. - -#. Check the Riser Card #1 - Root Port mapping information, on the available - slots and individual bus speeds. In S2600WT slot 1, slot 2 has high bus - speeds and are potential slots for NIC placement. - -Advanced Hugepage Setup -~~~~~~~~~~~~~~~~~~~~~~~ - -Allocate and mount 1 GB hugepages. - -- For persistent allocation of huge pages, add the following options to the - kernel bootline:: - - default_hugepagesz=1GB hugepagesz=1G hugepages=N - - For platforms supporting multiple huge page sizes, add multiple options:: - - default_hugepagesz= hugepagesz= hugepages=N - - where: - - ``N`` - number of huge pages requested - ``size`` - huge page size with an optional suffix ``[kKmMgG]`` - -- For run-time allocation of huge pages:: - - $ echo N > /sys/devices/system/node/nodeX/hugepages/hugepages-1048576kB/nr_hugepages - - where: - - ``N`` - number of huge pages requested - ``X`` - NUMA Node - - .. note:: - For run-time allocation of 1G huge pages, Contiguous Memory Allocator - (``CONFIG_CMA``) has to be supported by kernel, check your Linux distro. - -Now mount the huge pages, if not already done so:: - - $ mount -t hugetlbfs -o pagesize=1G none /dev/hugepages - -Enable HyperThreading -~~~~~~~~~~~~~~~~~~~~~ - -With HyperThreading, or SMT, enabled, a physical core appears as two logical -cores. SMT can be utilized to spawn worker threads on logical cores of the same -physical core there by saving additional cores. - -With DPDK, when pinning pmd threads to logical cores, care must be taken to set -the correct bits of the ``pmd-cpu-mask`` to ensure that the pmd threads are -pinned to SMT siblings. - -Take a sample system configuration, with 2 sockets, 2 * 10 core processors, HT -enabled. This gives us a total of 40 logical cores. To identify the physical -core shared by two logical cores, run:: - - $ cat /sys/devices/system/cpu/cpuN/topology/thread_siblings_list - -where ``N`` is the logical core number. - -In this example, it would show that cores ``1`` and ``21`` share the same -physical core. As cores are counted from 0, the ``pmd-cpu-mask`` can be used -to enable these two pmd threads running on these two logical cores (one -physical core) is:: - - $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x200002 - -Isolate Cores -~~~~~~~~~~~~~ - -The ``isolcpus`` option can be used to isolate cores from the Linux scheduler. -The isolated cores can then be used to dedicatedly run HPC applications or -threads. This helps in better application performance due to zero context -switching and minimal cache thrashing. To run platform logic on core 0 and -isolate cores between 1 and 19 from scheduler, add ``isolcpus=1-19`` to GRUB -cmdline. - -.. note:: - It has been verified that core isolation has minimal advantage due to mature - Linux scheduler in some circumstances. - -NUMA/Cluster-on-Die -~~~~~~~~~~~~~~~~~~~ - -Ideally inter-NUMA datapaths should be avoided where possible as packets will -go across QPI and there may be a slight performance penalty when compared with -intra NUMA datapaths. On Intel Xeon Processor E5 v3, Cluster On Die is -introduced on models that have 10 cores or more. This makes it possible to -logically split a socket into two NUMA regions and again it is preferred where -possible to keep critical datapaths within the one cluster. - -It is good practice to ensure that threads that are in the datapath are pinned -to cores in the same NUMA area. e.g. pmd threads and QEMU vCPUs responsible for -forwarding. If DPDK is built with ``CONFIG_RTE_LIBRTE_VHOST_NUMA=y``, vHost -User ports automatically detect the NUMA socket of the QEMU vCPUs and will be -serviced by a PMD from the same node provided a core on this node is enabled in -the ``pmd-cpu-mask``. ``libnuma`` packages are required for this feature. - -Compiler Optimizations -~~~~~~~~~~~~~~~~~~~~~~ - -The default compiler optimization level is ``-O2``. Changing this to more -aggressive compiler optimization such as ``-O3 -march=native`` with -gcc (verified on 5.3.1) can produce performance gains though not siginificant. -``-march=native`` will produce optimized code on local machine and should be -used when software compilation is done on Testbed. - -Performance Tuning ------------------- - -Affinity -~~~~~~~~ - -For superior performance, DPDK pmd threads and Qemu vCPU threads needs to be -affinitized accordingly. - -- PMD thread Affinity - - A poll mode driver (pmd) thread handles the I/O of all DPDK interfaces - assigned to it. A pmd thread shall poll the ports for incoming packets, - switch the packets and send to tx port. pmd thread is CPU bound, and needs - to be affinitized to isolated cores for optimum performance. - - By setting a bit in the mask, a pmd thread is created and pinned to the - corresponding CPU core. e.g. to run a pmd thread on core 2:: - - $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x4 - - .. note:: - pmd thread on a NUMA node is only created if there is at least one DPDK - interface from that NUMA node added to OVS. - -- QEMU vCPU thread Affinity - - A VM performing simple packet forwarding or running complex packet pipelines - has to ensure that the vCPU threads performing the work has as much CPU - occupancy as possible. - - For example, on a multicore VM, multiple QEMU vCPU threads shall be spawned. - When the DPDK ``testpmd`` application that does packet forwarding is invoked, - the ``taskset`` command should be used to affinitize the vCPU threads to the - dedicated isolated cores on the host system. - -Multiple Poll-Mode Driver Threads -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -With pmd multi-threading support, OVS creates one pmd thread for each NUMA node -by default. However, in cases where there are multiple ports/rxq's producing -traffic, performance can be improved by creating multiple pmd threads running -on separate cores. These pmd threads can share the workload by each being -responsible for different ports/rxq's. Assignment of ports/rxq's to pmd threads -is done automatically. - -A set bit in the mask means a pmd thread is created and pinned to the -corresponding CPU core. For example, to run pmd threads on core 1 and 2:: - - $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x6 - -When using dpdk and dpdkvhostuser ports in a bi-directional VM loopback as -shown below, spreading the workload over 2 or 4 pmd threads shows significant -improvements as there will be more total CPU occupancy available:: - - NIC port0 <-> OVS <-> VM <-> OVS <-> NIC port 1 - -DPDK Physical Port Rx Queues -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -:: - - $ ovs-vsctl set Interface options:n_rxq= - -The command above sets the number of rx queues for DPDK physical interface. -The rx queues are assigned to pmd threads on the same NUMA node in a -round-robin fashion. - -DPDK Physical Port Queue Sizes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -:: - - $ ovs-vsctl set Interface dpdk0 options:n_rxq_desc= - $ ovs-vsctl set Interface dpdk0 options:n_txq_desc= - -The command above sets the number of rx/tx descriptors that the NIC associated -with dpdk0 will be initialised with. - -Different ``n_rxq_desc`` and ``n_txq_desc`` configurations yield different -benefits in terms of throughput and latency for different scenarios. -Generally, smaller queue sizes can have a positive impact for latency at the -expense of throughput. The opposite is often true for larger queue sizes. -Note: increasing the number of rx descriptors eg. to 4096 may have a negative -impact on performance due to the fact that non-vectorised DPDK rx functions may -be used. This is dependant on the driver in use, but is true for the commonly -used i40e and ixgbe DPDK drivers. - -Exact Match Cache -~~~~~~~~~~~~~~~~~ - -Each pmd thread contains one Exact Match Cache (EMC). After initial flow setup -in the datapath, the EMC contains a single table and provides the lowest level -(fastest) switching for DPDK ports. If there is a miss in the EMC then the next -level where switching will occur is the datapath classifier. Missing in the -EMC and looking up in the datapath classifier incurs a significant performance -penalty. If lookup misses occur in the EMC because it is too small to handle -the number of flows, its size can be increased. The EMC size can be modified by -editing the define ``EM_FLOW_HASH_SHIFT`` in ``lib/dpif-netdev.c``. - -As mentioned above, an EMC is per pmd thread. An alternative way of increasing -the aggregate amount of possible flow entries in EMC and avoiding datapath -classifier lookups is to have multiple pmd threads running. - -Rx Mergeable Buffers -~~~~~~~~~~~~~~~~~~~~ - -Rx mergeable buffers is a virtio feature that allows chaining of multiple -virtio descriptors to handle large packet sizes. Large packets are handled by -reserving and chaining multiple free descriptors together. Mergeable buffer -support is negotiated between the virtio driver and virtio device and is -supported by the DPDK vhost library. This behavior is supported and enabled by -default, however in the case where the user knows that rx mergeable buffers are -not needed i.e. jumbo frames are not needed, it can be forced off by adding -``mrg_rxbuf=off`` to the QEMU command line options. By not reserving multiple -chains of descriptors it will make more individual virtio descriptors available -for rx to the guest using dpdkvhost ports and this can improve performance. - -OVS Testcases -------------- - -PHY-VM-PHY (vHost Loopback) -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The `DPDK installation guide`_ details steps for PHY-VM-PHY loopback testcase -and packet forwarding using DPDK testpmd application in the Guest VM. For users -wishing to do packet forwarding using kernel stack below, you need to run the -below commands on the guest:: - - $ ifconfig eth1 1.1.1.2/24 - $ ifconfig eth2 1.1.2.2/24 - $ systemctl stop firewalld.service - $ systemctl stop iptables.service - $ sysctl -w net.ipv4.ip_forward=1 - $ sysctl -w net.ipv4.conf.all.rp_filter=0 - $ sysctl -w net.ipv4.conf.eth1.rp_filter=0 - $ sysctl -w net.ipv4.conf.eth2.rp_filter=0 - $ route add -net 1.1.2.0/24 eth2 - $ route add -net 1.1.1.0/24 eth1 - $ arp -s 1.1.2.99 DE:AD:BE:EF:CA:FE - $ arp -s 1.1.1.99 DE:AD:BE:EF:CA:EE - -PHY-VM-PHY (IVSHMEM) -~~~~~~~~~~~~~~~~~~~~ - -IVSHMEM can also be validated using the PHY-VM-PHY configuration. To begin, -follow the steps described in the `DPDK installation guide`_ to create and -initialize the database, start ovs-vswitchd and add ``dpdk``-type devices to -bridge ``br0``. Once complete, follow the below steps: - -1. Add DPDK ring port to the bridge:: - - $ ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr - -2. Build modified QEMU - - QEMU must be patched to enable IVSHMEM support:: - - $ cd /usr/src/ - $ wget http://wiki.qemu.org/download/qemu-2.2.1.tar.bz2 - $ tar -jxvf qemu-2.2.1.tar.bz2 - $ cd /usr/src/qemu-2.2.1 - $ wget https://raw.githubusercontent.com/netgroup-polito/un-orchestrator/master/orchestrator/compute_controller/plugins/kvm-libvirt/patches/ivshmem-qemu-2.2.1.patch - $ patch -p1 < ivshmem-qemu-2.2.1.patch - $ ./configure --target-list=x86_64-softmmu --enable-debug --extra-cflags='-g' - $ make -j 4 - -3. Generate QEMU commandline:: - - $ mkdir -p /usr/src/cmdline_generator - $ cd /usr/src/cmdline_generator - $ wget https://raw.githubusercontent.com/netgroup-polito/un-orchestrator/master/orchestrator/compute_controller/plugins/kvm-libvirt/cmdline_generator/cmdline_generator.c - $ wget https://raw.githubusercontent.com/netgroup-polito/un-orchestrator/master/orchestrator/compute_controller/plugins/kvm-libvirt/cmdline_generator/Makefile - $ export RTE_SDK=/usr/src/dpdk-16.11 - $ export RTE_TARGET=x86_64-ivshmem-linuxapp-gcc - $ make - $ ./build/cmdline_generator -m -p dpdkr0 XXX - $ cmdline=`cat OVSMEMPOOL` - -4. Start guest VM:: - - $ export VM_NAME=ivshmem-vm - $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 - $ export QEMU_BIN=/usr/src/qemu-2.2.1/x86_64-softmmu/qemu-system-x86_64 - $ taskset 0x20 $QEMU_BIN -cpu host -smp 2,cores=2 -hda $QCOW2_IMAGE \ - -m 4096 --enable-kvm -name $VM_NAME -nographic -vnc :2 \ - -pidfile /tmp/vm1.pid $cmdline - -5. Build and run the sample ``dpdkr`` app in VM:: - - $ echo 1024 > /proc/sys/vm/nr_hugepages - $ mount -t hugetlbfs nodev /dev/hugepages (if not already mounted) - - # Build the DPDK ring application in the VM - $ export RTE_SDK=/root/dpdk-16.11 - $ export RTE_TARGET=x86_64-ivshmem-linuxapp-gcc - $ make - - # Run dpdkring application - $ ./build/dpdkr -c 1 -n 4 -- -n 0 - # where "-n 0" refers to ring '0' i.e dpdkr0 - -PHY-VM-PHY (vHost Multiqueue) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -vHost Multique functionality can also be validated using the PHY-VM-PHY -configuration. To begin, follow the steps described in the `DPDK installation -guide`_ to create and initialize the database, start ovs-vswitchd and add -``dpdk``-type devices to bridge ``br0``. Once complete, follow the below steps: - -1. Configure PMD and RXQs. - - For example, set the number of dpdk port rx queues to at least 2 The number - of rx queues at vhost-user interface gets automatically configured after - virtio device connection and doesn't need manual configuration:: - - $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0xC - $ ovs-vsctl set Interface dpdk0 options:n_rxq=2 - $ ovs-vsctl set Interface dpdk1 options:n_rxq=2 - -2. Instantiate Guest VM using QEMU cmdline - - We must configure with appropriate software versions to ensure this feature - is supported. - - .. list-table:: Recommended BIOS Settings - :header-rows: 1 - - * - Setting - - Value - * - QEMU version - - 2.5.0 - * - QEMU thread affinity - - 2 cores (taskset 0x30) - * - Memory - - 4 GB - * - Cores - - 2 - * - Distro - - Fedora 22 - * - Multiqueue - - Enabled - - To do this, instantiate the guest as follows:: - - $ export VM_NAME=vhost-vm - $ export GUEST_MEM=4096M - $ export QCOW2_IMAGE=/root/Fedora22_x86_64.qcow2 - $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch - $ taskset 0x30 qemu-system-x86_64 -cpu host -smp 2,cores=2 -m 4096M \ - -drive file=$QCOW2_IMAGE --enable-kvm -name $VM_NAME \ - -nographic -numa node,memdev=mem -mem-prealloc \ - -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ - -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ - -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce,queues=2 \ - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,vectors=6 \ - -chardev socket,id=char2,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ - -netdev type=vhost-user,id=mynet2,chardev=char2,vhostforce,queues=2 \ - -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mq=on,vectors=6 - - .. note:: - Queue value above should match the queues configured in OVS, The vector - value should be set to "number of queues x 2 + 2" - -3. Configure the guest interface - - Assuming there are 2 interfaces in the guest named eth0, eth1 check the - channel configuration and set the number of combined channels to 2 for - virtio devices:: - - $ ethtool -l eth0 - $ ethtool -L eth0 combined 2 - $ ethtool -L eth1 combined 2 - - More information can be found in vHost walkthrough section. - -4. Configure kernel packet forwarding - - Configure IP and enable interfaces:: - - $ ifconfig eth0 5.5.5.1/24 up - $ ifconfig eth1 90.90.90.1/24 up - - Configure IP forwarding and add route entries:: - - $ sysctl -w net.ipv4.ip_forward=1 - $ sysctl -w net.ipv4.conf.all.rp_filter=0 - $ sysctl -w net.ipv4.conf.eth0.rp_filter=0 - $ sysctl -w net.ipv4.conf.eth1.rp_filter=0 - $ ip route add 2.1.1.0/24 dev eth1 - $ route add default gw 2.1.1.2 eth1 - $ route add default gw 90.90.90.90 eth1 - $ arp -s 90.90.90.90 DE:AD:BE:EF:CA:FE - $ arp -s 2.1.1.2 DE:AD:BE:EF:CA:FA - - Check traffic on multiple queues:: - - $ cat /proc/interrupts | grep virtio - -vHost Walkthrough ------------------ - -Two types of vHost User ports are available in OVS: - -- vhost-user (``dpdkvhostuser``) - -- vhost-user-client (``dpdkvhostuserclient``) - -vHost User uses a client-server model. The server creates/manages/destroys the -vHost User sockets, and the client connects to the server. Depending on which -port type you use, ``dpdkvhostuser`` or ``dpdkvhostuserclient``, a different -configuration of the client-server model is used. - -For vhost-user ports, Open vSwitch acts as the server and QEMU the client. For -vhost-user-client ports, Open vSwitch acts as the client and QEMU the server. - -vhost-user -~~~~~~~~~~ - -1. Install the prerequisites: - - - QEMU version >= 2.2 - -2. Add vhost-user ports to the switch. - - Unlike DPDK ring ports, DPDK vhost-user ports can have arbitrary names, - except that forward and backward slashes are prohibited in the names. - - For vhost-user, the name of the port type is ``dpdkvhostuser``:: - - $ ovs-vsctl add-port br0 vhost-user-1 -- set Interface vhost-user-1 \ - type=dpdkvhostuser - - This action creates a socket located at - ``/usr/local/var/run/openvswitch/vhost-user-1``, which you must provide to - your VM on the QEMU command line. More instructions on this can be found in - the next section "Adding vhost-user ports to VM" - - .. note:: - If you wish for the vhost-user sockets to be created in a sub-directory of - ``/usr/local/var/run/openvswitch``, you may specify this directory in the - ovsdb like so:: - - $ ovs-vsctl --no-wait \ - set Open_vSwitch . other_config:vhost-sock-dir=subdir` - -3. Add vhost-user ports to VM - - 1. Configure sockets - - Pass the following parameters to QEMU to attach a vhost-user device:: - - -chardev socket,id=char1,path=/usr/local/var/run/openvswitch/vhost-user-1 - -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1 - - where ``vhost-user-1`` is the name of the vhost-user port added to the - switch. - - Repeat the above parameters for multiple devices, changing the chardev - ``path`` and ``id`` as necessary. Note that a separate and different - chardev ``path`` needs to be specified for each vhost-user device. For - example you have a second vhost-user port named ``vhost-user-2``, you - append your QEMU command line with an additional set of parameters:: - - -chardev socket,id=char2,path=/usr/local/var/run/openvswitch/vhost-user-2 - -netdev type=vhost-user,id=mynet2,chardev=char2,vhostforce - -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2 - - 2. Configure hugepages - - QEMU must allocate the VM's memory on hugetlbfs. vhost-user ports access - a virtio-net device's virtual rings and packet buffers mapping the VM's - physical memory on hugetlbfs. To enable vhost-user ports to map the VM's - memory into their process address space, pass the following parameters - to QEMU:: - - -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on - -numa node,memdev=mem -mem-prealloc - - 3. Enable multiqueue support (optional) - - QEMU needs to be configured to use multiqueue:: - - -chardev socket,id=char2,path=/usr/local/var/run/openvswitch/vhost-user-2 - -netdev type=vhost-user,id=mynet2,chardev=char2,vhostforce,queues=$q - -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mq=on,vectors=$v - - where: - - ``$q`` - The number of queues - ``$v`` - The number of vectors, which is ``$q`` * 2 + 2 - - The vhost-user interface will be automatically reconfigured with - required number of rx and tx queues after connection of virtio device. - Manual configuration of ``n_rxq`` is not supported because OVS will work - properly only if ``n_rxq`` will match number of queues configured in - QEMU. - - A least 2 PMDs should be configured for the vswitch when using - multiqueue. Using a single PMD will cause traffic to be enqueued to the - same vhost queue rather than being distributed among different vhost - queues for a vhost-user interface. - - If traffic destined for a VM configured with multiqueue arrives to the - vswitch via a physical DPDK port, then the number of rxqs should also be - set to at least 2 for that physical DPDK port. This is required to - increase the probability that a different PMD will handle the multiqueue - transmission to the guest using a different vhost queue. - - If one wishes to use multiple queues for an interface in the guest, the - driver in the guest operating system must be configured to do so. It is - recommended that the number of queues configured be equal to ``$q``. - - For example, this can be done for the Linux kernel virtio-net driver - with:: - - $ ethtool -L combined <$q> - - where: - - ``-L`` - Changes the numbers of channels of the specified network device - ``combined`` - Changes the number of multi-purpose channels. - -Configure the VM using libvirt -++++++++++++++++++++++++++++++ - -You can also build and configure the VM using libvirt rather than QEMU by -itself. - -1. Change the user/group, access control policty and restart libvirtd. - - - In ``/etc/libvirt/qemu.conf`` add/edit the following lines:: - - user = "root" - group = "root" - - - Disable SELinux or set to permissive mode:: - - $ setenforce 0 - - - Restart the libvirtd process, For example, on Fedora:: - - $ systemctl restart libvirtd.service - -2. Instantiate the VM - - - Copy the XML configuration described in the `DPDK installation guide`_. - - - Start the VM:: - - $ virsh create demovm.xml - - - Connect to the guest console:: - - $ virsh console demovm - -3. Configure the VM - - The demovm xml configuration is aimed at achieving out of box performance on - VM. - - - The vcpus are pinned to the cores of the CPU socket 0 using ``vcpupin``. - - - Configure NUMA cell and memory shared using ``memAccess='shared'``. - - - Disable ``mrg_rxbuf='off'`` - -Refer to the `libvirt documentation `__ -for more information. - -vhost-user-client -~~~~~~~~~~~~~~~~~ - -1. Install the prerequisites: - - - QEMU version >= 2.7 - -2. Add vhost-user-client ports to the switch. - - Unlike vhost-user ports, the name given to port does not govern the name of - the socket device. ``vhost-server-path`` reflects the full path of the - socket that has been or will be created by QEMU for the given vHost User - client port. - - For vhost-user-client, the name of the port type is - ``dpdkvhostuserclient``:: - - $ VHOST_USER_SOCKET_PATH=/path/to/socker - $ ovs-vsctl add-port br0 vhost-client-1 \ - -- set Interface vhost-client-1 type=dpdkvhostuserclient \ - options:vhost-server-path=$VHOST_USER_SOCKET_PATH - -3. Add vhost-user-client ports to VM - - 1. Configure sockets - - Pass the following parameters to QEMU to attach a vhost-user device:: - - -chardev socket,id=char1,path=$VHOST_USER_SOCKET_PATH,server - -netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1 - - where ``vhost-user-1`` is the name of the vhost-user port added to the - switch. - - If the corresponding dpdkvhostuserclient port has not yet been configured - in OVS with ``vhost-server-path=/path/to/socket``, QEMU will print a log - similar to the following:: - - QEMU waiting for connection on: disconnected:unix:/path/to/socket,server - - QEMU will wait until the port is created sucessfully in OVS to boot the VM. - - One benefit of using this mode is the ability for vHost ports to - 'reconnect' in event of the switch crashing or being brought down. Once - it is brought back up, the vHost ports will reconnect automatically and - normal service will resume. - -DPDK Backend Inside VM -~~~~~~~~~~~~~~~~~~~~~~ - -Additional configuration is required if you want to run ovs-vswitchd with DPDK -backend inside a QEMU virtual machine. Ovs-vswitchd creates separate DPDK TX -queues for each CPU core available. This operation fails inside QEMU virtual -machine because, by default, VirtIO NIC provided to the guest is configured to -support only single TX queue and single RX queue. To change this behavior, you -need to turn on ``mq`` (multiqueue) property of all ``virtio-net-pci`` devices -emulated by QEMU and used by DPDK. You may do it manually (by changing QEMU -command line) or, if you use Libvirt, by adding the following string to -```` sections of all network devices used by DPDK:: - - - -Where: - -``N`` - determines how many queues can be used by the guest. - -This requires QEMU >= 2.2. - -QoS ---- - -Assuming you have a vhost-user port transmitting traffic consisting of packets -of size 64 bytes, the following command would limit the egress transmission -rate of the port to ~1,000,000 packets per second:: - - $ ovs-vsctl set port vhost-user0 qos=@newqos -- \ - --id=@newqos create qos type=egress-policer other-config:cir=46000000 \ - other-config:cbs=2048` - -To examine the QoS configuration of the port, run:: - - $ ovs-appctl -t ovs-vswitchd qos/show vhost-user0 - -To clear the QoS configuration from the port and ovsdb, run:: - - $ ovs-vsctl destroy QoS vhost-user0 -- clear Port vhost-user0 qos - -Refer to vswitch.xml for more details on egress-policer. - -Rate Limiting --------------- - -Here is an example on Ingress Policing usage. Assuming you have a vhost-user -port receiving traffic consisting of packets of size 64 bytes, the following -command would limit the reception rate of the port to ~1,000,000 packets per -second:: - - $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=368000 \ - ingress_policing_burst=1000` - -To examine the ingress policer configuration of the port:: - - $ ovs-vsctl list interface vhost-user0 - -To clear the ingress policer configuration from the port:: - - $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 - -Refer to vswitch.xml for more details on ingress-policer. - -Flow Control ------------- - -Flow control can be enabled only on DPDK physical ports. To enable flow -control support at tx side while adding a port, run:: - - $ ovs-vsctl add-port br0 dpdk0 -- \ - set Interface dpdk0 type=dpdk options:tx-flow-ctrl=true - -Similarly, to enable rx flow control, run:: - - $ ovs-vsctl add-port br0 dpdk0 -- \ - set Interface dpdk0 type=dpdk options:rx-flow-ctrl=true - -To enable flow control auto-negotiation, run:: - - $ ovs-vsctl add-port br0 dpdk0 -- \ - set Interface dpdk0 type=dpdk options:flow-ctrl-autoneg=true - -To turn ON the tx flow control at run time(After the port is being added to -OVS):: - - $ ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=true - -The flow control parameters can be turned off by setting ``false`` to the -respective parameter. To disable the flow control at tx side, run:: - - $ ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=false - -pdump ------ - -Pdump allows you to listen on DPDK ports and view the traffic that is passing -on them. To use this utility, one must have libpcap installed on the system. -Furthermore, DPDK must be built with ``CONFIG_RTE_LIBRTE_PDUMP=y`` and -``CONFIG_RTE_LIBRTE_PMD_PCAP=y``. - -.. warning:: - A performance decrease is expected when using a monitoring application like - the DPDK pdump app. - -To use pdump, simply launch OVS as usual. Then, navigate to the ``app/pdump`` -directory in DPDK, ``make`` the application and run like so:: - - $ sudo ./build/app/dpdk-pdump -- \ - --pdump port=0,queue=0,rx-dev=/tmp/pkts.pcap \ - --server-socket-path=/usr/local/var/run/openvswitch - -The above command captures traffic received on queue 0 of port 0 and stores it -in ``/tmp/pkts.pcap``. Other combinations of port numbers, queues numbers and -pcap locations are of course also available to use. For example, to capture all -packets that traverse port 0 in a single pcap file:: - - $ sudo ./build/app/dpdk-pdump -- \ - --pdump 'port=0,queue=*,rx-dev=/tmp/pkts.pcap,tx-dev=/tmp/pkts.pcap' \ - --server-socket-path=/usr/local/var/run/openvswitch - -``server-socket-path`` must be set to the value of ovs_rundir() which typically -resolves to ``/usr/local/var/run/openvswitch``. - -Many tools are available to view the contents of the pcap file. Once example is -tcpdump. Issue the following command to view the contents of ``pkts.pcap``:: - - $ tcpdump -r pkts.pcap - -More information on the pdump app and its usage can be found in the `DPDK docs -`__. - -Jumbo Frames ------------- - -By default, DPDK ports are configured with standard Ethernet MTU (1500B). To -enable Jumbo Frames support for a DPDK port, change the Interface's -``mtu_request`` attribute to a sufficiently large value. For example, to add a -DPDK Phy port with MTU of 9000:: - - $ ovs-vsctl add-port br0 dpdk0 \ - -- set Interface dpdk0 type=dpdk \ - -- set Interface dpdk0 mtu_request=9000` - -Similarly, to change the MTU of an existing port to 6200:: - - $ ovs-vsctl set Interface dpdk0 mtu_request=6200 - -Some additional configuration is needed to take advantage of jumbo frames with -vHost ports: - -1. *mergeable buffers* must be enabled for vHost ports, as demonstrated in the - QEMU command line snippet below:: - - -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=on - -2. Where virtio devices are bound to the Linux kernel driver in a guest - environment (i.e. interfaces are not bound to an in-guest DPDK driver), the - MTU of those logical network interfaces must also be increased to a - sufficiently large value. This avoids segmentation of Jumbo Frames received - in the guest. Note that 'MTU' refers to the length of the IP packet only, - and not that of the entire frame. - - To calculate the exact MTU of a standard IPv4 frame, subtract the L2 header - and CRC lengths (i.e. 18B) from the max supported frame size. So, to set - the MTU for a 9018B Jumbo Frame:: - - $ ifconfig eth1 mtu 9000 - -When Jumbo Frames are enabled, the size of a DPDK port's mbuf segments are -increased, such that a full Jumbo Frame of a specific size may be accommodated -within a single mbuf segment. - -Jumbo frame support has been validated against 9728B frames, which is the -largest frame size supported by Fortville NIC using the DPDK i40e driver, but -larger frames and other DPDK NIC drivers may be supported. These cases are -common for use cases involving East-West traffic only. - -vsperf ------- - -The vsperf project aims to develop a vSwitch test framework that can be used to -validate the suitability of different vSwitch implementations in a telco -deployment environment. More information can be found on the `OPNFV wiki -`__. - -Bug Reporting -------------- - -Report problems to bugs@openvswitch.org. - -.. _DPDK installation guide: INSTALL.DPDK.rst diff --git a/INSTALL.DPDK.rst b/INSTALL.DPDK.rst deleted file mode 100644 index 10ccfdf3b..000000000 --- a/INSTALL.DPDK.rst +++ /dev/null @@ -1,604 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -====================== -Open vSwitch with DPDK -====================== - -This document describes how to build and install Open vSwitch using a DPDK -datapath. Open vSwitch can use the DPDK library to operate entirely in -userspace. - -.. warning:: - The DPDK support of Open vSwitch is considered 'experimental'. - -Build requirements ------------------- - -In addition to the requirements described in the `installation guide -`__, building Open vSwitch with DPDK will require the following: - -- DPDK 16.11 - -- A `DPDK supported NIC`_ - - Only required when physical ports are in use - -- A suitable kernel - - On Linux Distros running kernel version >= 3.0, only `IOMMU` needs to enabled - via the grub cmdline, assuming you are using **VFIO**. For older kernels, - ensure the kernel is built with ``UIO``, ``HUGETLBFS``, - ``PROC_PAGE_MONITOR``, ``HPET``, ``HPET_MMAP`` support. If these are not - present, it will be necessary to upgrade your kernel or build a custom kernel - with these flags enabled. - -Detailed system requirements can be found at `DPDK requirements`_, while more -detailed install information can be found in the `advanced installation guide -`__. - -.. _DPDK supported NIC: http://dpdk.org/doc/nics -.. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html - -Installing ----------- - -DPDK -~~~~ - -1. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: - - $ cd /usr/src/ - $ wget http://fast.dpdk.org/rel/dpdk-16.11.tar.xz - $ tar xf dpdk-16.11.tar.xz - $ export DPDK_DIR=/usr/src/dpdk-16.11 - $ cd $DPDK_DIR - -2. Configure and install DPDK - - Build and install the DPDK library:: - - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET - $ make install T=$DPDK_TARGET DESTDIR=install - - If IVSHMEM support is required, use a different target:: - - $ export DPDK_TARGET=x86_64-ivshmem-linuxapp-gcc - -.. _DPDK sources: http://dpdk.org/rel - -Install OVS -~~~~~~~~~~~ - -OVS can be installed using different methods. For OVS to use DPDK datapath, it -has to be configured with DPDK support (``--with-dpdk``). - -.. note:: - This section focuses on generic recipe that suits most cases. For - distribution specific instructions, refer to one of the more relevant guides. - -.. _OVS sources: http://openvswitch.org/releases/ - -1. Ensure the standard OVS requirements, described in the `installation guide - `__, are installed. - -2. Bootstrap, if required, as described in the `installation guide - `__. - -3. Configure the package using the ``--with-dpdk`` flag:: - - $ ./configure --with-dpdk=$DPDK_BUILD - - where ``DPDK_BUILD`` is the path to the built DPDK library. This can be - skipped if DPDK library is installed in its default location. - - .. note:: - While ``--with-dpdk`` is required, you can pass any other configuration - option described in the `installation guide `__. - -4. Build and install OVS, as described in the `installation guide - `__. - -Additional information can be found in the `installation guide -`__. - -Setup ------ - -Setup Hugepages -~~~~~~~~~~~~~~~ - -Allocate a number of 2M Huge pages: - -- For persistent allocation of huge pages, write to hugepages.conf file - in `/etc/sysctl.d`:: - - $ echo 'vm.nr_hugepages=2048' > /etc/sysctl.d/hugepages.conf - -- For run-time allocation of huge pages, use the ``sysctl`` utility:: - - $ sysctl -w vm.nr_hugepages=N # where N = No. of 2M huge pages - -To verify hugepage configuration:: - - $ grep HugePages_ /proc/meminfo - -Mount the hugepages, if not already mounted by default:: - - $ mount -t hugetlbfs none /dev/hugepages`` - -.. _dpdk-vfio: - -Setup DPDK devices using VFIO -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -VFIO is prefered to the UIO driver when using recent versions of DPDK. VFIO -support required support from both the kernel and BIOS. For the former, kernel -version > 3.6 must be used. For the latter, you must enable VT-d in the BIOS -and ensure this is configured via grub. To ensure VT-d is enabled via the BIOS, -run:: - - $ dmesg | grep -e DMAR -e IOMMU - -If VT-d is not enabled in the BIOS, enable it now. - -To ensure VT-d is enabled in the kernel, run:: - - $ cat /proc/cmdline | grep iommu=pt - $ cat /proc/cmdline | grep intel_iommu=on - -If VT-d is not enabled in the kernel, enable it now. - -Once VT-d is correctly configured, load the required modules and bind the NIC -to the VFIO driver:: - - $ modprobe vfio-pci - $ /usr/bin/chmod a+x /dev/vfio - $ /usr/bin/chmod 0666 /dev/vfio/* - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 - $ $DPDK_DIR/tools/dpdk-devbind.py --status - -Setup OVS -~~~~~~~~~ - -Open vSwitch should be started as described in the `installation guide -`__ with the exception of ovs-vswitchd, which requires some -special configuration to enable DPDK functionality. DPDK configuration -arguments can be passed to ovs-vswitchd via the ``other_config`` column of the -``Open_vSwitch`` table. At a minimum, the ``dpdk-init`` option must be set to -``true``. For example:: - - $ export DB_SOCK=/usr/local/var/run/openvswitch/db.sock - $ ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true - $ ovs-vswitchd unix:$DB_SOCK --pidfile --detach - -There are many other configuration options, the most important of which are -listed below. Defaults will be provided for all values not explicitly set. - -``dpdk-init`` - Specifies whether OVS should initialize and support DPDK ports. This is a - boolean, and defaults to false. - -``dpdk-lcore-mask`` - Specifies the CPU cores on which dpdk lcore threads should be spawned and - expects hex string (eg '0x123'). - -``dpdk-socket-mem`` - Comma separated list of memory to pre-allocate from hugepages on specific - sockets. - -``dpdk-hugepage-dir`` - Directory where hugetlbfs is mounted - -``vhost-sock-dir`` - Option to set the path to the vhost-user unix socket files. - -If allocating more than one GB hugepage (as for IVSHMEM), you can configure the -amount of memory used from any given NUMA nodes. For example, to use 1GB from -NUMA node 0, run:: - - $ ovs-vsctl --no-wait set Open_vSwitch . \ - other_config:dpdk-socket-mem="1024,0" - -Similarly, if you wish to better scale the workloads across cores, then -multiple pmd threads can be created and pinned to CPU cores by explicity -specifying ``pmd-cpu-mask``. Cores are numbered from 0, so to spawn two pmd -threads and pin them to cores 1,2, run:: - - $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x6 - -For details on using ivshmem with DPDK, refer to `the advanced installation -guide `__. - -Refer to ovs-vswitchd.conf.db(5) for additional information on configuration -options. - -.. note:: - Changing any of these options requires restarting the ovs-vswitchd - application - -Validating ----------- - -Creating bridges and ports -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can now use ovs-vsctl to set up bridges and other Open vSwitch features. -Bridges should be created with a ``datapath_type=netdev``:: - - $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev - -Now you can add DPDK devices. OVS expects DPDK device names to start with -``dpdk`` and end with a portid. ovs-vswitchd should print the number of dpdk -devices found in the log file:: - - $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk - $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk - -After the DPDK ports get added to switch, a polling thread continuously polls -DPDK devices and consumes 100% of the core, as can be checked from 'top' and -'ps' cmds:: - - $ top -H - $ ps -eLo pid,psr,comm | grep pmd - -Creating bonds of DPDK interfaces is slightly different to creating bonds of -system interfaces. For DPDK, the interface type must be explicitly set. For -example:: - - $ ovs-vsctl add-bond br0 dpdkbond dpdk0 dpdk1 \ - -- set Interface dpdk0 type=dpdk \ - -- set Interface dpdk1 type=dpdk - -To stop ovs-vswitchd & delete bridge, run:: - - $ ovs-appctl -t ovs-vswitchd exit - $ ovs-appctl -t ovsdb-server exit - $ ovs-vsctl del-br br0 - -PMD thread statistics -~~~~~~~~~~~~~~~~~~~~~ - -To show current stats:: - - $ ovs-appctl dpif-netdev/pmd-stats-show - -To clear previous stats:: - - $ ovs-appctl dpif-netdev/pmd-stats-clear - -Port/rxq assigment to PMD threads -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To show port/rxq assignment:: - - $ ovs-appctl dpif-netdev/pmd-rxq-show - -To change default rxq assignment to pmd threads, rxqs may be manually pinned to -desired cores using:: - - $ ovs-vsctl set Interface \ - other_config:pmd-rxq-affinity= - -where: - -- ```` ::= ``NULL`` | ```` -- ```` ::= ```` | - ```` , ```` -- ```` ::= ```` : ```` - -For example:: - - $ ovs-vsctl set interface dpdk0 options:n_rxq=4 \ - other_config:pmd-rxq-affinity="0:3,1:7,3:8" - -This will ensure: - -- Queue #0 pinned to core 3 -- Queue #1 pinned to core 7 -- Queue #2 not pinned -- Queue #3 pinned to core 8 - -After that PMD threads on cores where RX queues was pinned will become -``isolated``. This means that this thread will poll only pinned RX queues. - -.. warning:: - If there are no ``non-isolated`` PMD threads, ``non-pinned`` RX queues will - not be polled. Also, if provided ``core_id`` is not available (ex. this - ``core_id`` not in ``pmd-cpu-mask``), RX queue will not be polled by any PMD - thread. - -.. _dpdk-guest-setup: - -DPDK in the VM --------------- - -DPDK 'testpmd' application can be run in the Guest VM for high speed packet -forwarding between vhostuser ports. DPDK and testpmd application has to be -compiled on the guest VM. Below are the steps for setting up the testpmd -application in the VM. More information on the vhostuser ports can be found in -the `advanced install guide `__. - -.. note:: - Support for DPDK in the guest requires QEMU >= 2.2.0. - -To being, instantiate the guest:: - - $ export VM_NAME=Centos-vm export GUEST_MEM=3072M - $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 - $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch - - $ qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ - -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ - -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ - -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ - -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ - -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ - -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ - -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off \ - -Download the DPDK sourcs to VM and build DPDK:: - - $ cd /root/dpdk/ - $ wget http://fast.dpdk.org/rel/dpdk-16.11.tar.xz - $ tar xf dpdk-16.11.tar.xz - $ export DPDK_DIR=/root/dpdk/dpdk-16.11 - $ export DPDK_TARGET=x86_64-native-linuxapp-gcc - $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET - $ cd $DPDK_DIR - $ make install T=$DPDK_TARGET DESTDIR=install - -Build the test-pmd application:: - - $ cd app/test-pmd - $ export RTE_SDK=$DPDK_DIR - $ export RTE_TARGET=$DPDK_TARGET - $ make - -Setup huge pages and DPDK devices using UIO:: - - $ sysctl vm.nr_hugepages=1024 - $ mkdir -p /dev/hugepages - $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already mounted - $ modprobe uio - $ insmod $DPDK_BUILD/kmod/igb_uio.ko - $ $DPDK_DIR/tools/dpdk-devbind.py --status - $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 - -.. note:: - - vhost ports pci ids can be retrieved using:: - - lspci | grep Ethernet - -Testing -------- - -Below are few testcases and the list of steps to be followed. Before beginning, -ensure a userspace bridge has been created and two DPDK ports added:: - - $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev - $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk - $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk - -PHY-PHY -~~~~~~~ - -Add test flows to forward packets betwen DPDK port 0 and port 1:: - - # Clear current flows - $ ovs-ofctl del-flows br0 - - # Add flows between port 1 (dpdk0) to port 2 (dpdk1) - $ ovs-ofctl add-flow br0 in_port=1,action=output:2 - $ ovs-ofctl add-flow br0 in_port=2,action=output:1 - -Transmit traffic into either port. You should see it returned via the other. - -PHY-VM-PHY (vhost loopback) -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Add two ``dpdkvhostuser`` ports to bridge ``br0``:: - - $ ovs-vsctl add-port br0 dpdkvhostuser0 \ - -- set Interface dpdkvhostuser0 type=dpdkvhostuser - $ ovs-vsctl add-port br0 dpdkvhostuser1 \ - -- set Interface dpdkvhostuser1 type=dpdkvhostuser - -Add test flows to forward packets betwen DPDK devices and VM ports:: - - # Clear current flows - $ ovs-ofctl del-flows br0 - - # Add flows - $ ovs-ofctl add-flow br0 in_port=1,action=output:3 - $ ovs-ofctl add-flow br0 in_port=3,action=output:1 - $ ovs-ofctl add-flow br0 in_port=4,action=output:2 - $ ovs-ofctl add-flow br0 in_port=2,action=output:4 - - # Dump flows - $ ovs-ofctl dump-flows br0 - -Create a VM using the following configuration: - -+----------------------+--------+-----------------+ -| configuration | values | comments | -+----------------------+--------+-----------------+ -| qemu version | 2.2.0 | n/a | -| qemu thread affinity | core 5 | taskset 0x20 | -| memory | 4GB | n/a | -| cores | 2 | n/a | -| Qcow2 image | CentOS7| n/a | -| mrg_rxbuf | off | n/a | -+----------------------+--------+-----------------+ - -You can do this directly with QEMU via the ``qemu-system-x86_64`` -application:: - - $ export VM_NAME=vhost-vm - $ export GUEST_MEM=3072M - $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 - $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch - - $ taskset 0x20 qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ - -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ - -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ - -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ - -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ - -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ - -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ - -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off - -Alternatively, you can configure the guest using libvirt. Below is an XML -configuration for a 'demovm' guest that can be instantiated using `virsh`:: - - - demovm - 4a9b3f53-fa2a-47f3-a757-dd87720d9d1d - 4194304 - 4194304 - - - - - - 2 - - 4096 - - - - - - hvm - - - - - - - - - - - - - - destroy - restart - destroy - - /usr/bin/qemu-kvm - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Once the guest is configured and booted, configure DPDK packet forwarding -within the guest. To accomplish this, DPDK and testpmd application have to -be first compiled on the VM as described in **Guest Setup**. Once compiled, run -the ``test-pmd`` application:: - - $ cd $DPDK_DIR/app/test-pmd; - $ ./testpmd -c 0x3 -n 4 --socket-mem 1024 -- \ - --burst=64 -i --txqflags=0xf00 --disable-hw-vlan - $ set fwd mac retry - $ start - -When you finish testing, bind the vNICs back to kernel:: - - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 - -.. note:: - Appropriate PCI IDs to be passed in above example. The PCI IDs can be - retrieved like so:: - - $ $DPDK_DIR/tools/dpdk-devbind.py --status - -.. note:: - More information on the dpdkvhostuser ports can be found in the `advanced - installation guide `__. - -PHY-VM-PHY (IVSHMEM loopback) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Refer to the `advanced installation guide `__. - -Limitations ------------- - -- Currently DPDK ports does not use HW offload functionality. -- Network Interface Firmware requirements: Each release of DPDK is - validated against a specific firmware version for a supported Network - Interface. New firmware versions introduce bug fixes, performance - improvements and new functionality that DPDK leverages. The validated - firmware versions are available as part of the release notes for - DPDK. It is recommended that users update Network Interface firmware - to match what has been validated for the DPDK release. - - The latest list of validated firmware versions can be found in the `DPDK - release notes`_. - -.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_16_11.html - -Bug Reporting -------------- - -Please report problems to bugs@openvswitch.org. diff --git a/INSTALL.Debian.rst b/INSTALL.Debian.rst deleted file mode 100644 index 6086b827b..000000000 --- a/INSTALL.Debian.rst +++ /dev/null @@ -1,123 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -========================================= -Building Debian Packages for Open vSwitch -========================================= - -This document describes how to build Debian packages for Open vSwitch. To -install Open vSwitch on Debian without building Debian packages, see the -`installation guide `__ instead. - -.. note:: - These instructions should also work on Ubuntu and other Debian derivative - distributions. - -Before You Begin ----------------- - -Before you begin, consider whether you really need to build packages yourself. -Debian "wheezy" and "sid", as well as recent versions of Ubuntu, contain -pre-built Debian packages for Open vSwitch. It is easier to install these than -to build your own. To use packages from your distribution, skip ahead to -"Installing .deb Packages", below. - -Building Open vSwitch Debian packages -------------------------------------- - -You may build from an Open vSwitch distribution tarball or from an Open vSwitch -Git tree with these instructions. - -You do not need to be the superuser to build the Debian packages. - -1. Install the "build-essential" and "fakeroot" packages. For example:: - - $ apt-get install build-essential fakeroot - -2. Obtain and unpack an Open vSwitch source distribution and ``cd`` into its - top level directory. - -3. Install the build dependencies listed under "Build-Depends:" near the top of - ``debian/control``. You can install these any way you like, e.g. with - ``apt-get install``. - -Check your work by running ``dpkg-checkbuilddeps`` in the top level of your ovs -directory. If you've installed all the dependencies properly, -``dpkg-checkbuilddeps`` will exit without printing anything. If you forgot to -install some dependencies, it will tell you which ones. - -4. Build the package:: - - $ fakeroot debian/rules binary - - This will do a serial build that runs the unit tests. This will take - approximately 8 to 10 minutes. If you prefer, you can run a faster parallel - build:: - - $ DEB_BUILD_OPTIONS='parallel=8' fakeroot debian/rules binary - - If you are in a big hurry, you can even skip the unit tests:: - - $ DEB_BUILD_OPTIONS='parallel=8 nocheck' fakeroot debian/rules binary - -.. note:: - - There are a few pitfalls in the Debian packaging building system so that, - occasionally, you may find that in a tree that you have using for a while, - the build command above exits immediately without actually building anything. - To fix the problem, run:: - - $ fakeroot debian/rules clean - - or start over from a fresh copy of the source tree. - -5. The generated .deb files will be in the parent directory of the Open vSwitch - source distribution. - -Installing .deb Packages ------------------------- - -These instructions apply to installing from Debian packages that you built -yourself, as described in the previous section, or from packages provided by -Debian or a Debian derivative distribution such as Ubuntu. In the former case, -use a command such as ``dpkg -i`` to install the .deb files that you build, and -in the latter case use a program such as ``apt-get`` or ``aptitude`` to -download and install the provided packages. - -.. important:: - You must be superuser to install Debian packages. - -1. Start by installing an Open vSwitch kernel module. See - ``debian/openvswitch-switch.README.Debian`` for the available options. - -2. Install the ``openvswitch-switch`` and ``openvswitch-common`` packages. - These packages include the core userspace components of the switch. - -Open vSwitch ``.deb`` packages not mentioned above are rarely useful. Refer to -their individual package descriptions to find out whether any of them are -useful to you. - -Bug Reporting -------------- - -Please report problems to bugs@openvswitch.org. diff --git a/INSTALL.Docker.rst b/INSTALL.Docker.rst deleted file mode 100644 index 4d661720e..000000000 --- a/INSTALL.Docker.rst +++ /dev/null @@ -1,320 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -=================================== -Open Virtual Networking With Docker -=================================== - -This document describes how to use Open Virtual Networking with Docker 1.9.0 -or later. - -.. important:: - - Requires Docker version 1.9.0 or later. Only Docker 1.9.0+ comes with support - for multi-host networking. Consult www.docker.com for instructions on how to - install Docker. - -.. note:: - - You must build and install Open vSwitch before proceeding with the below - guide. Refer to the `installation guide `__ for more - information. - -Setup ------ - -For multi-host networking with OVN and Docker, Docker has to be started with a -destributed key-value store. For example, if you decide to use consul as your -distributed key-value store and your host IP address is ``$HOST_IP``, start -your Docker daemon with:: - - $ docker daemon --cluster-store=consul://127.0.0.1:8500 \ - --cluster-advertise=$HOST_IP:0 - -OVN provides network virtualization to containers. OVN's integration with -Docker currently works in two modes - the "underlay" mode or the "overlay" -mode. - -In the "underlay" mode, OVN requires a OpenStack setup to provide container -networking. In this mode, one can create logical networks and can have -containers running inside VMs, standalone VMs (without having any containers -running inside them) and physical machines connected to the same logical -network. This is a multi-tenant, multi-host solution. - -In the "overlay" mode, OVN can create a logical network amongst containers -running on multiple hosts. This is a single-tenant (extendable to multi-tenants -depending on the security characteristics of the workloads), multi-host -solution. In this mode, you do not need a pre-created OpenStack setup. - -For both the modes to work, a user has to install and start Open vSwitch in -each VM/host that they plan to run their containers on. - -.. _docker-overlay: - -The "overlay" mode ------------------- - -.. note:: - - OVN in "overlay" mode needs a minimum Open vSwitch version of 2.5. - -1. Start the central components. - - OVN architecture has a central component which stores your networking intent - in a database. On one of your machines, with an IP Address of - ``$CENTRAL_IP``, where you have installed and started Open vSwitch, you will - need to start some central components. - - Start ovn-northd daemon. This daemon translates networking intent from Docker - stored in the OVN\_Northbound database to logical flows in ``OVN_Southbound`` - database. For example:: - - $ /usr/share/openvswitch/scripts/ovn-ctl start_northd - -2. One time setup - - On each host, where you plan to spawn your containers, you will need to run - the below command once. You may need to run it again if your OVS database - gets cleared. It is harmless to run it again in any case:: - - $ ovs-vsctl set Open_vSwitch . \ - external_ids:ovn-remote="tcp:$CENTRAL_IP:6642" \ - external_ids:ovn-nb="tcp:$CENTRAL_IP:6641" \ - external_ids:ovn-encap-ip=$LOCAL_IP \ - external_ids:ovn-encap-type="$ENCAP_TYPE" - - where: - - ``$LOCAL_IP`` - is the IP address via which other hosts can reach this host. This acts as - your local tunnel endpoint. - - ``$ENCAP_TYPE`` - is the type of tunnel that you would like to use for overlay networking. - The options are ``geneve`` or ``stt``. Your kernel must have support for - your chosen ``$ENCAP_TYPE``. Both ``geneve`` and ``stt`` are part of the - Open vSwitch kernel module that is compiled from this repo. If you use the - Open vSwitch kernel module from upstream Linux, you will need a minumum - kernel version of 3.18 for ``geneve``. There is no ``stt`` support in - upstream Linux. You can verify whether you have the support in your kernel - as follows:: - - $ lsmod | grep $ENCAP_TYPE - - In addition, each Open vSwitch instance in an OVN deployment needs a unique, - persistent identifier, called the ``system-id``. If you install OVS from - distribution packaging for Open vSwitch (e.g. .deb or .rpm packages), or if - you use the ovs-ctl utility included with Open vSwitch, it automatically - configures a system-id. If you start Open vSwitch manually, you should set - one up yourself. For example:: - - $ id_file=/etc/openvswitch/system-id.conf - $ test -e $id_file || uuidgen > $id_file - $ ovs-vsctl set Open_vSwitch . external_ids:system-id=$(cat $id_file) - -3. Start the ``ovn-controller``. - - You need to run the below command on every boot:: - - $ /usr/share/openvswitch/scripts/ovn-ctl start_controller - -4. Start the Open vSwitch network driver. - - By default Docker uses Linux bridge for networking. But it has support for - external drivers. To use Open vSwitch instead of the Linux bridge, you will - need to start the Open vSwitch driver. - - The Open vSwitch driver uses the Python's flask module to listen to Docker's - networking api calls. So, if your host does not have Python's flask module, - install it:: - - $ sudo pip install Flask - - Start the Open vSwitch driver on every host where you plan to create your - containers. Refer to the note on ``$OVS_PYTHON_LIBS_PATH`` that is used below - at the end of this document:: - - $ PYTHONPATH=$OVS_PYTHON_LIBS_PATH ovn-docker-overlay-driver --detach - - .. note:: - - The ``$OVS_PYTHON_LIBS_PATH`` variable should point to the directory where - Open vSwitch Python modules are installed. If you installed Open vSwitch - Python modules via the Debian package of ``python-openvswitch`` or via pip - by running ``pip install ovs``, you do not need to specify the PATH. If - you installed it by following the instructions in the `installation guide - `__, then you should specify the PATH. In this case, the PATH - depends on the options passed to ``./configure``. It is usually either - ``/usr/share/openvswitch/python`` or - ``/usr/local/share/openvswitch/python`` - -Docker has inbuilt primitives that closely match OVN's logical switches and -logical port concepts. Consult Docker's documentation for all the possible -commands. Here are some examples. - -Create a logical switch -~~~~~~~~~~~~~~~~~~~~~~~ - -To create a logical switch with name 'foo', on subnet '192.168.1.0/24', run:: - - $ NID=`docker network create -d openvswitch --subnet=192.168.1.0/24 foo` - -List all logical switches -~~~~~~~~~~~~~~~~~~~~~~~~~ - -:: - - $ docker network ls - -You can also look at this logical switch in OVN's northbound database by -running the following command:: - - $ ovn-nbctl --db=tcp:$CENTRAL_IP:6640 ls-list - -Delete a logical switch -~~~~~~~~~~~~~~~~~~~~~~~ - -:: - - $ docker network rm bar - - -Create a logical port -~~~~~~~~~~~~~~~~~~~~~ - -Docker creates your logical port and attaches it to the logical network in a -single step. For example, to attach a logical port to network ``foo`` inside -container busybox, run:: - - $ docker run -itd --net=foo --name=busybox busybox - -List all logical ports -~~~~~~~~~~~~~~~~~~~~~~ - -Docker does not currently have a CLI command to list all logical ports but you -can look at them in the OVN database by running:: - - $ ovn-nbctl --db=tcp:$CENTRAL_IP:6640 lsp-list $NID - -Create and attach a logical port to a running container -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -:: - - $ docker network create -d openvswitch --subnet=192.168.2.0/24 bar - $ docker network connect bar busybox - -Detach and delete a logical port from a running container -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can delete your logical port and detach it from a running container -by running: - -:: - - $ docker network disconnect bar busybox - -.. _docker-underlay: - -The "underlay" mode -------------------- - -.. note:: - - This mode requires that you have a OpenStack setup pre-installed with - OVN providing the underlay networking. - -1. One time setup - - A OpenStack tenant creates a VM with a single network interface (or multiple) - that belongs to management logical networks. The tenant needs to fetch the - port-id associated with the interface via which he plans to send the container - traffic inside the spawned VM. This can be obtained by running the below - command to fetch the 'id' associated with the VM:: - - $ nova list - - and then by running:: - - $ neutron port-list --device_id=$id - - Inside the VM, download the OpenStack RC file that contains the tenant - information (henceforth referred to as ``openrc.sh``). Edit the file and add the - previously obtained port-id information to the file by appending the following - line:: - - $ export OS_VIF_ID=$port_id - - After this edit, the file will look something like:: - - #!/bin/bash - export OS_AUTH_URL=http://10.33.75.122:5000/v2.0 - export OS_TENANT_ID=fab106b215d943c3bad519492278443d - export OS_TENANT_NAME="demo" - export OS_USERNAME="demo" - export OS_VIF_ID=e798c371-85f4-4f2d-ad65-d09dd1d3c1c9 - -2. Create the Open vSwitch bridge - - If your VM has one ethernet interface (e.g.: 'eth0'), you will need to add - that device as a port to an Open vSwitch bridge 'breth0' and move its IP - address and route related information to that bridge. (If it has multiple - network interfaces, you will need to create and attach an Open vSwitch - bridge for the interface via which you plan to send your container - traffic.) - - If you use DHCP to obtain an IP address, then you should kill the DHCP - client that was listening on the physical Ethernet interface (e.g. eth0) and - start one listening on the Open vSwitch bridge (e.g. breth0). - - Depending on your VM, you can make the above step persistent across reboots. - For example, if your VM is Debian/Ubuntu-based, read - `openvswitch-switch.README.Debian` found in `debian` folder. If your VM is - RHEL-based, refer to the `RHEL install guide <../../INSTALL.RHEL.rst>`__. - -3. Start the Open vSwitch network driver - - The Open vSwitch driver uses the Python's flask module to listen to Docker's - networking api calls. The driver also uses OpenStack's - ``python-neutronclient`` libraries. If your host does not have Python's - ``flask`` module or ``python-neutronclient`` you must install them. For - example:: - - $ pip install python-neutronclient - $ pip install Flask - - Once installed, source the ``openrc`` file:: - - $ . ./openrc.sh - - Start the network driver and provide your OpenStack tenant password when - prompted:: - - $ PYTHONPATH=$OVS_PYTHON_LIBS_PATH ovn-docker-underlay-driver \ - --bridge breth0 --detach - -From here-on you can use the same Docker commands as described in -`docker-overlay`_. - -Refer the the ovs-architecture man pages (``man ovn-architecture``) to -understand OVN's architecture in detail. diff --git a/INSTALL.Fedora.rst b/INSTALL.Fedora.rst deleted file mode 100644 index b9be0edfe..000000000 --- a/INSTALL.Fedora.rst +++ /dev/null @@ -1,144 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -============================ -Open vSwitch on Fedora Linux -============================ - -This document provides instructions for building and installing Open vSwitch -RPM packages on a Fedora Linux host. Instructions for the installation of Open -vSwitch on a Fedora Linux host without using RPM packages can be found in the -`general installation guide `__. - -These instructions have been tested with Fedora 23, and are also applicable for -RHEL 7.x and its derivatives, including CentOS 7.x and Scientific Linux 7.x. - -Build Requirements ------------------- - -To build packages for a Fedora Linux host, you will need the packages described -in the `general installation guide `__. Specific packages (by -package name) include: - -- rpm-build -- autoconf automake libtool -- systemd-units openssl openssl-devel -- python python-twisted-core python-zope-interface python-six -- desktop-file-utils -- groff graphviz -- procps-ng - -And (optionally): - -- libcap-ng libcap-ng-devel -- dpdk-devel - -Bootstraping ------------- - -Refer to the *Bootstrapping* section of the `general installation guide -`__. - -Configuring ------------ - -Refer to the *Configuring* section of the `general installation guide -`__. - -Building --------- - -User Space RPMs -~~~~~~~~~~~~~~~ - -To build Open vSwitch user-space RPMs, execute the following from the directory -in which `./configure` was executed: - -:: - - $ make rpm-fedora - -This will create the RPMs `openvswitch`, `python-openvswitch`, -`openvswitch-test`, `openvswitch-devel`, `openvswitch-ovn-common`, -`openvswitch-ovn-central`, `openvswitch-ovn-host`, `openvswitch-ovn-vtep`, -`openvswitch-ovn-docker`, and `openvswitch-debuginfo`. - -To enable DPDK support in the openvswitch package, the ``--with dpdk`` option -can be added: - -:: - - $ make rpm-fedora RPMBUILD_OPT="--with dpdk" - -The above commands automatically run the Open vSwitch unit tests, which can -take several minutes. To reduce the build time by disabling the execution of -these tests, the ``--without check`` option can be added: - -:: - - $ make rpm-fedora RPMBUILD_OPT="--without check" - -Kernel OVS Tree Datapath RPM -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To build the Open vSwitch kernel module for the currently running kernel -version, run: - -:: - - $ make rpm-fedora-kmod - -To build the Open vSwitch kernel module for another kernel version, the desired -kernel version can be specified via the `kversion` macro. For example: - -:: - - $ make rpm-fedora-kmod \ - RPMBUILD_OPT='-D "kversion 4.3.4-300.fc23.x86_64"' - -Installing ----------- - -RPM packages can be installed by using the command ``rpm -i``. Package -installation requires superuser privileges. - -The `openvswitch-kmod` RPM should be installed first if the Linux OVS tree -datapath module is to be used. The `openvswitch-kmod` RPM should not be -installed if only the in-tree Linux datapath or user-space datapath is needed. -See the `FAQ document `__ for more information about the various Open -vSwitch datapath options. - -In most cases only the `openvswitch` RPM will need to be installed. The -`python-openvswitch`, `openvswitch-test`, `openvswitch-devel`, and -`openvswitch-debuginfo` RPMs are optional unless required for a specific -purpose. - -The `openvswitch-ovn-*` packages are only needed when using OVN. - -Refer to the `RHEL README `__ for additional usage and -configuration information. - -Reporting Bugs --------------- - -Report problems to bugs@openvswitch.org. diff --git a/INSTALL.KVM.rst b/INSTALL.KVM.rst deleted file mode 100644 index 4a5b372e6..000000000 --- a/INSTALL.KVM.rst +++ /dev/null @@ -1,104 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -================================ -How to Use Open vSwitch with KVM -================================ - -This document describes how to use Open vSwitch with the Kernel-based Virtual -Machine (KVM). - -.. note:: - - This document assumes that you have read and followed the `installation guide - `__ to get Open vSwitch setup on your Linux system. - -Setup ------ - -KVM uses tunctl to handle various bridging modes, which you can install with -the Debian/Ubuntu package ``uml-utilities``:: - - $ apt-get install uml-utilities - -Next, you will need to modify or create custom versions of the ``qemu-ifup`` -and ``qemu-ifdown`` scripts. In this guide, we'll create custom versions that -make use of example Open vSwitch bridges that we'll describe in this guide. - -Create the following two files and store them in known locations. For example:: - - echo << EOF > /etc/ovs-ifup - #!/bin/sh - - switch='br0' - /sbin/ifconfig $1 0.0.0.0 up - ovs-vsctl add-port ${switch} $1 - EOF - -:: - - echo << EOF > /etc/ovs-ifdown - #!/bin/sh - - switch='br0' - /sbin/ifconfig $1 0.0.0.0 down - ovs-vsctl del-port ${switch} $1 - -The basic usage of Open vSwitch is described at the end of the `installation -guide `__. If you haven't already, create a bridge named ``br0`` -with the following command:: - - $ ovs-vsctl add-br br0 - -Then, add a port to the bridge for the NIC that you want your guests to -communicate over (e.g. ``eth0``):: - - $ ovs-vsctl add-port br0 eth0 - -Refer to ovs-vsctl(8) for more details. - -Next, we'll start a guest that will use our ifup and ifdown scripts:: - - $ kvm -m 512 -net nic,macaddr=00:11:22:EE:EE:EE -net \ - tap,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown -drive \ - file=/path/to/disk-image,boot=on - -This will start the guest and associate a tap device with it. The ``ovs-ifup`` -script will add a port on the br0 bridge so that the guest will be able to -communicate over that bridge. - -To get some more information and for debugging you can use Open vSwitch -utilities such as ovs-dpctl and ovs-ofctl, For example:: - - $ ovs-dpctl show - $ ovs-ofctl show br0 - -You should see tap devices for each KVM guest added as ports to the bridge -(e.g. tap0) - -Refer to ovs-dpctl(8) and ovs-ofctl(8) for more details. - -Bug Reporting -------------- - -Please report problems to bugs@openvswitch.org. diff --git a/INSTALL.Libvirt.rst b/INSTALL.Libvirt.rst deleted file mode 100644 index b32944cf8..000000000 --- a/INSTALL.Libvirt.rst +++ /dev/null @@ -1,93 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -========================= -Open vSwitch with Libvirt -========================= - -This document describes how to use Open vSwitch with Libvirt 0.9.11 or later. -This document assumes that you followed the `general installation guide -`__ or installed Open vSwitch from distribution packaging such as -a .deb or .rpm. The Open vSwitch support is included by default in Libvirt -0.9.11. Consult www.libvirt.org for instructions on how to build the latest -Libvirt, if your Linux distribution by default comes with an older Libvirt -release. - -Limitations ------------ - -Currently there is no Open vSwitch support for networks that are managed by -libvirt (e.g. NAT). As of now, only bridged networks are supported (those where -the user has to manually create the bridge). - -Setup ------ - -First, create the Open vSwitch bridge by using the ovs-vsctl utility (this must -be done with administrative privileges):: - - $ ovs-vsctl add-br ovsbr - -Once that is done, create a VM, if necessary, and edit its Domain XML file:: - - $ virsh edit - -Lookup in the Domain XML file the ```` section. There should be one -such XML section for each interface the VM has:: - - - - -
- - -And change it to something like this:: - - - - - -
- - -The interface type must be set to ``bridge``. The ```` XML element -specifies to which bridge this interface will be attached to. The -```` element indicates that the bridge in ```` element is -an Open vSwitch bridge. - -Then (re)start the VM and verify if the guest's vnet interface is attached to -the ovsbr bridge:: - - $ ovs-vsctl show - -Troubleshooting ---------------- - -If the VM does not want to start, then try to run the libvirtd process either -from the terminal, so that all errors are printed in console, or inspect -Libvirt/Open vSwitch log files for possible root cause. - -Bug Reporting -------------- - -Report problems to bugs@openvswitch.org. diff --git a/INSTALL.NetBSD.rst b/INSTALL.NetBSD.rst deleted file mode 100644 index e204c57d4..000000000 --- a/INSTALL.NetBSD.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -====================== -Open vSwitch on NetBSD -====================== - -On NetBSD, you might want to install requirements from pkgsrc. In that case, -you need at least the following packages. - -- automake -- libtool-base -- gmake -- python27 -- py27-six -- py27-xml -- pkg_alternatives - -Some components have additional requirements. Refer to the `general -installation guide `__. - -Assuming you are running NetBSD/amd64 6.1.2, you can download and install -pre-built binary packages as the following:: - - $ PKG_PATH=http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/6.1.2/All/ - $ export PKG_PATH - $ pkg_add automake libtool-base gmake python27 py27-six py27-xml \ - pkg_alternatives - -.. note:: - You might get some warnings about minor version mismatch. These can be safely - ignored. - -NetBSD's ``/usr/bin/make`` is not GNU make. GNU make is installed as -``/usr/pkg/bin/gmake`` by the above mentioned ``gmake`` package. - -As all executables installed with pkgsrc are placed in ``/usr/pkg/bin/`` -directory, it might be a good idea to add it to your PATH. - -Open vSwitch on NetBSD is currently "userspace switch" implementation in the -sense described in the `userspace installation guide `__ -and `porting guide `__. diff --git a/INSTALL.RHEL.rst b/INSTALL.RHEL.rst deleted file mode 100644 index 125c55de1..000000000 --- a/INSTALL.RHEL.rst +++ /dev/null @@ -1,208 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -======================================== -Open vSwitch on Red Hat Enterprise Linux -======================================== - -This document describes how to build and install Open vSwitch on a Red Hat -Enterprise Linux (RHEL) host. If you want to install Open vSwitch on a generic -Linux host, see the `general installation guide `__ instead. - -We have tested these instructions with RHEL 5.6 and RHEL 6.0. - -For RHEL 7.x (or derivatives, such as CentOS 7.x), you should follow the -instructions in the `Fedora installation guide `__. The -Fedora spec files are used for RHEL 7.x. - -Prerequisites -------------- - -You may build from an Open vSwitch distribution tarball or from an Open vSwitch -Git tree. - -The default RPM build directory, ``_topdir``, has five directories in the -top-level. - -BUILD/ - where the software is unpacked and built -RPMS/ - where the newly created binary package files are written -SOURCES/ - contains the original sources, patches, and icon files -SPECS/ - contains the spec files for each package to be built -SRPMS/ - where the newly created source package files are written - -Before you begin, note the RPM sources directory on your version of RHEL. The -command ``rpmbuild --showrc`` will show the configuration for each of those -directories. Alternatively, the command ``rpm --eval '%{_topdir}'`` shows the -current configuration for the top level directory and the command ``rpm --eval -'%{_sourcedir}'`` does the same for the sources directory. On RHEL 5, the -default RPM ``_topdir`` is ``/usr/src/redhat`` and the default RPM sources -directory is ``/usr/src/redhat/SOURCES``. On RHEL 6, the default ``_topdir`` is -``$HOME/rpmbuild`` and the default RPM sources directory is -``$HOME/rpmbuild/SOURCES``. - -Build Requirements ------------------- - -To compile the RPMs, you will need to install the packages described in the -`general installation guide `__ along with some additional -packages. These can be installed with the below command:: - - $ yum install gcc make python-devel openssl-devel kernel-devel graphviz \ - kernel-debug-devel autoconf automake rpm-build redhat-rpm-config \ - libtool - -Bootstrapping and Configuring ------------------------------ - -If you are building from a distribution tarball, skip to *Building*. If not, -you must be building from an Open vSwitch Git tree. Determine what version of -Autoconf is installed (e.g. run ``autoconf --version``). If it is not at least -version 2.63, then you must upgrade or use another machine to build the -packages. - -Assuming all requirements have been met, build the tarball by running:: - - $ ./boot.sh - $ ./configure - $ make dist - -You must run this on a machine that has the tools listed in the `general -installation guide `__ as prerequisites for building from a Git -tree. Afterward, proceed with the rest of the instructions using the -distribution tarball. - -Now you have a distribution tarball, named something like -``openvswitch-x.y.z.tar.gz``. Copy this file into the RPM sources directory, -e.g.:: - - $ cp openvswitch-x.y.z.tar.gz $HOME/rpmbuild/SOURCES - -Broken ``build`` symlink -~~~~~~~~~~~~~~~~~~~~~~~~ - -Some versions of the RHEL 6 kernel-devel package contain a broken ``build`` -symlink. If you are using such a version, you must fix the problem before -continuing. - -To find out whether you are affected, run:: - - $ cd /lib/modules/ - $ ls -l build/ - -where ```` is the version number of the RHEL 6 kernel. - -.. note:: - The trailing slash in the final command is important. Be sure to include - it. - -If the ``ls`` command produces a directory listing, your kernel-devel package -is OK. If it produces a ``No such file or directory`` error, your kernel-devel -package is buggy. - -If your kernel-devel package is buggy, then you can fix it with:: - - $ cd /lib/modules/ - $ rm build - $ ln -s /usr/src/kernels/ build - -where ```` is the name of an existing directory under -``/usr/src/kernels``, whose name should be similar to ```` but may -contain some extra parts. Once you have done this, verify the fix with the -same procedure you used above to check for the problem. - -Building --------- - -You should have a distribution tarball named something like -openvswitch-x.y.z.tar.gz. Copy this file into the RPM sources directory:: - - $ cp openvswitch-x.y.z.tar.gz $HOME/rpmbuild/SOURCES - -Make another copy of the distribution tarball in a temporary directory. Then -unpack the tarball and ``cd`` into its root:: - - $ tar xzf openvswitch-x.y.z.tar.gz - $ cd openvswitch-x.y.z - -Userspace -~~~~~~~~~ - -To build Open vSwitch userspace, run:: - - $ rpmbuild -bb rhel/openvswitch.spec - -This produces two RPMs: "openvswitch" and "openvswitch-debuginfo". - -The above command automatically runs the Open vSwitch unit tests. To disable -the unit tests, run:: - - $ rpmbuild -bb --without check rhel/openvswitch.spec - -.. note:: - If the build fails with ``configure: error: source dir - /lib/modules/2.6.32-279.el6.x86_64/build doesn't exist`` or similar, then - the kernel-devel package is missing or buggy. - -Kernel Module -~~~~~~~~~~~~~ - -On RHEL 6, to build the Open vSwitch kernel module, copy -rhel/openvswitch-kmod.files into the RPM sources directory and run:: - - $ rpmbuild -bb rhel/openvswitch-kmod-rhel6.spec - -You might have to specify a kernel version and/or variants, e.g.: - - $ rpmbuild -bb \ - -D "kversion 2.6.32-131.6.1.el6.x86_64" \ - -D "kflavors default debug kdump" \ - rhel/openvswitch-kmod-rhel6.spec - -This produces an "kmod-openvswitch" RPM for each kernel variant, in this -example: "kmod-openvswitch", "kmod-openvswitch-debug", and -"kmod-openvswitch-kdump". - -Red Hat Network Scripts Integration ------------------------------------ - -A RHEL host has default firewall rules that prevent any Open vSwitch tunnel -traffic from passing through. If a user configures Open vSwitch tunnels like -Geneve, GRE, VXLAN, LISP etc., they will either have to manually add iptables -firewall rules to allow the tunnel traffic or add it through a startup script -Refer to the "enable-protocol" command in the ovs-ctl(8) manpage for more -information. - -In addition, simple integration with Red Hat network scripts has been -implemented. Refer to `README.RHEL.rst `__ in the source -tree or /usr/share/doc/openvswitch/README.RHEL.rst in the installed openvswitch -package for details. - -Reporting Bugs --------------- - -Report problems to bugs@openvswitch.org. diff --git a/INSTALL.SELinux.rst b/INSTALL.SELinux.rst deleted file mode 100644 index 8593d7f5a..000000000 --- a/INSTALL.SELinux.rst +++ /dev/null @@ -1,174 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -========================= -Open vSwitch with SELinux -========================= - -Security-Enhanced Linux (SELinux) is a Linux kernel security module that limits -"the malicious things" that certain processes, including OVS, can do to the -system in case they get compromised. In our case SELinux basically serves as -the "second line of defense" that limits the things that OVS processes are -allowed to do. The "first line of defense" is proper input validation that -eliminates code paths that could be used by attacker to do any sort of "escape -attacks", such as file name escape, shell escape, command line argument escape, -buffer escape. Since developers don't always implement proper input validation, -then SELinux Access Control's goal is to confine damage of such attacks, if -they turned out to be possible. - -Besides Type Enforcement there are other SELinux features, but they are out of -scope for this document. - -Currently there are two SELinux policies for Open vSwitch: - -- the one that ships with your Linux distribution (i.e. - selinux-policy-targeted package) - -- the one that ships with OVS (i.e. openvswitch-selinux-policy package) - -Limitations ------------ - -If Open vSwitch is directly started from command line, then it will run under -``unconfined_t`` SELinux domain that basically lets daemon to do whatever it -likes. This is very important for developers to understand, because they might -introduced code in OVS that invokes new system calls that SELinux policy did -not anticipate. This means that their feature may have worked out just fine -for them. However, if someone else would try to run the same code when Open -vSwitch is started through systemctl, then Open vSwitch would get Permission -Denied errors. - -Currently the only distributions that enforce SELinux on OVS by default are -RHEL, CentOS and Fedora. While Ubuntu and Debian also have some SELinux -support, they run Open vSwitch under the unrestricted ``unconfined`` domain. -Also, it seems that Ubuntu is leaning towards Apparmor that works slightly -differently than SELinux. - -SELinux and Open vSwitch are moving targets. What this means is that, if you -solely rely on your Linux distribution's SELinux policy, then this policy might -not have correctly anticipated that a newer Open vSwitch version needs extra -white list rules. However, if you solely rely on SELinux policy that ships -with Open vSwitch, then Open vSwitch developers might not have correctly -anticipated the feature set that your SELinux implementation supports. - -Installation ------------- - -Refer to the `Fedora installation guide `__ for -instructions on how to build all Open vSwitch rpm packages. - -Once the package is built, install it on your Linux distribution:: - - $ dnf install openvswitch-selinux-policy-2.4.1-1.el7.centos.noarch.rpm - -Restart Open vSwitch:: - - $ systemctl restart openvswitch - -Troubleshooting ---------------- - -When SELinux was implemented some of the standard system utilities acquired -``-Z`` flag (e.g. ``ps -Z``, ``ls -Z``). For example, to find out under which -SELinux security domain process runs, use:: - - $ ps -AZ | grep ovs-vswitchd - system_u:system_r:openvswitch_t:s0 854 ? ovs-vswitchd - -To find out the SELinux label of file or directory, use:: - - $ ls -Z /etc/openvswitch/conf.db - system_u:object_r:openvswitch_rw_t:s0 /etc/openvswitch/conf.db - -If, for example, SELinux policy for Open vSwitch is too strict, then you might -see in Open vSwitch log files "Permission Denied" errors:: - - $ cat /var/log/openvswitch/ovs-vswitchd.log - vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log - ovs_numa|INFO|Discovered 2 CPU cores on NUMA node 0 - ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU cores - reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting... - reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected - netlink_socket|ERR|fcntl: Permission denied - dpif_netlink|ERR|Generic Netlink family 'ovs_datapath' does not exist. - The Open vSwitch kernel module is probably not loaded. - dpif|WARN|failed to enumerate system datapaths: Permission denied - dpif|WARN|failed to create datapath ovs-system: Permission denied - -However, not all "Permission denied" errors are caused by SELinux. So, before -blaming too strict SELinux policy, make sure that indeed SELinux was the one -that denied OVS access to certain resources, for example, run: - - $ grep "openvswitch_t" /var/log/audit/audit.log | tail - type=AVC msg=audit(1453235431.640:114671): avc: denied { getopt } for pid=4583 comm="ovs-vswitchd" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_generic_socket permissive=0 - -If SELinux denied OVS access to certain resources, then make sure that you have -installed our SELinux policy package that "loosens" up distribution's SELinux -policy:: - - $ rpm -qa | grep openvswitch-selinux - openvswitch-selinux-policy-2.4.1-1.el7.centos.noarch - -Then verify that this module was indeed loaded:: - - # semodule -l | grep openvswitch - openvswitch-custom 1.0 - openvswitch 1.1.1 - -If you still see Permission denied errors, then take a look into -``selinux/openvswitch.te`` file in the OVS source tree and try to add white -list rules. This is really simple, just run SELinux audit2allow tool:: - - $ grep "openvswitch_t" /var/log/audit/audit.log | audit2allow -M ovslocal - -Contributing SELinux policy patches ------------------------------------ - -Here are few things to consider before proposing SELinux policy patches to Open -vSwitch developer mailing list: - -1. The SELinux policy that resides in Open vSwitch source tree amends SELinux - policy that ships with your distributions. - - Implications of this are that it is assumed that the distribution's Open - vSwitch SELinux module must be already loaded to satisfy dependencies. - -2. The SELinux policy that resides in Open vSwitch source tree must work on all - currently relevant Linux distributions. - - Implications of this are that you should use only those SELinux policy - features that are supported by the lowest SELinux version out there. - Typically this means that you should test your SELinux policy changes on the - oldest RHEL or CentOS version that this OVS version supports. Check - INSTALL.Fedora.rst file to find out this. - -3. The SELinux policy is enforced only when state transition to - ``openvswitch_t`` domain happens. - - Implications of this are that perhaps instead of loosening SELinux policy - you can do certain things at the time rpm package is installed. - -Reporting Bugs --------------- - -Report problems to bugs@openvswitch.org. diff --git a/INSTALL.SSL.rst b/INSTALL.SSL.rst deleted file mode 100644 index 8edd25ec9..000000000 --- a/INSTALL.SSL.rst +++ /dev/null @@ -1,338 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -===================== -Open vSwitch with SSL -===================== - -If you plan to configure Open vSwitch to connect across the network to an -OpenFlow controller, then we recommend that you build Open vSwitch with -OpenSSL. SSL support ensures integrity and confidentiality of the OpenFlow -connections, increasing network security. - -This document describes how to configure an Open vSwitch to connect to an -OpenFlow controller over SSL. Refer to the `general installation guide -`__ for instructions on building Open vSwitch with SSL support. - -Open vSwitch uses TLS version 1.0 or later (TLSv1), as specified by RFC 2246, -which is very similar to SSL version 3.0. TLSv1 was released in January 1999, -so all current software and hardware should implement it. - -This document assumes basic familiarity with public-key cryptography and -public-key infrastructure. - -SSL Concepts for OpenFlow -------------------------- - -This section is an introduction to the public-key infrastructure architectures -that Open vSwitch supports for SSL authentication. - -To connect over SSL, every Open vSwitch must have a unique private/public key -pair and a certificate that signs that public key. Typically, the Open vSwitch -generates its own public/private key pair. There are two common ways to obtain -a certificate for a switch: - -* Self-signed certificates: The Open vSwitch signs its certificate with its own - private key. In this case, each switch must be individually approved by the - OpenFlow controller(s), since there is no central authority. - - This is the only switch PKI model currently supported by NOX - (http://noxrepo.org). - -* Switch certificate authority: A certificate authority (the "switch CA") signs - each Open vSwitch's public key. The OpenFlow controllers then check that any - connecting switches' certificates are signed by that certificate authority. - - This is the only switch PKI model supported by the simple OpenFlow controller - included with Open vSwitch. - -Each Open vSwitch must also have a copy of the CA certificate for the -certificate authority that signs OpenFlow controllers' keys (the "controller -CA" certificate). Typically, the same controller CA certificate is installed -on all of the switches within a given administrative unit. There are two -common ways for a switch to obtain the controller CA certificate: - -* Manually copy the certificate to the switch through some secure means, e.g. - using a USB flash drive, or over the network with "scp", or even FTP or HTTP - followed by manual verification. - -* Open vSwitch "bootstrap" mode, in which Open vSwitch accepts and saves the - controller CA certificate that it obtains from the OpenFlow controller on its - first connection. Thereafter the switch will only connect to controllers - signed by the same CA certificate. - -Establishing a Public Key Infrastructure ----------------------------------------- - -Open vSwitch can make use of your existing public key infrastructure. If you -already have a PKI, you may skip forward to the next section. Otherwise, if -you do not have a PKI, the ovs-pki script included with Open vSwitch can help. -To create an initial PKI structure, invoke it as: - -:: - - $ ovs-pki init - -This will create and populate a new PKI directory. The default location for -the PKI directory depends on how the Open vSwitch tree was configured (to see -the configured default, look for the ``--dir`` option description in the output -of ``ovs-pki --help``). - -The pki directory contains two important subdirectories. The `controllerca` -subdirectory contains controller CA files, including the following: - -`cacert.pem` - Root certificate for the controller certificate authority. Each Open vSwitch - must have a copy of this file to allow it to authenticate valid controllers. - -`private/cakey.pem` - Private signing key for the controller certificate authority. This file must - be kept secret. There is no need for switches or controllers to have a copy - of it. - -The `switchca` subdirectory contains switch CA files, analogous to those in the -`controllerca` subdirectory: - -`cacert.pem` - Root certificate for the switch certificate authority. The OpenFlow - controller must have this file to enable it to authenticate valid switches. - -`private/cakey.pem` - Private signing key for the switch certificate authority. This file must be - kept secret. There is no need for switches or controllers to have a copy of - it. - -After you create the initial structure, you can create keys and certificates -for switches and controllers with ovs-pki. Refer to the ovs-pki(8) manage for -complete details. A few examples of its use follow: - -Controller Key Generation -~~~~~~~~~~~~~~~~~~~~~~~~~ - -To create a controller private key and certificate in files named -ctl-privkey.pem and ctl-cert.pem, run the following on the machine that -contains the PKI structure: - -:: - - $ ovs-pki req+sign ctl controller - -ctl-privkey.pem and ctl-cert.pem would need to be copied to the controller for -its use at runtime. If, for testing purposes, you were to use -ovs-testcontroller, the simple OpenFlow controller included with Open vSwitch, -then the --private-key and --certificate options, respectively, would point to -these files. - -It is very important to make sure that no stray copies of ctl-privkey.pem are -created, because they could be used to impersonate the controller. - -Switch Key Generation with Self-Signed Certificates -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you are using self-signed certificates (see "SSL Concepts for OpenFlow"), -this is one way to create an acceptable certificate for your controller to -approve. - -1. Run the following command on the Open vSwitch itself:: - - $ ovs-pki self-sign sc - - .. note:: - This command does not require a copy of any of the PKI files generated by - ``ovs-pki init``, and you should not copy them to the switch because some - of them have contents that must remain secret for security.) - - The ``ovs-pki self-sign`` command has the following output: - - sc-privkey.pem - the switch private key file. For security, the contents of this file must - remain secret. There is ordinarily no need to copy this file off the Open - vSwitch. - - sc-cert.pem - the switch certificate, signed by the switch's own private key. Its - contents are not a secret. - -2. Optionally, copy `controllerca/cacert.pem` from the machine that has the - OpenFlow PKI structure and verify that it is correct. (Otherwise, you will - have to use CA certificate bootstrapping when you configure Open vSwitch in - the next step.) - -3. Configure Open vSwitch to use the keys and certificates (see "Configuring - SSL Support", below). - -Switch Key Generation with a Switch PKI (Easy Method) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you are using a switch PKI (see "SSL Concepts for OpenFlow", above), this -method of switch key generation is a little easier than the alternate method -described below, but it is also a little less secure because it requires -copying a sensitive private key from file from the machine hosting the PKI to -the switch. - -1. Run the following on the machine that contains the PKI structure:: - - $ ovs-pki req+sign sc switch - - This command has the following output: - - sc-privkey.pem - the switch private key file. For security, the contents of this file must - remain secret. - - sc-cert.pem - the switch certificate. Its contents are not a secret. - -2. Copy sc-privkey.pem and sc-cert.pem, plus controllerca/cacert.pem, to the - Open vSwitch. - -3. Delete the copies of sc-privkey.pem and sc-cert.pem on the PKI machine and - any other copies that may have been made in transit. It is very important - to make sure that there are no stray copies of sc-privkey.pem, because they - could be used to impersonate the switch. - - .. warning:: - Don't delete controllerca/cacert.pem! It is not security-sensitive and - you will need it to configure additional switches. - -4. Configure Open vSwitch to use the keys and certificates (see "Configuring - SSL Support", below). - -Switch Key Generation with a Switch PKI (More Secure) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you are using a switch PKI (see "SSL Concepts for OpenFlow", above), then, -compared to the previous method, the method described here takes a little more -work, but it does not involve copying the private key from one machine to -another, so it may also be a little more secure. - -1. Run the following command on the Open vSwitch itself:: - - $ ovs-pki req sc - - .. note:: - This command does not require a copy of any of the PKI files generated by - "ovs-pki init", and you should not copy them to the switch because some of - them have contents that must remain secret for security. - - The "ovs-pki req" command has the following output: - - sc-privkey.pem - the switch private key file. For security, the contents of this file must - remain secret. There is ordinarily no need to copy this file off the Open - vSwitch. - - sc-req.pem - the switch "certificate request", which is essentially the switch's public - key. Its contents are not a secret. - - a fingerprint - this is output on stdout. - -2. Write the fingerprint down on a slip of paper and copy `sc-req.pem` to the - machine that contains the PKI structure. - -3. On the machine that contains the PKI structure, run:: - - $ ovs-pki sign sc switch - - This command will output a fingerprint to stdout and request that you verify - it. Check that it is the same as the fingerprint that you wrote down on the - slip of paper before you answer "yes". - - ``ovs-pki sign`` creates a file named `sc-cert.pem`, which is the switch - certificate. Its contents are not a secret. - -4. Copy the generated `sc-cert.pem`, plus `controllerca/cacert.pem` from the - PKI structure, to the Open vSwitch, and verify that they were copied - correctly. - - You may delete `sc-cert.pem` from the machine that hosts the PKI - structure now, although it is not important that you do so. - - .. warning:: - Don't delete `controllerca/cacert.pem`! It is not security-sensitive and - you will need it to configure additional switches. - -5. Configure Open vSwitch to use the keys and certificates (see "Configuring - SSL Support", below). - -Configuring SSL Support ------------------------ - -SSL configuration requires three additional configuration files. The first two -of these are unique to each Open vSwitch. If you used the instructions above -to build your PKI, then these files will be named `sc-privkey.pem` and -`sc-cert.pem`, respectively: - -- A private key file, which contains the private half of an RSA or DSA key. - - This file can be generated on the Open vSwitch itself, for the greatest - security, or it can be generated elsewhere and copied to the Open vSwitch. - - The contents of the private key file are secret and must not be exposed. - -- A certificate file, which certifies that the private key is that of a - trustworthy Open vSwitch. - - This file has to be generated on a machine that has the private key for the - switch certification authority, which should not be an Open vSwitch; ideally, - it should be a machine that is not networked at all. - - The certificate file itself is not a secret. - -The third configuration file is typically the same across all the switches in a -given administrative unit. If you used the instructions above to build your -PKI, then this file will be named `cacert.pem`: - -- The root certificate for the controller certificate authority. The Open - vSwitch verifies it that is authorized to connect to an OpenFlow controller - by verifying a signature against this CA certificate. - -Once you have these files, configure ovs-vswitchd to use them using the -``ovs-vsctl set-ssl`` command, e.g.:: - - $ ovs-vsctl set-ssl /etc/openvswitch/sc-privkey.pem \ - /etc/openvswitch/sc-cert.pem /etc/openvswitch/cacert.pem - -Substitute the correct file names, of course, if they differ from the ones used -above. You should use absolute file names (ones that begin with ``/``), -because ovs-vswitchd's current directory is unrelated to the one from which you -run ovs-vsctl. - -If you are using self-signed certificates (see "SSL Concepts for OpenFlow") and -you did not copy controllerca/cacert.pem from the PKI machine to the Open -vSwitch, then add the ``--bootstrap`` option, e.g.:: - - $ ovs-vsctl -- --bootstrap set-ssl /etc/openvswitch/sc-privkey.pem \ - /etc/openvswitch/sc-cert.pem /etc/openvswitch/cacert.pem - -After you have added all of these configuration keys, you may specify ``ssl:`` -connection methods elsewhere in the configuration database. ``tcp:`` connection -methods are still allowed even after SSL has been configured, so for security -you should use only ``ssl:`` connections. - -Reporting Bugs --------------- - -Report problems to bugs@openvswitch.org. diff --git a/INSTALL.Windows.rst b/INSTALL.Windows.rst deleted file mode 100644 index 251cd10e3..000000000 --- a/INSTALL.Windows.rst +++ /dev/null @@ -1,621 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -======================= -Open vSwitch on Windows -======================= - -.. _windows-build-reqs: - -Build Requirements ------------------- - -Open vSwitch on Linux uses autoconf and automake for generating Makefiles. It -will be useful to maintain the same build system while compiling on Windows -too. One approach is to compile Open vSwitch in a MinGW environment that -contains autoconf and automake utilities and then use Visual C++ as a compiler -and linker. - -The following explains the steps in some detail. - -- Mingw - - Install Mingw on a Windows machine by following the instructions on - `mingw.org `__. - - This should install mingw at ``C:\Mingw`` and msys at ``C:\Mingw\msys``. Add - ``C:\MinGW\bin`` and ``C:\Mingw\msys\1.0\bin`` to PATH environment variable - of Windows. - - You can either use the MinGW installer or the command line utility - ``mingw-get`` to install both the base packages and additional packages like - automake and autoconf(version 2.68). - - Also make sure that ``/mingw`` mount point exists. If its not, please - add/create the following entry in ``/etc/fstab``:: - - 'C:/MinGW /mingw'. - -- Python - - Install the latest Python 2.x from python.org and verify that its path is - part of Windows' PATH environment variable. - -- Visual Studio - - You will need at least Visual Studio 2013 (update 4) to compile userspace - binaries. In addition to that, if you want to compile the kernel module you - will also need to install Windows Driver Kit (WDK) 8.1 Update. - - It is important to get the Visual Studio related environment variables and to - have the $PATH inside the bash to point to the proper compiler and linker. - One easy way to achieve this for VS2013 is to get into the "VS2013 x86 Native - Tools Command Prompt" (in a default installation of Visual Studio 2013 this - can be found under the following location: ``C:\Program Files (x86)\Microsoft - Visual Studio 12.0\Common7\Tools\Shortcuts``) and through it enter into the - bash shell available from msys by typing ``bash --login``. - - There is support for generating 64 bit binaries too. To compile under x64, - open the "VS2013 x64 Native Tools Command Prompt" (if your current running OS - is 64 bit) or "VS2013 x64 Cross Tools Command Prompt" (if your current - running OS is not 64 bit) instead of opening its x86 variant. This will - point the compiler and the linker to their 64 bit equivalent. - - If after the above step, a ``which link`` inside MSYS's bash says, - ``/bin/link.exe``, rename ``/bin/link.exe`` to something else so that the - Visual studio's linker is used. You should also see a 'which sort' report - ``/bin/sort.exe``. - -- pthreads-win32 - - For pthread support, install the library, dll and includes of pthreads-win32 - project from `sourceware - `__ to a - directory (e.g.: ``C:/pthread``). You should add the pthread-win32's dll path - (e.g.: ``C:\pthread\dll\x86``) to the Windows' PATH environment variable. - -- OpenSSQL - - To get SSL support for Open vSwitch on Windows, you will need to install - `OpenSSL for Windows `__ - - Note down the directory where OpenSSL is installed (e.g.: - ``C:/OpenSSL-Win32``) for later use. - -Install Requirements --------------------- - -* Share network adaptors - - We require that you don't disable the "Allow management operating system to - share this network adapter" under 'Virtual Switch Properties' > 'Connection - type: External network', in the HyperV virtual network switch configuration. - -* Checksum Offloads - - While there is some support for checksum/segmentation offloads in software, - this is still a work in progress. Till the support is complete we recommend - disabling TX/RX offloads for both the VM's as well as the HyperV. - -Bootstrapping -------------- - -This step is not needed if you have downloaded a released tarball. If -you pulled the sources directly from an Open vSwitch Git tree or got a -Git tree snapshot, then run boot.sh in the top source directory to build -the "configure" script:: - - > ./boot.sh - -.. _windows-configuring: - -Configuring ------------ - -Configure the package by running the configure script. You should provide some -configure options to choose the right compiler, linker, libraries, Open vSwitch -component installation directories, etc. For example:: - - > ./configure CC=./build-aux/cccl LD="$(which link)" \ - LIBS="-lws2_32 -liphlpapi" --prefix="C:/openvswitch/usr" \ - --localstatedir="C:/openvswitch/var" \ - --sysconfdir="C:/openvswitch/etc" \ - --with-pthread="C:/pthread" - -.. note:: - By default, the above enables compiler optimization for fast code. For - default compiler optimization, pass the ``--with-debug`` configure option. - -To configure with SSL support, add the requisite additional options:: - - > ./configure CC=./build-aux/cccl LD="`which link`" \ - LIBS="-lws2_32 -liphlpapi" --prefix="C:/openvswitch/usr" \ - --localstatedir="C:/openvswitch/var" - --sysconfdir="C:/openvswitch/etc" \ - --with-pthread="C:/pthread" \ - --enable-ssl --with-openssl="C:/OpenSSL-Win32" - -Finally, to the kernel module also:: - - > ./configure CC=./build-aux/cccl LD="`which link`" \ - LIBS="-lws2_32 -liphlpapi" --prefix="C:/openvswitch/usr" \ - --localstatedir="C:/openvswitch/var" \ - --sysconfdir="C:/openvswitch/etc" \ - --with-pthread="C:/pthread" \ - --enable-ssl --with-openssl="C:/OpenSSL-Win32" \ - --with-vstudiotarget="" - -Possible values for ```` are: ``Debug`` and ``Release`` - -.. note:: - You can directly use the Visual Studio 2013 IDE to compile the kernel - datapath. Open the ovsext.sln file in the IDE and build the solution. - -Refer to the `installation guide ` for information on additional -configuration options. - -.. _windows-building: - -Building --------- - -Once correctly configured, building Open vSwitch on Windows is similar to -building on Linux, FreeBSD, or NetBSD. - -#. Run make for the ported executables in the top source directory, e.g.:: - - > make - - For faster compilation, you can pass the ``-j`` argument to make. For - example, to run 4 jobs simultaneously, run ``make -j4``. - - .. note:: - - MSYS 1.0.18 has a bug that causes parallel make to hang. You can overcome - this by downgrading to MSYS 1.0.17. A simple way to downgrade is to exit - all MinGW sessions and then run the below command from MSVC developers - command prompt.:: - - > mingw-get upgrade msys-core-bin=1.0.17-1 - -#. To run all the unit tests in Open vSwitch, one at a time:: - - > make check - - To run all the unit tests in Open vSwitch, up to 8 in parallel:: - - > make check TESTSUITEFLAGS="-j8" - -#. To install all the compiled executables on the local machine, run:: - - > make install - - .. note:: - - This will install the Open vSwitch executables in ``C:/openvswitch``. You - can add ``C:\openvswitch\usr\bin`` and ``C:\openvswitch\usr\sbin`` to - Windows' PATH environment variable for easy access. - -The Kernel Module -~~~~~~~~~~~~~~~~~ - -If you are building the kernel module, you will need to copy the below files to -the target Hyper-V machine. - -- ``./datapath-windows/x64/Win8.1Debug/package/ovsext.inf`` -- ``./datapath-windows/x64/Win8.1Debug/package/OVSExt.sys`` -- ``./datapath-windows/x64/Win8.1Debug/package/ovsext.cat`` -- ``./datapath-windows/misc/install.cmd`` -- ``./datapath-windows/misc/uninstall.cmd`` - -.. note:: - The above path assumes that the kernel module has been built using Windows - DDK 8.1 in Debug mode. Change the path appropriately, if a different WDK - has been used. - -Now run ``./uninstall.cmd`` to remove the old extension. Once complete, run -``./install.cmd`` to insert the new one. For this to work you will have to -turn on ``TESTSIGNING`` boot option or 'Disable Driver Signature -Enforcement' during boot. The following commands can be used:: - - > bcdedit /set LOADOPTIONS DISABLE_INTEGRITY_CHECKS - > bcdedit /set TESTSIGNING ON - > bcdedit /set nointegritychecks ON - -.. note:: - You may have to restart the machine for the settings to take effect. - -In the Virtual Switch Manager configuration you can enable the Open vSwitch -Extension on an existing switch or create a new switch. If you are using an -existing switch, make sure to enable the "Allow Management OS" option for VXLAN -to work (covered later). - -The command to create a new switch named 'OVS-Extended-Switch' using a physical -NIC named 'Ethernet 1' is:: - - PS > New-VMSwitch "OVS-Extended-Switch" -AllowManagementOS $true \ - -NetAdapterName "Ethernet 1" - -.. note:: - You can obtain the list of physical NICs on the host using 'Get-NetAdapter' - command. - -In the properties of any switch, you should should now see "Open vSwitch -Extension" under 'Extensions'. Click the check box to enable the extension. -An alternative way to do the same is to run the following command:: - - PS > Enable-VMSwitchExtension "Open vSwitch Extension" OVS-Extended-Switch - -.. note:: - If you enabled the extension using the command line, a delay of a few seconds - has been observed for the change to be reflected in the UI. This is not a - bug in Open vSwitch. - -Starting --------- - -.. important:: - The following steps assume that you have installed the Open vSwitch utilities - in the local machine via 'make install'. - -Before starting ovs-vswitchd itself, you need to start its configuration -database, ovsdb-server. Each machine on which Open vSwitch is installed should -run its own copy of ovsdb-server. Before ovsdb-server itself can be started, -configure a database that it can use:: - - > ovsdb-tool create C:\openvswitch\etc\openvswitch\conf.db \ - C:\openvswitch\usr\share\openvswitch\vswitch.ovsschema - -Configure ovsdb-server to use database created above and to listen on a Unix -domain socket:: - - > ovsdb-server -vfile:info --remote=punix:db.sock --log-file \ - --pidfile --detach - -.. note:: - The logfile is created at ``C:/openvswitch/var/log/openvswitch/`` - -Initialize the database using ovs-vsctl. This is only necessary the first time -after you create the database with ovsdb-tool, though running it at any time is -harmless:: - - > ovs-vsctl --no-wait init - -.. tip:: - If you would later like to terminate the started ovsdb-server, run:: - - > ovs-appctl -t ovsdb-server exit - -Start the main Open vSwitch daemon, telling it to connect to the same Unix -domain socket:: - - > ovs-vswitchd -vfile:info --log-file --pidfile --detach - -.. tip:: - If you would like to terminate the started ovs-vswitchd, run:: - - > ovs-appctl exit - -.. note:: - The logfile is created at ``C:/openvswitch/var/log/openvswitch/`` - -Validating ----------- - -At this point you can use ovs-vsctl to set up bridges and other Open vSwitch -features. - -Add bridges -~~~~~~~~~~~ - -Let's start by creating an integration bridge, ``br-int`` and a PIF bridge, -``br-pif``:: - - > ovs-vsctl add-br br-int - > ovs-vsctl add-br br-pif - -.. note:: - There's a known bug that running the ovs-vsctl command does not terminate. - This is generally solved by having ovs-vswitchd running. If you face the - issue despite that, hit Ctrl-C to terminate ovs-vsctl and check the output to - see if your command succeeded. - -Validate that ports are added by dumping from both ovs-dpctl and ovs-vsctl:: - - > ovs-dpctl show - system@ovs-system: - lookups: hit:0 missed:0 lost:0 - flows: 0 - port 2: br-pif (internal) <<< internal port on 'br-pif' bridge - port 1: br-int (internal) <<< internal port on 'br-int' bridge - - > ovs-vsctl show - a56ec7b5-5b1f-49ec-a795-79f6eb63228b - Bridge br-pif - Port br-pif - Interface br-pif - type: internal - Bridge br-int - Port br-int - Interface br-int - type: internal - -.. note:: - There's a known bug that the ports added to OVSDB via ovs-vsctl don't get to - the kernel datapath immediately, ie. they don't show up in the output of - ``ovs-dpctl show`` even though they show up in output of ``ovs-vsctl show``. - In order to workaround this issue, restart ovs-vswitchd. (You can terminate - ovs-vswitchd by running ``ovs-appctl exit``.) - -Add physicals NICs (PIF) -~~~~~~~~~~~~~~~~~~~~~~~~ - -Now, let's add the physical NIC and the internal port to ``br-pif``. In OVS for -Hyper-V, we use the name of the adapter on top of which the Hyper-V virtual -switch was created, as a special name to refer to the physical NICs connected -to the Hyper-V switch, e.g. if we created the Hyper-V virtual switch on top of -the adapter named ``Ethernet0``, then in OVS we use that name (``Ethernet0``) -as a special name to refer to that adapter. - -.. note:: - we assume that the Hyper-V switch on which OVS extension is enabled has a - single physical NIC connected to it. - -An internal port is the virtual adapter created on the Hyper-V switch using the -``AllowManagementOS`` setting. This has already been setup while creating the -switch using the instructions above. In OVS for Hyper-V, we use a the name of -that specific adapter as a special name to refer to that adapter. By default it -is created under the following rule ``vEthernet ()``. - -As a whole example, if we issue the following in a powershell console:: - - PS C:\package\binaries> Get-NetAdapter | select Name,MacAddress,InterfaceDescription - Name MacAddress InterfaceDescription - ---- ---------- -------------------- - Ethernet1 00-0C-29-94-05-65 Intel(R) PRO/1000 MT Network Connection - vEthernet (external) 00-0C-29-94-05-5B Hyper-V Virtual Ethernet Adapter #2 - Ethernet0 00-0C-29-94-05-5B Intel(R) PRO/1000 MT Network Connection #2 - - PS C:\package\binaries> Get-VMSwitch - Name SwitchType NetAdapterInterfaceDescription - ---- ---------- ------------------------------ - external External Intel(R) PRO/1000 MT Network Connection #2 - -We can see that we have a switch(external) created upon adapter name -'Ethernet0' with an internal port under name ``vEthernet (external)``. Thus -resulting into the following ovs-vsctl commands:: - - > ovs-vsctl add-port br-pif Ethernet0 - > ovs-vsctl add-port br-pif "vEthernet (external)" - -Dumping the ports should show the additional ports that were just added:: - - > ovs-dpctl show - system@ovs-system: - lookups: hit:0 missed:0 lost:0 - flows: 0 - port 4: vEthernet (external) (internal) <<< 'AllowManagementOS' - adapter on - Hyper-V switch - port 2: br-pif (internal) - port 1: br-int (internal) - port 3: Ethernet0 <<< Physical NIC - - > ovs-vsctl show - a56ec7b5-5b1f-49ec-a795-79f6eb63228b - Bridge br-pif - Port "vEthernet (external)" - Interface "vEthernet (external)" - Port br-pif - Interface br-pif - type: internal - Port "Ethernet0" - Interface "Ethernet0" - Bridge br-int - Port br-int - Interface br-int - type: internal - -Add virtual interfaces (VIFs) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Adding VIFs to openvswitch is a two step procedure. The first step is to -assign a 'OVS port name' which is a unique name across all VIFs on this -Hyper-V. The next step is to add the VIF to the ovsdb using its 'OVS port -name' as key. - -First, assign a unique 'OVS port name' to the VIF. The VIF needs to have been -disconnected from the Hyper-V switch before assigning a 'OVS port name' to it. -In the example below, we assign a 'OVS port name' called ``ovs-port-a`` to a -VIF on a VM ``VM1``. By using index 0 for ``$vnic``, the first VIF of the VM -is being addressed. After assigning the name ``ovs-port-a``, the VIF is -connected back to the Hyper-V switch with name ``OVS-HV-Switch``, which is -assumed to be the Hyper-V switch with OVS extension enabled.:: - - PS> import-module .\datapath-windows\misc\OVS.psm1 - PS> $vnic = Get-VMNetworkAdapter - PS> Disconnect-VMNetworkAdapter -VMNetworkAdapter $vnic[0] - PS> $vnic[0] | Set-VMNetworkAdapterOVSPort -OVSPortName ovs-port-a - PS> Connect-VMNetworkAdapter -VMNetworkAdapter $vnic[0] \ - -SwitchName OVS-Extended-Switch - -Next, add the VIFs to ``br-int``:: - - > ovs-vsctl add-port br-int ovs-port-a - -Dumping the ports should show the additional ports that were just added:: - - > ovs-dpctl show - system@ovs-system: - lookups: hit:0 missed:0 lost:0 - flows: 0 - port 4: vEthernet (external) (internal) - port 5: ovs-port-a - port 2: br-pif (internal) - port 1: br-int (internal - port 3: Ethernet0 - - > ovs-vsctl show - 4cd86499-74df-48bd-a64d-8d115b12a9f2 - Bridge br-pif - Port "vEthernet (external)" - Interface "vEthernet (external)" - Port "Ethernet0" - Interface "Ethernet0" - Port br-pif - Interface br-pif - type: internal - Bridge br-int - Port br-int - Interface br-int - type: internal - Port "ovs-port-a" - Interface "ovs-port-a" - -Add patch ports and configure VLAN tagging -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The Windows Open vSwitch implementation support VLAN tagging in the switch. -Switch VLAN tagging along with patch ports between ``br-int`` and ``br-pif`` is -used to configure VLAN tagging functionality between two VMs on different -Hyper-Vs. To start, add a patch port from ``br-int`` to ``br-pif``:: - - > ovs-vsctl add-port br-int patch-to-pif - > ovs-vsctl set interface patch-to-pif type=patch \ - options:peer=patch-to-int - -Add a patch port from ``br-pif`` to ``br-int``:: - - > ovs-vsctl add-port br-pif patch-to-int - > ovs-vsctl set interface patch-to-int type=patch \ - options:peer=patch-to-pif - -Re-Add the VIF ports with the VLAN tag:: - - > ovs-vsctl add-port br-int ovs-port-a tag=900 - > ovs-vsctl add-port br-int ovs-port-b tag=900 - -Add tunnels -~~~~~~~~~~~ - -The Windows Open vSwitch implementation support VXLAN and STT tunnels. To add -tunnels. For example, first add the tunnel port between 172.168.201.101 <-> -172.168.201.102:: - - > ovs-vsctl add-port br-int tun-1 - > ovs-vsctl set Interface tun-1 type= - > ovs-vsctl set Interface tun-1 options:local_ip=172.168.201.101 - > ovs-vsctl set Interface tun-1 options:remote_ip=172.168.201.102 - > ovs-vsctl set Interface tun-1 options:in_key=flow - > ovs-vsctl set Interface tun-1 options:out_key=flow - -...and the tunnel port between 172.168.201.101 <-> 172.168.201.105:: - - > ovs-vsctl add-port br-int tun-2 - > ovs-vsctl set Interface tun-2 type= - > ovs-vsctl set Interface tun-2 options:local_ip=172.168.201.102 - > ovs-vsctl set Interface tun-2 options:remote_ip=172.168.201.105 - > ovs-vsctl set Interface tun-2 options:in_key=flow - > ovs-vsctl set Interface tun-2 options:out_key=flow - -Where ```` is one of: ``stt`` or ``vxlan`` - -.. note:: - Any patch ports created between br-int and br-pif MUST be be deleted prior to - adding tunnels. - -Windows Services ----------------- - -Open vSwitch daemons come with support to run as a Windows service. The -instructions here assume that you have installed the Open vSwitch utilities and -daemons via ``make install``. - -.. note:: - The commands shown here can be run from MSYS bash or Windows command prompt. - -To start, create the database:: - - > ovsdb-tool create C:/openvswitch/etc/openvswitch/conf.db \ - "C:/openvswitch/usr/share/openvswitch/vswitch.ovsschema" - -Create the ovsdb-server service and start it:: - - > sc create ovsdb-server \ - binpath="C:/openvswitch/usr/sbin/ovsdb-server.exe \ - C:/openvswitch/etc/openvswitch/conf.db \ - -vfile:info --log-file --pidfile \ - --remote=punix:db.sock --service --service-monitor" - > sc start ovsdb-server - -.. tip:: - One of the common issues with creating a Windows service is with mungled - paths. You can make sure that the correct path has been registered with the - Windows services manager by running:: - - > sc qc ovsdb-server - -Check that the service is healthy by running:: - - > sc query ovsdb-server - -Initialize the database:: - - > ovs-vsctl --no-wait init - -Create the ovs-vswitchd service and start it:: - - > sc create ovs-vswitchd \ - binpath="C:/openvswitch/usr/sbin/ovs-vswitchd.exe \ - --pidfile -vfile:info --log-file --service --service-monitor" - > sc start ovs-vswitchd - -Check that the service is healthy by running:: - - > sc query ovs-vswitchd - -To stop and delete the services, run:: - - > sc stop ovs-vswitchd - > sc stop ovsdb-server - > sc delete ovs-vswitchd - > sc delete ovsdb-server - -Windows CI Service ------------------- - -`AppVeyor `__ provides a free Windows autobuild service for -opensource projects. Open vSwitch has integration with AppVeyor for continuous -build. A developer can build test his changes for Windows by logging into -appveyor.com using a github account, creating a new project by linking it to -his development repository in github and triggering a new build. - -TODO ----- - -* Investigate the working of sFlow on Windows and re-enable the unit tests. - -* Investigate and add the feature to provide QoS. - -* Sign the driver & create an MSI for installing the different OpenvSwitch - components on Windows. diff --git a/INSTALL.XenServer.rst b/INSTALL.XenServer.rst deleted file mode 100644 index 5f0fa4136..000000000 --- a/INSTALL.XenServer.rst +++ /dev/null @@ -1,229 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -=============================================== -How to Install Open vSwitch on Citrix XenServer -=============================================== - -This document describes how to build and install Open vSwitch on a Citrix -XenServer host. If you want to install Open vSwitch on a generic Linux or BSD -host, see the `installation guide `__ instead. - -Open vSwitch should work with XenServer 5.6.100 and later. However, Open -vSwitch requires Python 2.7 or later, so using Open vSwitch with XenServer 6.5 -or earlier requires installing Python 2.7. - -Building --------- - -You may build from an Open vSwitch distribution tarball or from an Open vSwitch -Git tree. The recommended build environment to build RPMs for Citrix XenServer -is the DDK VM available from Citrix. - -1. If you are building from an Open vSwitch Git tree, then you will need to - first create a distribution tarball by running:: - - $ ./boot.sh - $ ./configure - $ make dist - - You cannot run this in the DDK VM, because it lacks tools that are necessary - to bootstrap the Open vSwitch distribution. Instead, you must run this on a - machine that has the tools listed in the installation guide `__ - as prerequisites for building from a Git tree. - -2. Copy the distribution tarball into ``/usr/src/redhat/SOURCES`` inside - the DDK VM. - -3. In the DDK VM, unpack the distribution tarball into a temporary directory - and "cd" into the root of the distribution tarball. - -4. To build Open vSwitch userspace, run:: - - $ rpmbuild -bb xenserver/openvswitch-xen.spec - - This produces three RPMs in ``/usr/src/redhat/RPMS/i386``: - - - ``openvswitch`` - - ``openvswitch-modules-xen`` - - ``openvswitch-debuginfo`` - - The above command automatically runs the Open vSwitch unit tests. To - disable the unit tests, run:: - - $ rpmbuild -bb --without check xenserver/openvswitch-xen.spec - -Build Parameters ----------------- - -``openvswitch-xen.spec`` needs to know a number of pieces of information about -the XenServer kernel. Usually, it can figure these out for itself, but if it -does not do it correctly then you can specify them yourself as parameters to -the build. Thus, the final ``rpmbuild`` step above can be elaborated as:: - - $ VERSION= - $ KERNEL_NAME= - $ KERNEL_VERSION= - $ KERNEL_FLAVOR= - $ rpmbuild \ - -D "openvswitch_version $VERSION" \ - -D "kernel_name $KERNEL_NAME" \ - -D "kernel_version $KERNEL_VERSION" \ - -D "kernel_flavor $KERNEL_FLAVOR" \ - -bb xenserver/openvswitch-xen.spec - -where: - -```` - is the version number that appears in the name of the Open vSwitch tarball, - e.g. 0.90.0. - -```` - is the name of the XenServer kernel package, e.g. ``kernel-xen`` or - ``kernel-NAME-xen``, without the ``kernel-`` prefix. - -```` - is the output of:: - - $ rpm -q --queryformat "%{Version}-%{Release}" , - - e.g. ``2.6.32.12-0.7.1.xs5.6.100.323.170596``, where - ```` is the name of the ``-devel`` package - corresponding to ````. - -```` - is either ``xen`` or ``kdump``, where ``xen`` flavor is the main running - kernel flavor and the ``kdump`` flavor is the crashdump kernel flavor. - Commonly, one would specify ``xen`` here. - -For XenServer 6.5 or above, the kernel version naming no longer contains -KERNEL_FLAVOR. In fact, only providing the ``uname -r`` output is enough. So, -the final ``rpmbuild`` step changes to:: - - $ KERNEL_UNAME=<`uname -r` output> - $ rpmbuild \ - -D "kenel_uname $KERNEL_UNAME" \ - -bb xenserver/openvswitch-xen.spec - -Installing Open vSwitch for XenServer -------------------------------------- - -To install Open vSwitch on a XenServer host, or to upgrade to a newer version, -copy the ``openvswitch`` and ``openvswitch-modules-xen`` RPMs to that host with -``scp``, then install them with ``rpm -U``, e.g.:: - - $ scp openvswitch-$VERSION-1.i386.rpm \ - openvswitch-modules-xen-$XEN_KERNEL_VERSION-$VERSION-1.i386.rpm \ - root@: - # Enter 's root password. - $ ssh root@ - # Enter 's root password again. - $ rpm -U openvswitch-$VERSION-1.i386.rpm \ - openvswitch-modules-xen-$XEN_KERNEL_VERSION-$VERSION-1.i386.rpm - -To uninstall Open vSwitch from a XenServer host, remove the packages:: - - $ ssh root@ - # Enter 's root password again. - $ rpm -e openvswitch openvswitch-modules-xen-$XEN_KERNEL_VERSION - -After installing or uninstalling Open vSwitch, the XenServer should be rebooted -as soon as possible. - -Open vSwitch Boot Sequence on XenServer ---------------------------------------- - -When Open vSwitch is installed on XenServer, its startup script -``/etc/init.d/openvswitch`` runs early in boot. It does roughly the following: - -* Loads the OVS kernel module, openvswitch. - -* Starts ovsdb-server, the OVS configuration database. - -* XenServer expects there to be no bridges configured at startup, but the OVS - configuration database likely still has bridges configured from before - reboot. To match XenServer expectations, the startup script deletes all - configured bridges from the database. - -* Starts ovs-vswitchd, the OVS switching daemon. - -At this point in the boot process, then, there are no Open vSwitch bridges, -even though all of the Open vSwitch daemons are running. Later on in boot, -``/etc/init.d/management-interface`` (part of XenServer, not Open vSwitch) -creates the bridge for the XAPI management interface by invoking -``/opt/xensource/libexec/interface-reconfigure``. Normally this program -consults XAPI's database to obtain information about how to configure the -bridge, but XAPI is not running yet(\*) so it instead consults -``/var/xapi/network.dbcache``, which is a cached copy of the most recent -network configuration. - -(\*) Even if XAPI were running, if this XenServer node is a pool slave then the - query would have to consult the master, which requires network access, - which begs the question of how to configure the management interface. - -XAPI starts later on in the boot process. XAPI can then create other bridges -on demand using ``/opt/xensource/libexec/interface-reconfigure``. Now that -XAPI is running, that program consults XAPI directly instead of reading the -cache. - -As part of its own startup, XAPI invokes the Open vSwitch XAPI plugin script -``/etc/xapi.d/openvswitch-cfg-update`` passing the ``update`` command. The -plugin script does roughly the following: - -* Calls ``/opt/xensource/libexec/interface-reconfigure`` with the ``rewrite`` - command, to ensure that the network cache is up-to-date. - -* Queries the Open vSwitch manager setting (named ``vswitch_controller``) from - the XAPI database for the XenServer pool. - -* If XAPI and OVS are configured for different managers, or if OVS is - configured for a manager but XAPI is not, runs ``ovs-vsctl emer-reset`` to - bring the Open vSwitch configuration to a known state. One effect of - emer-reset is to deconfigure any manager from the OVS database. - -* If XAPI is configured for a manager, configures the OVS manager to match with - ``ovs-vsctl set-manager``. - -Notes ------ - -* The Open vSwitch boot sequence only configures an OVS configuration database - manager. There is no way to directly configure an OpenFlow controller on - XenServer and, as a consequence of the step above that deletes all of the - bridges at boot time, controller configuration only persists until XenServer - reboot. The configuration database manager can, however, configure - controllers for bridges. See the BUGS section of ovs-testcontroller(8) for - more information on this topic. - -* The Open vSwitch startup script automatically adds a firewall rule to allow - GRE traffic. This rule is needed for the XenServer feature called "Cross-Host - Internal Networks" (CHIN) that uses GRE. If a user configures tunnels other - than GRE (ex: Geneve, VXLAN, LISP), they will have to either manually add a - iptables firewall rule to allow the tunnel traffic or add it through a - startup script (Please refer to the "enable-protocol" command in the - ovs-ctl(8) manpage). - -Reporting Bugs --------------- - -Please report problems to bugs@openvswitch.org. diff --git a/INSTALL.rst b/INSTALL.rst deleted file mode 100644 index d934c1f51..000000000 --- a/INSTALL.rst +++ /dev/null @@ -1,871 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -========================================= -Open vSwitch on Linux, FreeBSD and NetBSD -========================================= - -This document describes how to build and install Open vSwitch on a generic -Linux, FreeBSD, or NetBSD host. For specifics around installation on a specific -platform, refer to one of these installation guides: - -- `Debian `__ -- `Fedora `__ -- `RHEL `__ -- `XenServer `__ -- `NetBSD `__ -- `Windows `__ -- `DPDK `__ - -.. _general-build-reqs: - -Build Requirements ------------------- - -To compile the userspace programs in the Open vSwitch distribution, you will -need the following software: - -- GNU make - -- A C compiler, such as: - - - GCC 4.x. - - - Clang. Clang 3.4 and later provide useful static semantic analysis and - thread-safety checks. For Ubuntu, there are nightly built packages - available on clang's website. - - - MSVC 2013. See the `Windows installation guide `__ - for additional Windows build instructions. - - While OVS may be compatible with other compilers, optimal support for atomic - operations may be missing, making OVS very slow (see ``lib/ovs-atomic.h``). - -- libssl, from OpenSSL, is optional but recommended if you plan to connect the - Open vSwitch to an OpenFlow controller. libssl is required to establish - confidentiality and authenticity in the connections from an Open vSwitch to - an OpenFlow controller. If libssl is installed, then Open vSwitch will - automatically build with support for it. - -- libcap-ng, written by Steve Grubb, is optional but recommended. It is - required to run OVS daemons as a non-root user with dropped root privileges. - If libcap-ng is installed, then Open vSwitch will automatically build with - support for it. - -- Python 2.7. You must also have the Python ``six`` library. - -On Linux, you may choose to compile the kernel module that comes with the Open -vSwitch distribution or to use the kernel module built into the Linux kernel -(version 3.3 or later). See the `FAQ `__ question "What features -are not available in the Open vSwitch kernel datapath that ships as part of the -upstream Linux kernel?" for more information on this trade-off. You may also -use the userspace-only implementation, at some cost in features and performance -(see the `userspace installation guide `__ for details). - -To compile the kernel module on Linux, you must also install the -following: - -- A supported Linux kernel version. Refer to the `README `__ - for a list of supported versions. - - For optional support of ingress policing, you must enable kernel - configuration options ``NET_CLS_BASIC``, ``NET_SCH_INGRESS``, and - ``NET_ACT_POLICE``, either built-in or as modules. ``NET_CLS_POLICE`` is - obsolete and not needed.) - - On kernels before 3.11, the ``ip_gre`` module, for GRE tunnels over IP - (``NET_IPGRE``), must not be loaded or compiled in. - - To configure HTB or HFSC quality of service with Open vSwitch, you must - enable the respective configuration options. - - To use Open vSwitch support for TAP devices, you must enable ``CONFIG_TUN``. - -- To build a kernel module, you need the same version of GCC that was used to - build that kernel. - -- A kernel build directory corresponding to the Linux kernel image the module - is to run on. Under Debian and Ubuntu, for example, each linux-image package - containing a kernel binary has a corresponding linux-headers package with - the required build infrastructure. - -If you are working from a Git tree or snapshot (instead of from a distribution -tarball), or if you modify the Open vSwitch build system or the database -schema, you will also need the following software: - -- Autoconf version 2.63 or later. - -- Automake version 1.10 or later. - -- libtool version 2.4 or later. (Older versions might work too.) - -To run the unit tests, you also need: - -- Perl. Version 5.10.1 is known to work. Earlier versions should also - work. - -The datapath tests for userspace and Linux datapaths also rely upon: - -- pyftpdlib. Version 1.2.0 is known to work. Earlier versions should - also work. - -- GNU wget. Version 1.16 is known to work. Earlier versions should also - work. - -The ovs-vswitchd.conf.db(5) manpage will include an E-R diagram, in formats -other than plain text, only if you have the following: - -- dot from graphviz (http://www.graphviz.org/). - -- Perl. Version 5.10.1 is known to work. Earlier versions should also - work. - -If you are going to extensively modify Open vSwitch, consider installing the -following to obtain better warnings: - -- "sparse" version 0.4.4 or later - (http://www.kernel.org/pub/software/devel/sparse/dist/). - -- GNU make. - -- clang, version 3.4 or later - -- flake8, version 2.X, along with the hacking flake8 plugin (for Python code). - The automatic flake8 check that runs against Python code has some warnings - enabled that come from the "hacking" flake8 plugin. If it's not installed, - the warnings just won't occur until it's run on a system with "hacking" - installed. Note that there are problems with flake8 3.0 and the "hacking" - plugin. To ensure you get flake8 2.X, you can use:: - - $ pip install 'flake8<3.0' - -You may find the ovs-dev script found in ``utilities/ovs-dev.py`` useful. - -.. _general-install-reqs: - -Installation Requirements -------------------------- - -The machine you build Open vSwitch on may not be the one you run it on. To -simply install and run Open vSwitch you require the following software: - -- libc compatible with the libc used for build. - -- libssl compatible with the libssl used for build, if OpenSSL was used - for the build. - -- On Linux, the same kernel version configured as part of the build. - -- For optional support of ingress policing on Linux, the "tc" program - from iproute2 (part of all major distributions and available at - http://www.linux-foundation.org/en/Net:Iproute2). - -- Python 2.7. You must also have the Python six library. - -On Linux you should ensure that ``/dev/urandom`` exists. To support TAP -devices, you must also ensure that ``/dev/net/tun`` exists. - -.. _general-bootstrapping: - -Bootstrapping -------------- - -This step is not needed if you have downloaded a released tarball. If -you pulled the sources directly from an Open vSwitch Git tree or got a -Git tree snapshot, then run boot.sh in the top source directory to build -the "configure" script:: - - $ ./boot.sh - -.. _general-configuring: - -Configuring ------------ - -Configure the package by running the configure script. You can usually -invoke configure without any arguments. For example:: - - $ ./configure - -By default all files are installed under ``/usr/local``. Open vSwitch also -expects to find its database in ``/usr/local/etc/openvswitch`` by default. If -you want to install all files into, e.g., ``/usr`` and ``/var`` instead of -``/usr/local`` and ``/usr/local/var`` and expect to use ``/etc/openvswitch`` as -the default database directory, add options as shown here:: - - $ ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc - -.. note:: - - Open vSwitch installed with packages like .rpm (e.g. via ``yum install`` or - ``rpm -ivh``) and .deb (e.g. via ``apt-get install`` or ``dpkg -i``) use the - above configure options. - -By default, static libraries are built and linked against. If you want to use -shared libraries instead:: - - $ ./configure --enable-shared - -To use a specific C compiler for compiling Open vSwitch user programs, also -specify it on the configure command line, like so:: - - $ ./configure CC=gcc-4.2 - -To use 'clang' compiler:: - - $ ./configure CC=clang - -To supply special flags to the C compiler, specify them as ``CFLAGS`` on the -configure command line. If you want the default CFLAGS, which include ``-g`` to -build debug symbols and ``-O2`` to enable optimizations, you must include them -yourself. For example, to build with the default CFLAGS plus ``-mssse3``, you -might run configure as follows:: - - $ ./configure CFLAGS="-g -O2 -mssse3" - -For efficient hash computation special flags can be passed to leverage built-in -intrinsics. For example on X86_64 with SSE4.2 instruction set support, CRC32 -intrinsics can be used by passing ``-msse4.2``:: - - $ ./configure CFLAGS="-g -O2 -msse4.2"` - -If you are on a different processor and don't know what flags to choose, it is -recommended to use ``-march=native`` settings:: - - $ ./configure CFLAGS="-g -O2 -march=native" - -With this, GCC will detect the processor and automatically set appropriate -flags for it. This should not be used if you are compiling OVS outside the -target machine. - -.. note:: - CFLAGS are not applied when building the Linux kernel module. Custom CFLAGS - for the kernel module are supplied using the ``EXTRA_CFLAGS`` variable when - running make. For example:: - - $ make EXTRA_CFLAGS="-Wno-error=date-time" - -To build the Linux kernel module, so that you can run the kernel-based switch, -pass the location of the kernel build directory on ``--with-linux``. For -example, to build for a running instance of Linux:: - - $ ./configure --with-linux=/lib/modules/$(uname -r)/build - -.. note:: - If ``--with-linux`` requests building for an unsupported version of Linux, - then ``configure`` will fail with an error message. Refer to the `FAQ - `__ for advice in that case. - -If you wish to build the kernel module for an architecture other than the -architecture of the machine used for the build, you may specify the kernel -architecture string using the KARCH variable when invoking the configure -script. For example, to build for MIPS with Linux:: - - $ ./configure --with-linux=/path/to/linux KARCH=mips - -If you plan to do much Open vSwitch development, you might want to add -``--enable-Werror``, which adds the ``-Werror`` option to the compiler command -line, turning warnings into errors. That makes it impossible to miss warnings -generated by the build. For example:: - - $ ./configure --enable-Werror - -To build with gcov code coverage support, add ``--enable-coverage``:: - - $ ./configure --enable-coverage - -The configure script accepts a number of other options and honors additional -environment variables. For a full list, invoke configure with the ``--help`` -option:: - - $ ./configure --help - -You can also run configure from a separate build directory. This is helpful if -you want to build Open vSwitch in more than one way from a single source -directory, e.g. to try out both GCC and Clang builds, or to build kernel -modules for more than one Linux version. For example:: - - $ mkdir _gcc && (cd _gcc && ./configure CC=gcc) - $ mkdir _clang && (cd _clang && ./configure CC=clang) - -Under certains loads the ovsdb-server and other components perform better when -using the jemalloc memory allocator, instead of the glibc memory allocator. If -you wish to link with jemalloc add it to LIBS:: - - $ ./configure LIBS=-ljemalloc - -.. _general-building: - -Building --------- - -1. Run GNU make in the build directory, e.g.:: - - $ make - - or if GNU make is installed as "gmake":: - - $ gmake - - If you used a separate build directory, run make or gmake from that - directory, e.g.:: - - $ make -C _gcc - $ make -C _clang - - For improved warnings if you installed ``sparse`` (see "Prerequisites"), add - ``C=1`` to the command line. - - .. note:: - Some versions of Clang and ccache are not completely compatible. If you - see unusual warnings when you use both together, consider disabling - ccache. - -2. Consider running the testsuite. Refer to **Testing** for instructions. - -3. Run ``make install`` to install the executables and manpages into the - running system, by default under ``/usr/local``:: - - $ make install - -5. If you built kernel modules, you may install them, e.g.:: - - $ make modules_install - - It is possible that you already had a Open vSwitch kernel module installed - on your machine that came from upstream Linux (in a different directory). To - make sure that you load the Open vSwitch kernel module you built from this - repository, you should create a ``depmod.d`` file that prefers your newly - installed kernel modules over the kernel modules from upstream Linux. The - following snippet of code achieves the same:: - - $ config_file="/etc/depmod.d/openvswitch.conf" - $ for module in datapath/linux/*.ko; do - modname="$(basename ${module})" - echo "override ${modname%.ko} * extra" >> "$config_file" - echo "override ${modname%.ko} * weak-updates" >> "$config_file" - done - $ depmod -a - - Finally, load the kernel modules that you need. e.g.:: - - $ /sbin/modprobe openvswitch - - To verify that the modules have been loaded, run ``/sbin/lsmod`` and check - that openvswitch is listed:: - - $ /sbin/lsmod | grep openvswitch - - .. note:: - If the ``modprobe`` operation fails, look at the last few kernel log - messages (e.g. with ``dmesg | tail``). Generally, issues like this occur - when Open vSwitch is built for a kernel different from the one into which - you are trying to load it. Run ``modinfo`` on ``openvswitch.ko`` and on a - module built for the running kernel, e.g.:: - - $ /sbin/modinfo openvswitch.ko - $ /sbin/modinfo /lib/modules/$(uname -r)/kernel/net/bridge/bridge.ko - - Compare the "vermagic" lines output by the two commands. If they differ, - then Open vSwitch was built for the wrong kernel. - - If you decide to report a bug or ask a question related to module loading, - include the output from the ``dmesg`` and ``modinfo`` commands mentioned - above. - -.. _general-starting: - -Starting --------- - -Before starting ovs-vswitchd itself, you need to start its configuration -database, ovsdb-server. Each machine on which Open vSwitch is installed should -run its own copy of ovsdb-server. Before ovsdb-server itself can be started, -configure a database that it can use:: - - $ mkdir -p /usr/local/etc/openvswitch - $ ovsdb-tool create /usr/local/etc/openvswitch/conf.db \ - vswitchd/vswitch.ovsschema - -Configure ovsdb-server to use database created above, to listen on a Unix -domain socket, to connect to any managers specified in the database itself, and -to use the SSL configuration in the database:: - - $ mkdir -p /usr/local/var/run/openvswitch - $ ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ - --remote=db:Open_vSwitch,Open_vSwitch,manager_options \ - --private-key=db:Open_vSwitch,SSL,private_key \ - --certificate=db:Open_vSwitch,SSL,certificate \ - --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \ - --pidfile --detach --log-file - -.. note:: - If you built Open vSwitch without SSL support, then omit ``--private-key``, - ``--certificate``, and ``--bootstrap-ca-cert``.) - -Initialize the database using ovs-vsctl. This is only necessary the first time -after you create the database with ovsdb-tool, though running it at any time is -harmless:: - - $ ovs-vsctl --no-wait init - -Start the main Open vSwitch daemon, telling it to connect to the same Unix -domain socket:: - - $ ovs-vswitchd --pidfile --detach --log-file - -Validating ----------- - -At this point you can use ovs-vsctl to set up bridges and other Open vSwitch -features. For example, to create a bridge named ``br0`` and add ports ``eth0`` -and ``vif1.0`` to it:: - - $ ovs-vsctl add-br br0 - $ ovs-vsctl add-port br0 eth0 - $ ovs-vsctl add-port br0 vif1.0 - -Refer to ovs-vsctl(8) for more details. - -Upgrading ---------- - -When you upgrade Open vSwitch from one version to another you should also -upgrade the database schema: - -1. Stop the Open vSwitch daemons, e.g.:: - - $ kill `cd /usr/local/var/run/openvswitch && cat ovsdb-server.pid ovs-vswitchd.pid` - -2. Install the new Open vSwitch release by using the same configure options as - was used for installing the previous version. If you do not use the same - configure options, you can end up with two different versions of Open - vSwitch executables installed in different locations. - -3. Upgrade the database, in one of the following two ways: - - - If there is no important data in your database, then you may delete the - database file and recreate it with ovsdb-tool, following the instructions - under "Building and Installing Open vSwitch for Linux, FreeBSD or NetBSD". - - - If you want to preserve the contents of your database, back it up first, - then use ``ovsdb-tool convert`` to upgrade it, e.g.:: - - $ ovsdb-tool convert /usr/local/etc/openvswitch/conf.db \ - vswitchd/vswitch.ovsschema - -4. Start the Open vSwitch daemons as described under **Starting** above. - -Hot Upgrading -------------- - -Upgrading Open vSwitch from one version to the next version with minimum -disruption of traffic going through the system that is using that Open vSwitch -needs some considerations: - -1. If the upgrade only involves upgrading the userspace utilities and daemons - of Open vSwitch, make sure that the new userspace version is compatible with - the previously loaded kernel module. - -2. An upgrade of userspace daemons means that they have to be restarted. - Restarting the daemons means that the OpenFlow flows in the ovs-vswitchd - daemon will be lost. One way to restore the flows is to let the controller - re-populate it. Another way is to save the previous flows using a utility - like ovs-ofctl and then re-add them after the restart. Restoring the old - flows is accurate only if the new Open vSwitch interfaces retain the old - 'ofport' values. - -3. When the new userspace daemons get restarted, they automatically flush the - old flows setup in the kernel. This can be expensive if there are hundreds - of new flows that are entering the kernel but userspace daemons are busy - setting up new userspace flows from either the controller or an utility like - ovs-ofctl. Open vSwitch database provides an option to solve this problem - through the ``other_config:flow-restore-wait`` column of the - ``Open_vSwitch`` table. Refer to the ovs-vswitchd.conf.db(5) manpage for - details. - -4. If the upgrade also involves upgrading the kernel module, the old kernel - module needs to be unloaded and the new kernel module should be loaded. This - means that the kernel network devices belonging to Open vSwitch is recreated - and the kernel flows are lost. The downtime of the traffic can be reduced if - the userspace daemons are restarted immediately and the userspace flows are - restored as soon as possible. - -The ovs-ctl utility's ``restart`` function only restarts the userspace daemons, -makes sure that the 'ofport' values remain consistent across restarts, restores -userspace flows using the ovs-ofctl utility and also uses the -``other_config:flow-restore-wait`` column to keep the traffic downtime to the -minimum. The ovs-ctl utility's ``force-reload-kmod`` function does all of the -above, but also replaces the old kernel module with the new one. Open vSwitch -startup scripts for Debian, XenServer and RHEL use ovs-ctl's functions and it -is recommended that these functions be used for other software platforms too. - -.. _general-testing: - -Testing -------- - -This section describe Open vSwitch's built-in support for various test -suites. You must bootstrap, configure and build Open vSwitch (steps are -in "Building and Installing Open vSwitch for Linux, FreeBSD or NetBSD" -above) before you run the tests described here. You do not need to -install Open vSwitch or to build or load the kernel module to run these -test suites. You do not need supervisor privilege to run these test -suites. - -Unit Tests -~~~~~~~~~~ - -Open vSwitch includes a suite of self-tests. Before you submit patches -upstream, we advise that you run the tests and ensure that they pass. If you -add new features to Open vSwitch, then adding tests for those features will -ensure your features don't break as developers modify other areas of Open -vSwitch. - -To run all the unit tests in Open vSwitch, one at a time, run:: - - $ make check - -This takes under 5 minutes on a modern desktop system. - -To run all the unit tests in Open vSwitch in parallel, run:: - - $ make check TESTSUITEFLAGS=-j8 - -You can run up to eight threads. This takes under a minute on a modern 4-core -desktop system. - -To see a list of all the available tests, run: - - $ make check TESTSUITEFLAGS=--list - -To run only a subset of tests, e.g. test 123 and tests 477 through 484, run:: - - $ make check TESTSUITEFLAGS='123 477-484' - -Tests do not have inter-dependencies, so you may run any subset. - -To run tests matching a keyword, e.g. ``ovsdb``, run:: - - $ make check TESTSUITEFLAGS='-k ovsdb' - -To see a complete list of test options, run:: - - $ make check TESTSUITEFLAGS=--help - -The results of a testing run are reported in ``tests/testsuite.log``. Report -report test failures as bugs and include the ``testsuite.log`` in your report. - -.. note:: - Sometimes a few tests may fail on some runs but not others. This is usually a - bug in the testsuite, not a bug in Open vSwitch itself. If you find that a - test fails intermittently, please report it, since the developers may not - have noticed. You can make the testsuite automatically rerun tests that fail, - by adding ``RECHECK=yes`` to the ``make`` command line, e.g.:: - - $ make check TESTSUITEFLAGS=-j8 RECHECK=yes - -Coverage -++++++++ - -If the build was configured with ``--enable-coverage`` and the ``lcov`` utility -is installed, you can run the testsuite and generate a code coverage report by -using the ``check-lcoc`` target:: - - $ make check-lcov - -All the same options are avaiable via TESTSUITEFLAGS. For example:: - - $ make check-lcov TESTSUITEFLAGS=-j8 -k ovn - -Valgrind -++++++++ - -If you have ``valgrind`` installed, you can run the testsuite under -valgrind by using the ``check-valgrind`` target:: - - $ make check-valgrind - -When you do this, the "valgrind" results for test ```` are reported in files -named ``tests/testsuite.dir//valgrind.*``. - -All the same options are available via TESTSUITEFLAGS. - -.. hint:: - You may find that the valgrind results are easier to interpret if you put - ``-q`` in ``~/.valgrindrc``, since that reduces the amount of output. - -.. _general-oftest: - -OFTest -~~~~~~ - -OFTest is an OpenFlow protocol testing suite. Open vSwitch includes a Makefile -target to run OFTest with Open vSwitch in "dummy mode". In this mode of -testing, no packets travel across physical or virtual networks. Instead, Unix -domain sockets stand in as simulated networks. This simulation is imperfect, -but it is much easier to set up, does not require extra physical or virtual -hardware, and does not require supervisor privileges. - -To run OFTest with Open vSwitch, first read and follow the instructions under -**Testing** above. Second, obtain a copy of OFTest and install its -prerequisites. You need a copy of OFTest that includes commit 406614846c5 (make -ovs-dummy platform work again). This commit was merged into the OFTest -repository on Feb 1, 2013, so any copy of OFTest more recent than that should -work. Testing OVS in dummy mode does not require root privilege, so you may -ignore that requirement. - -Optionally, add the top-level OFTest directory (containing the ``oft`` program) -to your ``$PATH``. This slightly simplifies running OFTest later. - -To run OFTest in dummy mode, run the following command from your Open vSwitch -build directory:: - - $ make check-oftest OFT= - -where ```` is the absolute path to the ``oft`` program in OFTest. -If you added "oft" to your $PATH, you may omit the OFT variable -assignment - -By default, ``check-oftest`` passes ``oft`` just enough options to enable dummy -mode. You can use ``OFTFLAGS`` to pass additional options. For example, to run -just the ``basic.Echo`` test instead of all tests (the default) and enable -verbose logging, run:: - - $ make check-oftest OFT= OFTFLAGS='--verbose -T basic.Echo' - -If you use OFTest that does not include commit 4d1f3eb2c792 (oft: change -default port to 6653), merged into the OFTest repository in October 2013, then -you need to add an option to use the IETF-assigned controller port:: - - $ make check-oftest OFT= OFTFLAGS='--port=6653' - -Interpret OFTest results cautiously. Open vSwitch can fail a given test in -OFTest for many reasons, including bugs in Open vSwitch, bugs in OFTest, bugs -in the "dummy mode" integration, and differing interpretations of the OpenFlow -standard and other standards. - -.. note:: - Open vSwitch has not been validated against OFTest. Report test failures that - you believe to represent bugs in Open vSwitch. Include the precise versions - of Open vSwitch and OFTest in your bug report, plus any other information - needed to reproduce the problem. - -Ryu -~~~ - -Ryu is an OpenFlow controller written in Python that includes an extensive -OpenFlow testsuite. Open vSwitch includes a Makefile target to run Ryu in -"dummy mode". See **OFTest** above for an explanation of dummy mode. - -To run Ryu tests with Open vSwitch, first read and follow the instructions -under **Testing** above. Second, obtain a copy of Ryu, install its -prerequisites, and build it. You do not need to install Ryu (some of the tests -do not get installed, so it does not help). - -To run Ryu tests, run the following command from your Open vSwitch build -directory:: - - $ make check-ryu RYUDIR=`` - -where ```` is the absolute path to the root of the Ryu source -distribution. The default ```` is ``$srcdir/../ryu`` -where ``$srcdir`` is your Open vSwitch source directory. If this is correct, -omit ``RYUDIR`` - -.. note:: - Open vSwitch has not been validated against Ryu. Report test failures that - you believe to represent bugs in Open vSwitch. Include the precise versions - of Open vSwitch and Ryu in your bug report, plus any other information - needed to reproduce the problem. - -Datapath testing -~~~~~~~~~~~~~~~~ - -Open vSwitch includes a suite of tests specifically for datapath functionality, -which can be run against the userspace or kernel datapaths. If you are -developing datapath features, it is recommended that you use these tests and -build upon them to verify your implementation. - -The datapath tests make some assumptions about the environment. They must be -run under root privileges on a Linux system with support for network -namespaces. For ease of use, the OVS source tree includes a vagrant box to -invoke these tests. Running the tests inside Vagrant provides kernel isolation, -protecting your development host from kernel panics or configuration conflicts -in the testsuite. If you wish to run the tests without using the vagrant box, -there are further instructions below. - -Vagrant -+++++++ - -.. important:: - - Requires Vagrant (version 1.7.0 or later) and a compatible hypervisor - -.. note:: - You must **Bootstrap** and **Configure** the sources before you run the steps - described here. - -A Vagrantfile is provided allowing to compile and provision the source tree as -found locally in a virtual machine using the following command:: - - $ vagrant up - -This will bring up a Fedora 23 VM by default. If you wish to use a different -box or a vagrant backend not supported by the default box, the ``Vagrantfile`` -can be modified to use a different box as base. - -The VM can be reprovisioned at any time:: - - $ vagrant provision - -OVS out-of-tree compilation environment can be set up with:: - - $ ./boot.sh - $ vagrant provision --provision-with configure_ovs,build_ovs - -This will set up an out-of-tree build environment inside the VM in -``/root/build``. The source code can be found in ``/vagrant``. - -To recompile and reinstall OVS in the VM using RPM:: - - $ ./boot.sh - $ vagrant provision --provision-with configure_ovs,install_rpm - -Two provisioners are included to run system tests with the OVS kernel module or -with a userspace datapath. This tests are different from the self-tests -mentioned above. To run them:: - - $ ./boot.sh - $ vagrant provision --provision-with \ - configure_ovs,test_ovs_kmod,test_ovs_system_userspace - -The results of the testsuite reside in the VM root user's home directory:: - - $ vagrant ssh - $ sudo -s - $ cd /root/build - $ ls tests/system* - -Native -++++++ - -The datapath testsuite as invoked by Vagrant above may also be run manually on -a Linux system with root privileges. These tests may take several minutes to -complete, and cannot be run in parallel. - -Userspace datapath -''''''''''''''''''' - -To invoke the datapath testsuite with the userspace datapath, run:: - - $ make check-system-userspace - -The results of the testsuite are in ``tests/system-userspace-traffic.dir``. - -Kernel datapath -''''''''''''''' - -Make targets are also provided for testing the Linux kernel module. Note that -these tests operate by inserting modules into the running Linux kernel, so if -the tests are able to trigger a bug in the OVS kernel module or in the upstream -kernel then the kernel may panic. - -To run the testsuite against the kernel module which is currently installed on -your system, run:: - - $ make check-kernel - -To install the kernel module from the current build directory and run the -testsuite against that kernel module:: - - $ make check-kmod - -The results of the testsuite are in ``tests/system-kmod-traffic.dir``. - -Continuous Integration with Travis-CI -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A .travis.yml file is provided to automatically build Open vSwitch with various -build configurations and run the testsuite using travis-ci. Builds will be -performed with gcc, sparse and clang with the -Werror compiler flag included, -therefore the build will fail if a new warning has been introduced. - -The CI build is triggered via git push (regardless of the specific branch) or -pull request against any Open vSwitch GitHub repository that is linked to -travis-ci. - -Instructions to setup travis-ci for your GitHub repository: - -1. Go to http://travis-ci.org/ and sign in using your GitHub ID. -2. Go to the "Repositories" tab and enable the ovs repository. You may disable - builds for pushes or pull requests. -3. In order to avoid forks sending build failures to the upstream mailing list, - the notification email recipient is encrypted. If you want to receive email - notification for build failures, replace the the encrypted string: - - 1. Install the travis-ci CLI (Requires ruby >=2.0): gem install travis - 2. In your Open vSwitch repository: travis encrypt mylist@mydomain.org - 3. Add/replace the notifications section in .travis.yml and fill in the - secure string as returned by travis encrypt:: - - notifications: - email: - recipients: - - secure: "....." - - .. note:: - You may remove/omit the notifications section to fall back to default - notification behaviour which is to send an email directly to the author and - committer of the failing commit. Note that the email is only sent if the - author/committer have commit rights for the particular GitHub repository. - -4. Pushing a commit to the repository which breaks the build or the - testsuite will now trigger a email sent to mylist@mydomain.org - -Static Code Analysis -~~~~~~~~~~~~~~~~~~~~ - -Static Analysis is a method of debugging Software by examining code rather than -actually executing it. This can be done through 'scan-build' commandline -utility which internally uses clang (or) gcc to compile the code and also -invokes a static analyzer to do the code analysis. At the end of the build, the -reports are aggregated in to a common folder and can later be analyzed using -'scan-view'. - -Open vSwitch includes a Makefile target to trigger static code analysis:: - - $ ./boot.sh - $ ./configure CC=clang # clang - # or - $ ./configure CC=gcc CFLAGS="-std=gnu99" # gcc - $ make clang-analyze - -You should invoke scan-view to view analysis results. The last line of output -from ``clang-analyze`` will list the command (containing results directory) -that you should invoke to view the results on a browser. - -Bug Reporting -------------- - -Please report problems to bugs@openvswitch.org. diff --git a/INSTALL.userspace.rst b/INSTALL.userspace.rst deleted file mode 100644 index 10223555b..000000000 --- a/INSTALL.userspace.rst +++ /dev/null @@ -1,106 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -=================================== -Open vSwitch Without Kernel Support -=================================== - -Open vSwitch can operate, at a cost in performance, entirely in userspace, -without assistance from a kernel module. This file explains how to install -Open vSwitch in such a mode. - -This version of Open vSwitch should be built manually with ``configure`` and -``make``. Debian packaging for Open vSwitch is also included, but it has not -been recently tested, and so Debian packages are not a recommended way to use -this version of Open vSwitch. - -.. warning:: - The userspace-only mode of Open vSwitch is considered experimental. It has - not been thoroughly tested. - -Building and Installing ------------------------ - -The requirements and procedure for building, installing, and configuring Open -vSwitch are the same as those given in the `installation guide -`__. You may omit configuring, building, and installing the -kernel module, and the related requirements. - -On Linux, the userspace switch additionally requires the kernel TUN/TAP driver -to be available, either built into the kernel or loaded as a module. If you -are not sure, check for a directory named ``/sys/class/misc/tun``. If it does -not exist, then attempt to load the module with ``modprobe tun``. - -The tun device must also exist as ``/dev/net/tun``. If it does not exist, then -create ``/dev/net`` (if necessary) with ``mkdir /dev/net``, then create -``/dev/net/tun`` with ``mknod /dev/net/tun c 10 200``. - -On FreeBSD and NetBSD, the userspace switch additionally requires the kernel -tap(4) driver to be available, either built into the kernel or loaded as a -module. - -Using the Userspace Datapath with ovs-vswitchd ----------------------------------------------- - -To use ovs-vswitchd in userspace mode, create a bridge with -``datapath_type=netdev`` in the configuration database. For example:: - - $ ovs-vsctl add-br br0 - $ ovs-vsctl set bridge br0 datapath_type=netdev - $ ovs-vsctl add-port br0 eth0 - $ ovs-vsctl add-port br0 eth1 - $ ovs-vsctl add-port br0 eth2 - -ovs-vswitchd will create a TAP device as the bridge's local interface, named -the same as the bridge, as well as for each configured internal interface. - -Currently, on FreeBSD, the functionality required for in-band control support -is not implemented. To avoid related errors, you can disable the in-band -support with the following command:: - - $ ovs-vsctl set bridge br0 other_config:disable-in-band=true - -Firewall Rules --------------- - -On Linux, when a physical interface is in use by the userspace datapath, -packets received on the interface still also pass into the kernel TCP/IP stack. -This can cause surprising and incorrect behavior. You can use "iptables" to -avoid this behavior, by using it to drop received packets. For example, to -drop packets received on eth0:: - - $ iptables -A INPUT -i eth0 -j DROP - $ iptables -A FORWARD -i eth0 -j DROP - -Other Settings --------------- - -On NetBSD, depending on your network topology and applications, the following -configuration might help. See sysctl(7).:: - - $ sysctl -w net.inet.ip.checkinterface=1 - -Bug Reporting -------------- - -Please report problems to bugs@openvswitch.org. diff --git a/Makefile.am b/Makefile.am index 0036fecff..df4214952 100644 --- a/Makefile.am +++ b/Makefile.am @@ -70,21 +70,6 @@ docs = \ CONTRIBUTING.rst \ DESIGN.rst \ FAQ.rst \ - INSTALL.rst \ - INSTALL.Debian.rst \ - INSTALL.Docker.rst \ - INSTALL.DPDK-ADVANCED.rst \ - INSTALL.DPDK.rst \ - INSTALL.Fedora.rst \ - INSTALL.KVM.rst \ - INSTALL.Libvirt.rst \ - INSTALL.NetBSD.rst \ - INSTALL.RHEL.rst \ - INSTALL.SELinux.rst \ - INSTALL.SSL.rst \ - INSTALL.XenServer.rst \ - INSTALL.userspace.rst \ - INSTALL.Windows.rst \ IntegrationGuide.rst \ MAINTAINERS.rst \ OPENFLOW.rst \ diff --git a/PORTING.rst b/PORTING.rst index 33db41de7..bae8cd9cd 100644 --- a/PORTING.rst +++ b/PORTING.rst @@ -141,13 +141,15 @@ Porting Strategies After a netdev provider has been implemented for a system's network devices, you may choose among three basic porting strategies. +.. TODO(stephenfin): Update the link to the installation guide when this is + moved + The lowest-effort strategy is to use the "userspace switch" implementation built into Open vSwitch. This ought to work, without writing any more code, as long as the netdev provider that you implemented supports receiving packets. It yields poor performance, however, because every packet passes through the -ovs-vswitchd process. See the `userspace installation guide -` for instructions on how to configure a userspace -switch. +ovs-vswitchd process. See the `userspace installation guide` for instructions +on how to configure a userspace switch. If the userspace switch is not the right choice for your port, then you will have to write more code. You may implement either an "ofproto provider" or a diff --git a/README.rst b/README.rst index 715f969e5..c8227f80b 100644 --- a/README.rst +++ b/README.rst @@ -73,36 +73,16 @@ Open vSwitch also provides some tools: What other documentation is available? -------------------------------------- -To install Open vSwitch on a regular Linux or FreeBSD host, please read the -`installation guide `__. For specifics around installation on a -specific platform, please see one of the below installation guides: - -- `Debian `__ -- `Fedora `__ -- `RHEL `__ -- `XenServer `__ -- `Windows `__ - -To use Open vSwitch... - -- ...with Docker on Linux, see `here `__. - -- ...with KVM on Linux, see `here `__ and `here - `__. +.. TODO(stephenfin): Update with a link to the hosting site of the docs, once + we know where that is -- ...with Libvirt, see `here `__. - -- ...without using a kernel module, see `here `__. - -- ...with DPDK, see `here `__. - -- ...with SELinux, see `here `__. +To install Open vSwitch on a regular Linux or FreeBSD host, please read the +`installation guide `__. For specifics +around installation on a specific platform, refer to one of the `other +installation guides `__ For answers to common questions, refer to the `FAQ `__. -To learn how to set up SSL support for Open vSwitch, see `here -`__. - To learn about some advanced features of the Open vSwitch software switch, read the `tutorial `__. diff --git a/debian/openvswitch-common.docs b/debian/openvswitch-common.docs index c5612ddee..27f51626a 100644 --- a/debian/openvswitch-common.docs +++ b/debian/openvswitch-common.docs @@ -1,3 +1 @@ FAQ.rst -INSTALL.DPDK.rst -README-native-tunneling.rst diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in index d1f33af30..d70934a48 100644 --- a/rhel/openvswitch-fedora.spec.in +++ b/rhel/openvswitch-fedora.spec.in @@ -481,8 +481,8 @@ fi %{_mandir}/man8/ovs-vswitchd.8* %{_mandir}/man8/ovs-parse-backtrace.8* %{_mandir}/man8/ovs-testcontroller.8* -%doc COPYING DESIGN.rst INSTALL.SSL.rst NOTICE README.rst WHY-OVS.rst -%doc FAQ.rst NEWS INSTALL.DPDK.rst rhel/README.RHEL.rst +%doc COPYING DESIGN.rst NOTICE README.rst WHY-OVS.rst +%doc FAQ.rst NEWS rhel/README.RHEL.rst /var/lib/openvswitch /var/log/openvswitch %ghost %attr(755,root,root) %{_rundir}/openvswitch diff --git a/rhel/openvswitch.spec.in b/rhel/openvswitch.spec.in index 846f8e0b9..261736803 100644 --- a/rhel/openvswitch.spec.in +++ b/rhel/openvswitch.spec.in @@ -248,8 +248,8 @@ exit 0 /usr/share/openvswitch/scripts/sysconfig.template /usr/share/openvswitch/vswitch.ovsschema /usr/share/openvswitch/vtep.ovsschema -%doc COPYING DESIGN.rst INSTALL.SSL.rst NOTICE README.rst WHY-OVS.rst FAQ.rst NEWS -%doc INSTALL.DPDK.rst rhel/README.RHEL.rst README-native-tunneling.rst +%doc COPYING DESIGN.rst NOTICE README.rst WHY-OVS.rst FAQ.rst NEWS +%doc rhel/README.RHEL.rst README-native-tunneling.rst /var/lib/openvswitch /var/log/openvswitch diff --git a/tutorial/ovn-tutorial.rst b/tutorial/ovn-tutorial.rst index 8d9fc1a43..bb5f6201f 100644 --- a/tutorial/ovn-tutorial.rst +++ b/tutorial/ovn-tutorial.rst @@ -873,11 +873,12 @@ of what the resulting OpenFlow flows look like. Container Ports --------------- +.. TODO(stephenfin): Update Docker link when this is moved. + OVN supports containers running directly on the hypervisors and running containers inside VMs. This example shows how OVN supports network virtualization to containers when run inside VMs. Details about how to use -docker containers in OVS can be found in the `Docker installlation guide -<../INSTALL.Docker.rst>`__. +docker containers in OVS can be found in the `Docker installlation guide`. To support container traffic created inside a VM and to distinguish network traffic coming from different container vifs, for each container a logical port diff --git a/tutorial/tutorial.rst b/tutorial/tutorial.rst index fab635030..422bc0d61 100644 --- a/tutorial/tutorial.rst +++ b/tutorial/tutorial.rst @@ -54,6 +54,8 @@ hardware or even supervisor privilege on your system. Instead, we will use a script called ``ovs-sandbox``, which accompanies the tutorial, that constructs a software simulated network environment based on Open vSwitch. +.. TODO(stephenfin): Update installation guide link when this is moved. + You can use ``ovs-sandbox`` three ways: * If you have already installed Open vSwitch on your system, then you should be @@ -61,9 +63,9 @@ You can use ``ovs-sandbox`` three ways: * If you have not installed Open vSwitch (and you do not want to install it), then you can build Open vSwitch according to the instructions in the - `installation guide `__, without installing it. Then run - ``./ovs-sandbox -b DIRECTORY`` from this directory, substituting the Open - vSwitch build directory for ``DIRECTORY``. + `installation guide`, without installing it. Then run ``./ovs-sandbox -b + DIRECTORY`` from this directory, substituting the Open vSwitch build + directory for ``DIRECTORY``. * As a slight variant on the latter, you can run ``make sandbox`` from an Open vSwitch build directory. diff --git a/utilities/automake.mk b/utilities/automake.mk index 014e477d1..2590dc0e6 100644 --- a/utilities/automake.mk +++ b/utilities/automake.mk @@ -37,11 +37,9 @@ noinst_SCRIPTS += utilities/ovs-sim utilities/ovs-lib: $(top_builddir)/config.status -docs += utilities/ovs-command-bashcomp.INSTALL.rst EXTRA_DIST += \ utilities/ovs-appctl-bashcomp.bash \ utilities/ovs-check-dead-ifs.in \ - utilities/ovs-command-bashcomp.INSTALL.rst \ utilities/ovs-ctl.in \ utilities/ovs-dev.py \ utilities/ovs-docker \ diff --git a/utilities/ovs-command-bashcomp.INSTALL.rst b/utilities/ovs-command-bashcomp.INSTALL.rst deleted file mode 100644 index 6df60c65c..000000000 --- a/utilities/ovs-command-bashcomp.INSTALL.rst +++ /dev/null @@ -1,111 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -==================================== -Bash command-line completion scripts -==================================== - -There are two completion scripts available, ovs-appctl-bashcomp.bash and -ovs-vsctl-bashcomp.bash respectively. - -ovs-appctl-bashcomp -------------------- - -ovs-appctl-bashcomp.bash adds bash command-line completion support for -ovs-appctl, ovs-dpctl, ovs-ofctl and ovsdb-tool commands. - -Features -~~~~~~~~ - -- Display available completion or complete on unfinished user input (long - option, subcommand, and argument). - -- Subcommand hints - -- Convert between keywords like ``bridge``, ``port``, ``interface``, or ``dp`` - and the available record in ovsdb. - -Limitations -~~~~~~~~~~~ - -- Only supports a small set of important keywords (``dp``, ``datapath``, ``bridge``, ``switch``, - ``port``, ``interface``, ``iface``). - -- Does not support parsing of nested options. For example: - - - :: - - $ ovsdb-tool create [db [schema]] - -- Does not support expansion on repeated argument. For example: - - :: - - $ ovs-dpctl show [dp...]). - -- Only supports matching on long options, and only in the format ``--option - [arg]``. Do not use ``--option=[arg]``. - -ovs-vsctl-bashcomp -------------------- - -ovs-vsctl-bashcomp.bash adds bash command-line completion support for ovs-vsctl -command. - -Features -~~~~~~~~ - -- Display available completion and complete on user input for global/local - options, command, and argument. - -- Query database and expand keywords like ``table``, ``record``, ``column``, or - ``key``, to available completions. - -- Deal with argument relations like 'one and more', 'zero or one'. - -- Complete multiple ovs-vsctl commands cascaded via ``--``. - -Limitations -~~~~~~~~~~~ - -Completion of very long ``ovs-vsctl`` commands can take up to several seconds. - -Usage ------ - -The bashcomp scripts should be placed at ``/etc/bash_completion.d/`` to be -available for all bash sessions. Running ``make install`` will place the -scripts to ``$(sysconfdir)/bash_completion.d/``, thus, the user should specify -``--sysconfdir=/etc`` at configuration. If OVS is installed from packages, the -scripts will automatically be placed inside ``/etc/bash_completion.d/``. - -If you just want to run the scripts in one bash, you can remove them from -``/etc/bash_completion.d/`` and run the scripts via ``. -ovs-appctl-bashcomp.bash`` or ``. ovs-vsctl-bashcomp.bash``. - -Tests ------ - -Unit tests are added in ``tests/completion.at`` and integrated into autotest -framework. To run the tests, just run ``make check``. diff --git a/utilities/ovs-ctl.8 b/utilities/ovs-ctl.8 index cd7c2674f..1fe0a3c94 100644 --- a/utilities/ovs-ctl.8 +++ b/utilities/ovs-ctl.8 @@ -502,5 +502,4 @@ distribution are good examples of how to use \fBovs\-ctl\fR. . .SH "SEE ALSO" . -\fBREADME.rst\fR, \fBINSTALL.Linux.rst\fR, \fBovsdb\-server\fR(8), -\fBovs\-vswitchd\fR(8). +\fBREADME.rst\fR, \fBovsdb\-server\fR(8), \fBovs\-vswitchd\fR(8). diff --git a/utilities/ovs-vsctl.8.in b/utilities/ovs-vsctl.8.in index 41288018c..19179060e 100644 --- a/utilities/ovs-vsctl.8.in +++ b/utilities/ovs-vsctl.8.in @@ -399,9 +399,9 @@ Sets the configured failure mode. These commands manipulate the \fBmanager_options\fR column in the \fBOpen_vSwitch\fR table and rows in the \fBManagers\fR table. When \fBovsdb\-server\fR is configured to use the \fBmanager_options\fR column for -OVSDB connections (as described in \fBINSTALL.Linux\fR and in the startup -scripts provided with Open vSwitch), this allows the administrator to use -\fBovs\-vsctl\fR to configure database connections. +OVSDB connections (as described in the startup scripts provided with +Open vSwitch), this allows the administrator to use \fBovs\-vsctl\fR to +configure database connections. . .IP "\fBget\-manager\fR" Prints the configured manager(s). diff --git a/vswitchd/ovs-vswitchd.8.in b/vswitchd/ovs-vswitchd.8.in index c745dbc8b..fc19f3b32 100644 --- a/vswitchd/ovs-vswitchd.8.in +++ b/vswitchd/ovs-vswitchd.8.in @@ -67,9 +67,8 @@ to modify datapaths when \fBovs\-vswitchd\fR is running can interfere with its operation. (\fBovs\-dpctl\fR may still be useful for diagnostics.) .PP An Open vSwitch datapath kernel module must be loaded for \fBovs\-vswitchd\fR -to be useful. Please refer to the \fBINSTALL.Linux\fR file included in the -Open vSwitch distribution for instructions on how to build and load -the Open vSwitch kernel module. +to be useful. Refer to the documentation for instructions on how to build and +load the Open vSwitch kernel module. .PP .SH OPTIONS .IP "\fB\-\-mlockall\fR" @@ -85,7 +84,7 @@ unavailable or unsuccessful. . .SS "DPDK Options" For details on initializing the \fBovs\-vswitchd\fR DPDK datapath, -refer to INSTALL.DPDK.rst or \fBovs\-vswitchd.conf.db\fR(5) for +refer to the documentation or \fBovs\-vswitchd.conf.db\fR(5) for details. .SS "Daemon Options" .ds DD \ @@ -348,5 +347,4 @@ time linear in the number of flows. . .SH "SEE ALSO" .BR ovs\-appctl (8), -.BR ovsdb\-server (1), -\fBINSTALL.Linux\fR in the Open vSwitch distribution. +.BR ovsdb\-server (1). diff --git a/vtep/README.ovs-vtep.rst b/vtep/README.ovs-vtep.rst index 75f03d0b7..668ee08f2 100644 --- a/vtep/README.ovs-vtep.rst +++ b/vtep/README.ovs-vtep.rst @@ -36,13 +36,12 @@ The VTEP emulator is a Python script that invokes calls to tools like vtep-ctl and ovs-vsctl. It is only useful when Open vSwitch daemons like ovsdb-server and ovs-vswitchd are running and installed. To do this, either: -- Follow the instructions in the `installation guide <..INSTALL.rst>`__ (don't +- Follow the instructions in ``Documentation/intro/install/general`` (don't start any daemons yet). -- Follow the instructions in `Debian installation guide - <../INSTALL.Debian.rst>`__ and then install the ``openvswitch-vtep`` package - (if operating on a debian based machine). This will automatically start the - daemons. +- Follow the instructions in ``Documentation/intro/install/debian`` and then + install the ``openvswitch-vtep`` package (if operating on a debian based + machine). This will automatically start the daemons. Design ------ @@ -152,9 +151,9 @@ using the debian packages as mentioned in step 2 of the "Requirements" section. $ vtep-ctl add-ps br0 $ vtep-ctl set Physical_Switch br0 tunnel_ips=10.2.2.1 -6. Start the VTEP emulator. If you installed the components following the - `installation guide <../INSTALL.rst>`__ file, run the following from the - same directory as this README: +6. Start the VTEP emulator. If you installed the components following + ``Documentation/intro/install/general``, run the following from the same + directory as this README: :: diff --git a/vtep/vtep-ctl.8.in b/vtep/vtep-ctl.8.in index e0f49a78d..190135665 100644 --- a/vtep/vtep-ctl.8.in +++ b/vtep/vtep-ctl.8.in @@ -325,9 +325,9 @@ List the remote MAC bindings for \fIlswitch\fR, one per line. These commands manipulate the \fBmanagers\fR column in the \fBGlobal\fR table and rows in the \fBManagers\fR table. When \fBovsdb\-server\fR is configured to use the \fBmanagers\fR column for OVSDB connections (as -described in \fBINSTALL.Linux\fR and in the startup scripts provided -with Open vSwitch), this allows the administrator to use \fBvtep\-ctl\fR -to configure database connections. +described in the startup scripts provided with Open vSwitch), this +allows the administrator to use \fBvtep\-ctl\fR to configure database +connections. . .IP "\fBget\-manager\fR" Prints the configured manager(s). -- cgit v1.2.1