Add a tutorial for advanced Open vSwitch features.

Signed-off-by: Ben Pfaff <blp@nicira.com>

10 files changed, 1141 insertions, 0 deletions

diff --git a/tutorial/.gitignore b/tutorial/.gitignore
new file mode 100644
index 000000000..97f3e3127
--- /dev/null
+++ b/tutorial/.gitignore
@@ -0,0 +1 @@
+sandbox/
diff --git a/tutorial/Tutorial b/tutorial/Tutorial
new file mode 100644
index 000000000..b8dfc5488
--- /dev/null
+++ b/tutorial/Tutorial
@@ -0,0 +1,835 @@
+Open vSwitch Advanced Features Tutorial
+=======================================
+
+Many tutorials cover the basics of OpenFlow.  This is not such a
+tutorial.  Rather, a knowledge of the basics of OpenFlow is a
+prerequisite.  If you do not already understand how an OpenFlow flow
+table works, please go read a basic tutorial and then continue reading
+here afterward.
+
+It is also important to understand the basics of Open vSwitch before
+you begin.  If you have never used ovs-vsctl or ovs-ofctl before, you
+should learn a little about them before proceeding.
+
+Most of the features covered in this tutorial are Open vSwitch
+extensions to OpenFlow.  Also, most of the features in this tutorial
+are specific to the software Open vSwitch implementation.  If you are
+using an Open vSwitch port to an ASIC-based hardware switch, this
+tutorial will not help you.
+
+This tutorial does not cover every aspect of the features that it
+mentions.  You can find the details elsewhere in the Open vSwitch
+documentation, especially ovs-ofctl(8) and the comments in the
+include/openflow/nicira-ext.h header file.
+
+>>> In this tutorial, paragraphs set off like this designate notes
+    with additional information that readers may wish to skip on a
+    first read.
+
+
+Getting Started
+===============
+
+This is a hands-on tutorial.  To get the most out of it, you will need
+Open vSwitch binaries.  You do not, on the other hand, need any
+physical networking hardware or even supervisor privilege on your
+system.  Instead, we will use a script called "ovs-sandbox", which
+accompanies the tutorial, that constructs a software simulated network
+environment based on Open vSwitch.
+
+You can use "ovs-sandbox" three ways:
+
+    * If you have already installed Open vSwitch on your system, then
+      you should be able to just run "ovs-sandbox" from this directory
+      without any options.
+
+    * If you have not installed Open vSwitch (and you do not want to
+      install it), then you can build Open vSwitch according to the
+      instructions in INSTALL, without installing it.  Then run
+      "./ovs-sandbox -b DIRECTORY" from this directory, substituting
+      the Open vSwitch build directory for DIRECTORY.
+
+    * As a slight variant on the latter, you can run "make sandbox"
+      from an Open vSwitch build directory.
+
+When you run ovs-sandbox, it does the following:
+
+    1. CAUTION: Deletes any subdirectory of the current directory
+       named "sandbox" and any files in that directory.
+
+    2. Creates a new directory "sandbox" in the current directory.
+
+    3. Sets up special environment variables that ensure that Open
+       vSwitch programs will look inside the "sandbox" directory
+       instead of in the Open vSwitch installation directory.
+
+    4. If you are using a built but not installed Open vSwitch,
+       installs the Open vSwitch manpages in a subdirectory of
+       "sandbox" and adjusts the MANPATH environment variable to point
+       to this directory.  This means that you can use, for example,
+       "man ovs-vsctl" to see a manpage for the ovs-vsctl program that
+       you built.
+
+    5. Creates an empty Open vSwitch configuration database under
+       "sandbox".
+
+    6. Starts ovsdb-server running under "sandbox".
+
+    7. Starts ovs-vswitchd running under "sandbox", passing special
+       options that enable a special "dummy" mode for testing.
+
+    8. Starts a nested interactive shell inside "sandbox".
+
+At this point, you can run all the usual Open vSwitch utilities from
+the nested shell environment.  You can, for example, use ovs-vsctl to
+create a bridge:
+
+    ovs-vsctl add-br br0
+
+From Open vSwitch's perspective, the bridge that you create this way
+is as real as any other.  You can, for example, connect it to an
+OpenFlow controller or use "ovs-ofctl" to examine and modify it and
+its OpenFlow flow table.  On the other hand, the bridge is not visible
+to the operating system's network stack, so "ifconfig" or "ip" cannot
+see it or affect it, which means that utilities like "ping" and
+"tcpdump" will not work either.  (That has its good side, too: you
+can't screw up your computer's network stack by manipulating a
+sandboxed OVS.)
+
+When you're done using OVS from the sandbox, exit the nested shell (by
+entering the "exit" shell command or pressing Control+D).  This will
+kill the daemons that ovs-sandbox started, but it leaves the "sandbox"
+directory and its contents in place.
+
+The sandbox directory contains log files for the Open vSwitch dameons.
+You can examine them while you're running in the sandboxed environment
+or after you exit.
+
+
+Motivation
+==========
+
+The goal of this tutorial is to demonstrate the power of Open vSwitch
+flow tables.  The tutorial works through the implementation of a
+MAC-learning switch with VLAN trunk and access ports.  Outside of the
+Open vSwitch features that we will discuss, OpenFlow provides at least
+two ways to implement such a switch:
+
+    1. An OpenFlow controller to implement MAC learning in a
+       "reactive" fashion.  Whenever a new MAC appears on the switch,
+       or a MAC moves from one switch port to another, the controller
+       adjusts the OpenFlow flow table to match.
+
+    2. The "normal" action.  OpenFlow defines this action to submit a
+       packet to "the traditional non-OpenFlow pipeline of the
+       switch".  That is, if a flow uses this action, then the packets
+       in the flow go through the switch in the same way that they
+       would if OpenFlow was not configured on the switch.
+
+Each of these approaches has unfortunate pitfalls.  In the first
+approach, using an OpenFlow controller to implement MAC learning, has
+a significant cost in terms of network bandwidth and latency.  It also
+makes the controller more difficult to scale to large numbers of
+switches, which is especially important in environments with thousands
+of hypervisors (each of which contains a virtual OpenFlow switch).
+MAC learning at an OpenFlow controller also behaves poorly if the
+OpenFlow controller fails, slows down, or becomes unavailable due to
+network problems.
+
+The second approach, using the "normal" action, has different
+problems.  First, little about the "normal" action is standardized, so
+it behaves differently on switches from different vendors, and the
+available features and how those features are configured (usually not
+through OpenFlow) varies widely.  Second, "normal" does not work well
+with other OpenFlow actions.  It is "all-or-nothing", with little
+potential to adjust its behavior slightly or to compose it with other
+features.
+
+
+Scenario
+========
+
+We will construct Open vSwitch flow tables for a VLAN-capable,
+MAC-learning switch that has four ports:
+
+    * p1, a trunk port that carries all VLANs, on OpenFlow port 1.
+
+    * p2, an access port for VLAN 20, on OpenFlow port 2.
+
+    * p3 and p4, both access ports for VLAN 30, on OpenFlow ports 3
+      and 4, respectively.
+
+>>> The ports' names are not significant.  You could call them eth1
+    through eth4, or any other names you like.
+
+>>> An OpenFlow switch always has a "local" port as well.  This
+    scenario won't use the local port.
+
+Our switch design will consist of five main flow tables, each of which
+implements one stage in the switch pipeline:
+
+    Table 0: Admission control.
+
+    Table 1: VLAN input processing.
+
+    Table 2: Learn source MAC and VLAN for ingress port.
+
+    Table 3: Look up learned port for destination MAC and VLAN.
+
+    Table 4: Output processing.
+
+The section below describes how to set up the scenario, followed by a
+section for each OpenFlow table.
+
+You can cut and paste the "ovs-vsctl" and "ovs-ofctl" commands in each
+of the sections below into your "ovs-sandbox" shell.  They are also
+available as shell scripts in this directory, named t-setup, t-stage0,
+t-stage1, ..., t-stage4.  The "ovs-appctl" test commands are intended
+for cutting and pasting and are not supplied separately.
+
+
+Setup
+=====
+
+To get started, start "ovs-sandbox".  Inside the interactive shell
+that it starts, run this command:
+
+    ovs-vsctl add-br br0 -- set Bridge br0 fail-mode=secure
+
+This command creates a new bridge "br0" and puts "br0" into so-called
+"fail-secure" mode.  For our purpose, this just means that the
+OpenFlow flow table starts out empty.
+
+>>> If we did not do this, then the flow table would start out with a
+    single flow that executes the "normal" action.  We could use that
+    feature to yield a switch that behaves the same as the switch we
+    are currently building, but with the caveats described under
+    "Motivation" above.)
+
+The new bridge has only one port on it so far, the "local port" br0.
+We need to add p1, p2, p3, and p4.  A shell "for" loop is one way to
+do it:
+
+    for i in 1 2 3 4; do
+        ovs-vsctl add-port br0 p$i -- set Interface p$i ofport_request=$i
+	ovs-ofctl mod-port br0 p$i up
+    done
+
+In addition to adding a port, the ovs-vsctl command above sets its
+"ofport_request" column to ensure that port p1 is assigned OpenFlow
+port 1, p2 is assigned OpenFlow port 2, and so on.
+
+>>> We could omit setting the ofport_request and let Open vSwitch
+    choose port numbers for us, but it's convenient for the purposes
+    of this tutorial because we can talk about OpenFlow port 1 and
+    know that it corresponds to p1.
+
+The ovs-ofctl command above brings up the simulated interfaces, which
+are down initially, using an OpenFlow request.  The effect is similar
+to "ifconfig up", but the sandbox's interfaces are not visible to the
+operating system and therefore "ifconfig" would not affect them.
+
+We have not configured anything related to VLANs or MAC learning.
+That's because we're going to implement those features in the flow
+table.
+
+To see what we've done so far to set up the scenario, you can run a
+command like "ovs-vsctl show" or "ovs-ofctl show br0".
+
+
+Implementing Table 0: Admission control
+=======================================
+
+Table 0 is where packets enter the switch.  We use this stage to
+discard packets that for one reason or another are invalid.  For
+example, packets with a multicast source address are not valid, so we
+can add a flow to drop them at ingress to the switch with:
+
+    ovs-ofctl add-flow br0 \
+        "table=0, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=drop"
+
+A switch should also not forward IEEE 802.1D Spanning Tree Protocol
+(STP) packets, so we can also add a flow to drop those and other
+packets with reserved multicast protocols:
+
+    ovs-ofctl add-flow br0 \
+        "table=0, dl_dst=01:08:c2:00:00:00/ff:ff:ff:ff:ff:f0, actions=drop"
+
+We could add flows to drop other protocols, but these demonstrate the
+pattern.
+
+We need one more flow, with a priority lower than the default, so that
+flows that don't match either of the "drop" flows we added above go on
+to pipeline stage 1 in OpenFlow table 1:
+
+    ovs-ofctl add-flow br0 "table=0, priority=0, actions=resubmit(,1)"
+
+(The "resubmit" action is an Open vSwitch extension to OpenFlow.)
+
+
+Testing Table 0
+---------------
+
+If we were using Open vSwitch to set up a physical or a virtual
+switch, then we would naturally test it by sending packets through it
+one way or another, perhaps with common network testing tools like
+"ping" and "tcpdump" or more specialized tools like Scapy.  That's
+difficult with our simulated switch, since it's not visible to the
+operating system.
+
+Bur our simulated switch has a few specialized testing tools.  The
+most powerful of these tools is "ofproto/trace".  Given a switch and
+the specification of a flow, "ofproto/trace" shows, step-by-step, how
+such a flow would be treated as it goes through the switch.
+
+
+== EXAMPLE 1 ==
+
+Try this command:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=01:08:c2:00:00:05
+
+The output should look something like this:
+
+    Flow: metadata=0,in_port=1,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=01:08:c2:00:00:05,dl_type=0x0000
+    Rule: table=0 cookie=0 dl_dst=01:08:c2:00:00:00/ff:ff:ff:ff:ff:f0
+    OpenFlow actions=drop
+
+    Final flow: unchanged
+    Datapath actions: drop
+
+The first block of lines describes an OpenFlow table lookup.  The
+first line shows the fields used for the table lookup (which is mostly
+zeros because that's the default if we don't specify everything).  The
+second line gives the OpenFlow flow that the fields matched (called a
+"rule" because that is the name used inside Open vSwitch for an
+OpenFlow flow).  In this case, we see that this packet that has a
+reserved multicast destination address matches the rule that drops
+those packets.  The third line gives the rule's OpenFlow actions.
+
+The second block of lines summarizes the results, which are not very
+interesting here.
+
+
+== EXAMPLE 2 ==
+
+Try another command:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=01:08:c2:00:00:10
+
+The output should be:
+
+    Flow: metadata=0,in_port=1,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=01:08:c2:00:00:10,dl_type=0x0000
+    Rule: table=0 cookie=0 priority=0
+    OpenFlow actions=resubmit(,1)
+
+	    Resubmitted flow: unchanged
+	    Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+	    Resubmitted  odp: drop
+	    No match
+
+    Final flow: unchanged
+    Datapath actions: drop
+
+This time the flow we handed to "ofproto/trace" doesn't match any of
+our "drop" rules, so it falls through to the low-priority "resubmit"
+rule, which we see in the rule and the actions selected in the first
+block.  The "resubmit" causes a second lookup in OpenFlow table 1,
+described by the additional block of indented text in the output.  We
+haven't yet added any flows to OpenFlow table 1, so no flow actually
+matches in the second lookup.  Therefore, the packet is still actually
+dropped, which means that the externally observable results would be
+identical to our first example.
+
+
+Implementing Table 1: VLAN Input Processing
+===========================================
+
+A packet that enters table 1 has already passed basic validation in
+table 0.  The purpose of table 1 is validate the packet's VLAN, based
+on the VLAN configuration of the switch port through which the packet
+entered the switch.  We will also use it to attach a VLAN header to
+packets that arrive on an access port, which allows later processing
+stages to rely on the packet's VLAN always being part of the VLAN
+header, reducing special cases.
+
+Let's start by adding a low-priority flow that drops all packets,
+before we add flows that pass through acceptable packets.  You can
+think of this as a "default drop" rule:
+
+    ovs-ofctl add-flow br0 "table=1, priority=0, actions=drop"
+
+Our trunk port p1, on OpenFlow port 1, is an easy case.  p1 accepts
+any packet regardless of whether it has a VLAN header or what the VLAN
+was, so we can add a flow that resubmits everything on input port 1 to
+the next table:
+
+    ovs-ofctl add-flow br0 \
+        "table=1, priority=99, in_port=1, actions=resubmit(,2)"
+
+On the access ports, we want to accept any packet that has no VLAN
+header, tag it with the access port's VLAN number, and then pass it
+along to the next stage:
+
+    ovs-ofctl add-flows br0 - <<'EOF'
+	table=1, priority=99, in_port=2, vlan_tci=0, actions=mod_vlan_vid:20, resubmit(,2)
+	table=1, priority=99, in_port=3, vlan_tci=0, actions=mod_vlan_vid:30, resubmit(,2)
+	table=1, priority=99, in_port=4, vlan_tci=0, actions=mod_vlan_vid:30, resubmit(,2)
+EOF
+
+We don't write any rules that match packets with 802.1Q that enter
+this stage on any of the access ports, so the "default drop" rule we
+added earlier causes them to be dropped, which is ordinarily what we
+want for access ports.
+
+>>> Another variation of access ports allows ingress of packets tagged
+    with VLAN 0 (aka 802.1p priority tagged packets).  To allow such
+    packets, replace "vlan_tci=0" by "vlan_tci=0/0xfff" above.
+
+
+Testing Table 1
+---------------
+
+"ofproto/trace" allows us to test the ingress VLAN rules that we added
+above.
+
+
+== EXAMPLE 1: Packet on Trunk Port ==
+
+Here's a test of a packet coming in on the trunk port:
+
+    ovs-appctl ofproto/trace br0 in_port=1,vlan_tci=5
+
+The output shows the lookup in table 0, the resubmit to table 1, and
+the resubmit to table 2 (which does nothing because we haven't put
+anything there yet):
+
+    Flow: metadata=0,in_port=1,vlan_tci=0x0005,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000
+    Rule: table=0 cookie=0 priority=0
+    OpenFlow actions=resubmit(,1)
+
+	    Resubmitted flow: unchanged
+	    Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+	    Resubmitted  odp: drop
+	    Rule: table=1 cookie=0 priority=99,in_port=1
+	    OpenFlow actions=resubmit(,2)
+
+		    Resubmitted flow: unchanged
+		    Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+		    Resubmitted  odp: drop
+		    No match
+
+    Final flow: unchanged
+    Datapath actions: drop
+
+
+== EXAMPLE 2: Valid Packet on Access Port ==
+
+Here's a test of a valid packet (a packet without an 802.1Q header)
+coming in on access port p2:
+
+    ovs-appctl ofproto/trace br0 in_port=2
+
+The output is similar to that for the previous case, except that it
+additionally tags the packet with p2's VLAN 20 before it passes it
+along to table 2:
+
+    Flow: metadata=0,in_port=2,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000
+    Rule: table=0 cookie=0 priority=0
+    OpenFlow actions=resubmit(,1)
+
+	    Resubmitted flow: unchanged
+	    Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+	    Resubmitted  odp: drop
+	    Rule: table=1 cookie=0 priority=99,in_port=2,vlan_tci=0x0000
+	    OpenFlow actions=mod_vlan_vid:20,resubmit(,2)
+
+		    Resubmitted flow: metadata=0,in_port=2,dl_vlan=20,dl_vlan_pcp=0,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000
+		    Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+		    Resubmitted  odp: drop
+		    No match
+
+    Final flow: unchanged
+    Datapath actions: drop
+
+
+== EXAMPLE 3: Invalid Packet on Access Port ==
+
+This tests an invalid packet (one that includes an 802.1Q header)
+coming in on access port p2:
+
+    ovs-appctl ofproto/trace br0 in_port=2,vlan_tci=5
+
+The output shows the packet matching the default drop rule:
+
+    Flow: metadata=0,in_port=2,vlan_tci=0x0005,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000
+    Rule: table=0 cookie=0 priority=0
+    OpenFlow actions=resubmit(,1)
+
+	    Resubmitted flow: unchanged
+	    Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+	    Resubmitted  odp: drop
+	    Rule: table=1 cookie=0 priority=0
+	    OpenFlow actions=drop
+
+    Final flow: unchanged
+    Datapath actions: drop
+
+
+Implementing Table 2: MAC+VLAN Learning for Ingress Port
+========================================================
+
+This table allows the switch we're implementing to learn that the
+packet's source MAC is located on the packet's ingress port in the
+packet's VLAN.
+
+>>> This table is a good example why table 1 added a VLAN tag to
+    packets that entered the switch through an access port.  We want
+    to associate a MAC+VLAN with a port regardless of whether the VLAN
+    in question was originally part of the packet or whether it was an
+    assumed VLAN associated with an access port.
+
+It only takes a single flow to do this.  The following command adds
+it:
+
+    ovs-ofctl add-flow br0 \
+        "table=2 actions=learn(table=10, NXM_OF_VLAN_TCI[0..11], \
+                               NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[], \
+                               load:NXM_OF_IN_PORT[]->NXM_NX_REG0[0..15]), \
+                         resubmit(,3)"
+
+The "learn" action (an Open vSwitch extension to OpenFlow) modifies a
+flow table based on the content of the flow currently being processed.
+Here's how you can interpret each part of the "learn" action above:
+
+    table=10
+
+        Modify flow table 10.  This will be the MAC learning table.
+
+    NXM_OF_VLAN_TCI[0..11]
+
+        Make the flow that we add to flow table 10 match the same VLAN
+        ID that the packet we're currently processing contains.  This
+        effectively scopes the MAC learning entry to a single VLAN,
+        which is the ordinary behavior for a VLAN-aware siwtch.
+
+    NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[]
+
+        Make the flow that we add to flow table 10 match, as Ethernet
+        destination, the Ethernet source address of the packet we're
+        currently processing.
+
+    load:NXM_OF_IN_PORT[]->NXM_NX_REG0[0..15]
+
+        Whereas the preceding parts specify fields for the new flow to
+        match, this specifies an action for the flow to take when it
+        matches.  The action is for the flow to load the ingress port
+        number of the current packet into register 0 (a special field
+        that is an Open vSwitch extension to OpenFlow).
+
+>>> A real use of "learn" for MAC learning would probably involve two
+    additional elements.  First, the "learn" action would specify a
+    hard_timeout for the new flow, to enable a learned MAC to
+    eventually expire if no new packets were seen from a given source
+    within a reasonable interval.  Second, one would usually want to
+    limit resource consumption by using the Flow_Table table in the
+    Open vSwitch configuration database to specify a maximum number of
+    flows in table 10.
+
+This definitely calls for examples.
+
+
+Testing Table 2
+---------------
+
+== EXAMPLE 1 ==
+
+Try the following test command:
+
+    ovs-appctl ofproto/trace br0 in_port=1,vlan_tci=20,dl_src=50:00:00:00:00:01 -generate
+
+The output shows that "learn" was executed, but it isn't otherwise
+informative, so we won't include it here.
+
+The "-generate" keyword is new.  Ordinarily, "ofproto/trace" has no
+side effects: "output" actions do not actually output packets, "learn"
+actions do not actually modify the flow table, and so on.  With
+"-generate", though, "ofproto/trace" does execute "learn" actions.
+That's important now, because we want to see the effect of the "learn"
+action on table 10.  You can see that by running:
+
+    ovs-ofctl dump-flows br0 table=10
+
+which (omitting the "duration" and "idle_age" fields, which will vary
+based on how soon you ran this command after the previous one, as well
+as some other uninteresting fields) prints something like:
+
+    NXST_FLOW reply (xid=0x4):
+     table=10, vlan_tci=0x0014/0x0fff,dl_dst=50:00:00:00:00:01 actions=load:0x1->NXM_NX_REG0[0..15]
+
+You can see that the packet coming in on VLAN 20 with source MAC
+50:00:00:00:00:01 became a flow that matches VLAN 20 (written in
+hexadecimal) and destination MAC 50:00:00:00:00:01.  The flow loads
+port number 1, the input port for the flow we tested, into register 0.
+
+
+== EXAMPLE 2 ==
+
+Here's a second test command:
+
+    ovs-appctl ofproto/trace br0 in_port=2,dl_src=50:00:00:00:00:01 -generate
+
+The flow that this command tests has the same source MAC and VLAN as
+example 1, although the VLAN comes from an access port VLAN rather
+than an 802.1Q header.  If we again dump the flows for table 10 with:
+
+    ovs-ofctl dump-flows br0 table=10
+
+then we see that the flow we saw previously has changed to indicate
+that the learned port is port 2, as we would expect:
+
+    NXST_FLOW reply (xid=0x4):
+     table=10, vlan_tci=0x0014/0x0fff,dl_dst=50:00:00:00:00:01 actions=load:0x2->NXM_NX_REG0[0..15]
+
+
+Implementing Table 3: Look Up Destination Port
+==============================================
+
+This table figures out what port we should send the packet to based on
+the destination MAC and VLAN.  That is, if we've learned the location
+of the destination (from table 2 processing some previous packet with
+that destination as its source), then we want to send the packet
+there.
+
+We need only one flow to do the lookup:
+
+    ovs-ofctl add-flow br0 \
+        "table=3 priority=50 actions=resubmit(,10), resubmit(,4)"
+
+The flow's first action resubmits to table 10, the table that the
+"learn" action modifies.  As you saw previously, the learned flows in
+this table write the learned port into register 0.  If the destination
+for our packet hasn't been learned, then there will be no matching
+flow, and so the "resubmit" turns into a no-op.  Because registers are
+initialized to 0, we can use a register 0 value of 0 in our next
+pipeline stage as a signal to flood the packet.
+
+The second action resubmits to table 4, continuing to the next
+pipeline stage.
+
+We can add another flow to skip the learning table lookup for
+multicast and broadcast packets, since those should always be flooded:
+
+    ovs-ofctl add-flow br0 \
+        "table=3 priority=99 dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 \
+          actions=resubmit(,4)"
+
+>>> We don't strictly need to add this flow, because multicast
+    addresses will never show up in our learning table.  (In turn,
+    that's because we put a flow into table 0 to drop packets that
+    have a multicast source address.)
+
+
+Testing Table 3
+---------------
+
+== EXAMPLE ==
+
+Here's a command that should cause OVS to learn that f0:00:00:00:00:01
+is on p1 in VLAN 20:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_vlan=20,dl_src=f0:00:00:00:00:01,dl_dst=90:00:00:00:00:01 -generate
+
+Here's an excerpt from the output that shows (from the "no match"
+looking up the resubmit to table 10) that the flow's destination was
+unknown:
+
+			Resubmitted flow: unchanged
+			Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+			Resubmitted  odp: drop
+			Rule: table=3 cookie=0 priority=50
+			OpenFlow actions=resubmit(,10),resubmit(,4)
+
+				Resubmitted flow: unchanged
+				Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+				Resubmitted  odp: drop
+				No match
+
+You can verify that the packet's source was learned two ways.  The
+most direct way is to dump the learning table with:
+
+    ovs-ofctl dump-flows br0 table=10
+
+which ought to show roughly the following, with extraneous details
+removed:
+
+    table=10, vlan_tci=0x0014/0x0fff,dl_dst=f0:00:00:00:00:01 actions=load:0x1->NXM_NX_REG0[0..15]
+
+>>> If you tried the examples for the previous step, or if you did
+    some of your own experiments, then you might see additional flows
+    there.  These additional flows are harmless.  If they bother you,
+    then you can remove them with "ovs-ofctl del-flows br0 table=10".
+
+The other way is to inject a packet to take advantage of the learning
+entry.  For example, we can inject a packet on p2 whose destination is
+the MAC address that we just learned on p1:
+
+    ovs-appctl ofproto/trace br0 in_port=2,dl_src=90:00:00:00:00:01,dl_dst=f0:00:00:00:00:01 -generate
+
+Here's an interesting excerpt from that command's output.  This group
+of lines traces the "resubmit(,10)", showing that the packet matched
+the learned flow for the first MAC we used, loading the OpenFlow port
+number for the learned port p1 into register 0:
+
+				Resubmitted flow: unchanged
+				Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+				Resubmitted  odp: drop
+				Rule: table=10 cookie=0 vlan_tci=0x0014/0x0fff,dl_dst=f0:00:00:00:00:01
+				OpenFlow actions=load:0x1->NXM_NX_REG0[0..15]
+
+
+If you read the commands above carefully, then you might have noticed
+that they simply have the Ethernet source and destination addresses
+exchanged.  That means that if we now rerun the first ovs-appctl
+command above, e.g.:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_vlan=20,dl_src=f0:00:00:00:00:01,dl_dst=90:00:00:00:00:01 -generate
+
+then we see in the output that the destination has now been learned:
+
+				Resubmitted flow: unchanged
+				Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0
+				Resubmitted  odp: drop
+				Rule: table=10 cookie=0 vlan_tci=0x0014/0x0fff,dl_dst=90:00:00:00:00:01
+				OpenFlow actions=load:0x2->NXM_NX_REG0[0..15]
+
+
+Implementing Table 4: Output Processing
+=======================================
+
+At entry to stage 4, we know that register 0 contains either the
+desired output port or is zero if the packet should be flooded.  We
+also know that the packet's VLAN is in its 802.1Q header, even if the
+VLAN was implicit because the packet came in on an access port.
+
+The job of the final pipeline stage is to actually output packets.
+The job is trivial for output to our trunk port p1:
+
+    ovs-ofctl add-flow br0 "table=4 reg0=1 actions=1"
+
+For output to the access ports, we just have to strip the VLAN header
+before outputting the packet:
+
+    ovs-ofctl add-flows br0 - <<'EOF'
+        table=4 reg0=2 actions=strip_vlan,2
+        table=4 reg0=3 actions=strip_vlan,3
+        table=4 reg0=4 actions=strip_vlan,4
+EOF
+
+The only slightly tricky part is flooding multicast and broadcast
+packets and unicast packets with unlearned destinations.  For those,
+we need to make sure that we only output the packets to the ports that
+carry our packet's VLAN, and that we include the 802.1Q header in the
+copy output to the trunk port but not in copies output to access
+ports:
+
+    ovs-ofctl add-flows br0 - <<'EOF'
+        table=4 reg0=0 priority=99 dl_vlan=20 actions=1,strip_vlan,2
+        table=4 reg0=0 priority=99 dl_vlan=30 actions=1,strip_vlan,3,4
+        table=4 reg0=0 priority=50            actions=1
+EOF
+
+>>> Our rules rely on the standard OpenFlow behavior that an output
+    action will not forward a packet back out the port it came in on.
+    That is, if a packet comes in on p1, and we've learned that the
+    packet's destination MAC is also on p1, so that we end up with
+    "actions=1" as our actions, the switch will not forward the packet
+    back out its input port.  The multicast/broadcast/unknown
+    destination cases above also rely on this behavior.
+
+
+Testing Table 4
+---------------
+
+== EXAMPLE 1: Broadcast, Multicast, and Unknown Destination ==
+
+Try tracing a broadcast packet arriving on p1 in VLAN 30:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=ff:ff:ff:ff:ff:ff,dl_vlan=30
+
+The interesting part of the output is the final line, which shows that
+the switch would remove the 802.1Q header and then output the packet to
+p3 and p4, which are access ports for VLAN 30:
+
+    Datapath actions: pop_vlan,3,4
+
+Similarly, if we trace a broadcast packet arriving on p3:
+
+    ovs-appctl ofproto/trace br0 in_port=3,dl_dst=ff:ff:ff:ff:ff:ff
+
+then we see that it is output to p1 with an 802.1Q tag and then to p4
+without one:
+
+    Datapath actions: push_vlan(vid=30,pcp=0),1,pop_vlan,4
+
+>>> Open vSwitch could simplify the datapath actions here to just
+    "4,push_vlan(vid=30,pcp=0),1" but it is not smart enough to do so.
+
+The following are also broadcasts, but the result is to drop the
+packets because the VLAN only belongs to the input port:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=ff:ff:ff:ff:ff:ff
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=ff:ff:ff:ff:ff:ff,dl_vlan=55
+
+Try some other broadcast cases on your own:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=ff:ff:ff:ff:ff:ff,dl_vlan=20
+    ovs-appctl ofproto/trace br0 in_port=2,dl_dst=ff:ff:ff:ff:ff:ff
+    ovs-appctl ofproto/trace br0 in_port=4,dl_dst=ff:ff:ff:ff:ff:ff
+
+You can see the same behavior with multicast packets and with unicast
+packets whose destination has not been learned, e.g.:
+
+    ovs-appctl ofproto/trace br0 in_port=4,dl_dst=01:00:00:00:00:00
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=90:12:34:56:78:90,dl_vlan=20
+    ovs-appctl ofproto/trace br0 in_port=1,dl_dst=90:12:34:56:78:90,dl_vlan=30
+
+
+== EXAMPLE 2: MAC Learning ==
+
+Let's follow the same pattern as we did for table 3.  First learn a
+MAC on port p1 in VLAN 30:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_vlan=30,dl_src=10:00:00:00:00:01,dl_dst=20:00:00:00:00:01 -generate
+
+You can see from the last line of output that the packet's destination
+is unknown, so it gets flooded to both p3 and p4, the other ports in
+VLAN 30:
+
+    Datapath actions: pop_vlan,3,4
+
+Then reverse the MACs and learn the first flow's destination on port
+p4:
+
+    ovs-appctl ofproto/trace br0 in_port=4,dl_src=20:00:00:00:00:01,dl_dst=10:00:00:00:00:01 -generate
+
+The last line of output shows that the this packet's destination is
+known to be p1, as learned from our previous command:
+
+    Datapath actions: push_vlan(vid=30,pcp=0),1
+
+Now, if we rerun our first command:
+
+    ovs-appctl ofproto/trace br0 in_port=1,dl_vlan=30,dl_src=10:00:00:00:00:01,dl_dst=20:00:00:00:00:01 -generate
+
+we can see that the result is no longer a flood but to the specified
+learned destination port p4:
+
+    Datapath actions: pop_vlan,4
+
+
+Contact 
+=======
+
+bugs@openvswitch.org
+http://openvswitch.org/
diff --git a/tutorial/automake.mk b/tutorial/automake.mk
new file mode 100644
index 000000000..d88d5b45e
--- /dev/null
+++ b/tutorial/automake.mk
@@ -0,0 +1,12 @@
+EXTRA_DIST += \
+	tutorial/Tutorial \
+	tutorial/ovs-sandbox \
+	tutorial/t-setup \
+	tutorial/t-stage0 \
+	tutorial/t-stage1 \
+	tutorial/t-stage2 \
+	tutorial/t-stage3 \
+	tutorial/t-stage4
+
+sandbox: all
+	cd $(srcdir)/tutorial && ./ovs-sandbox -b $(abs_builddir)
diff --git a/tutorial/ovs-sandbox b/tutorial/ovs-sandbox
new file mode 100755
index 000000000..04e885861
--- /dev/null
+++ b/tutorial/ovs-sandbox
@@ -0,0 +1,234 @@
+#! /bin/sh
+#
+# Copyright (c) 2013 Nicira, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at:
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -e
+
+run () {
+    echo "$@"
+    (cd "$sandbox" && "$@") || exit 1
+}
+
+builddir=
+srcdir=
+schema=
+installed=false
+built=false
+for option; do
+    # This option-parsing mechanism borrowed from a Autoconf-generated
+    # configure script under the following license:
+
+    # Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001,
+    # 2002, 2003, 2004, 2005, 2006, 2009, 2013 Free Software Foundation, Inc.
+    # This configure script is free software; the Free Software Foundation
+    # gives unlimited permission to copy, distribute and modify it.
+
+    # If the previous option needs an argument, assign it.
+    if test -n "$prev"; then
+        eval $prev=\$option
+        prev=
+        continue
+    fi
+    case $option in
+        *=*) optarg=`expr "X$option" : '[^=]*=\(.*\)'` ;;
+        *) optarg=yes ;;
+    esac
+
+    case $dashdash$option in
+        --)
+            dashdash=yes ;;
+        -h|--help)
+            cat <<EOF
+ovs-sandbox, for starting a sandboxed dummy Open vSwitch environment
+usage: $0 [OPTION...]
+
+If you run ovs-sandbox from an OVS build directory, it uses the OVS that
+you built.  Otherwise, if you have an installed Open vSwitch, it uses
+the installed version.
+
+These options force ovs-sandbox to use a particular OVS build:
+  -b, --builddir=DIR   specify Open vSwitch build directory
+  -s, --srcdir=DIR     specify Open vSwitch source directory
+These options force ovs-sandbox to use an installed Open vSwitch:
+  -i, --installed      use installed Open vSwitch
+  -S, --schema=FILE    use FILE as vswitch.ovsschema
+
+Other options:
+  -h, --help           Print this usage message.
+EOF
+            exit 0
+            ;;
+
+        --b*=*)
+            builddir=$optarg
+            built=:
+            ;;
+        -b|--b*)
+            prev=builddir
+            built=:
+            ;;
+        --sr*=*)
+            srcdir=$optarg
+            built=false
+            ;;
+        -s|--sr*)
+            prev=srcdir
+            built=false
+            ;;
+        -i|--installed)
+            installed=:
+            ;;
+        --sc*=*)
+            schema=$optarg
+            installed=:
+            ;;
+        -S|--sc*)
+            prev=schema
+            installed=:
+            ;;
+        -*)
+            echo "unrecognized option $option (use --help for help)" >&2
+            exit 1
+            ;;
+        *)
+            echo "$option: non-option arguments not supported (use --help for help)" >&2
+            exit 1
+            ;;
+    esac
+    shift
+done
+
+if $installed && $built; then
+    echo "sorry, conflicting options (use --help for help)" >&2
+    exit 1
+elif $installed || $built; then
+    :
+elif test -e vswitchd/ovs-vswitchd; then
+    built=:
+    builddir=.
+elif (ovs-vswitchd --version) >/dev/null 2>&1; then
+    installed=:
+else
+    echo "can't find an OVS build or install (use --help for help)" >&2
+    exit 1
+fi
+
+if $built; then
+    if test ! -e "$builddir"/vswitchd/ovs-vswitchd; then
+        echo "$builddir does not appear to be an OVS build directory" >&2
+        exit 1
+    fi
+    builddir=`cd $builddir && pwd`
+
+    # Find srcdir.
+    case $srcdir in
+        '')
+            srcdir=$builddir
+            if test ! -e "$srcdir"/WHY-OVS; then
+                srcdir=`cd $builddir/.. && pwd`
+            fi
+            ;;
+        /*) ;;
+        *) srcdir=`pwd`/$srcdir ;;
+    esac
+    schema=$srcdir/vswitchd/vswitch.ovsschema
+    if test ! -e "$schema"; then
+        echo >&2 'source directory not found, please use --srcdir'
+        exit 1
+    fi
+
+    # Put built tools early in $PATH.
+    if test ! -e $builddir/vswitchd/ovs-vswitchd; then
+        echo >&2 'build not found, please change set $builddir or change directory'
+        exit 1
+    fi
+    PATH=$builddir/ovsdb:$builddir/vswitchd:$builddir/utilities:$PATH
+    export PATH
+else
+    case $schema in
+        '')
+            for schema in \
+                /usr/local/share/openvswitch/vswitch.ovsschema \
+                /usr/share/openvswitch/vswitch.ovsschema \
+                none; do
+                if test -r $schema; then
+                    break
+                fi
+            done
+            ;;
+        /*) ;;
+        *) schema=`pwd`/$schema ;;
+    esac
+    if test ! -r "$schema"; then
+        echo "can't find vswitch.ovsschema, please specify --schema" >&2
+        exit 1
+    fi
+fi
+
+# Create sandbox.
+rm -rf sandbox
+mkdir sandbox
+sandbox=`cd sandbox && pwd`
+
+# Set up environment for OVS programs to sandbox themselves.
+OVS_RUNDIR=$sandbox; export OVS_RUNDIR
+OVS_LOGDIR=$sandbox; export OVS_LOGDIR
+OVS_DBDIR=$sandbox; export OVS_DBDIR
+OVS_SYSCONFDIR=$sandbox; export OVS_SYSCONFDIR
+
+if $built; then
+    # Easy access to OVS manpages.
+    (cd "$builddir" && make install-man mandir="$sandbox"/man)
+    MANPATH=$sandbox/man:; export MANPATH
+fi
+
+# Ensure cleanup.
+trap 'kill `cat "$sandbox"/*.pid`' 0 1 2 3 13 14 15
+
+# Create database and start ovsdb-server.
+touch "$sandbox"/.conf.db.~lock~
+run ovsdb-tool create conf.db "$srcdir"/vswitchd/vswitch.ovsschema
+run ovsdb-server --detach --no-chdir --pidfile -vconsole:off --log-file \
+    --remote=punix:"$sandbox"/db.sock
+
+# Start ovs-vswitchd.
+run ovs-vswitchd --detach --no-chdir --pidfile -vconsole:off --log-file \
+    --enable-dummy=override -vvconn -vnetdev_dummy
+
+cat <<EOF
+
+
+
+----------------------------------------------------------------------
+You are running in a dummy Open vSwitch environment.  You can use
+ovs-vsctl, ovs-ofctl, ovs-appctl, and other tools to work with the
+dummy switch.  
+
+Log files, pidfiles, and the configuration database are in the
+"sandbox" subdirectory.
+
+Exit the shell to kill the running daemons.
+EOF
+
+status=0; $SHELL || status=$?
+
+cat <<EOF
+----------------------------------------------------------------------
+
+
+
+EOF
+
+exit $status
diff --git a/tutorial/t-setup b/tutorial/t-setup
new file mode 100755
index 000000000..4925d8270
--- /dev/null
+++ b/tutorial/t-setup
@@ -0,0 +1,8 @@
+#! /bin/sh -ve
+
+ovs-vsctl add-br br0 -- set Bridge br0 fail-mode=secure
+
+for i in 1 2 3 4; do
+    ovs-vsctl add-port br0 p$i -- set Interface p$i ofport_request=$i
+    ovs-ofctl mod-port br0 p$i up
+done
diff --git a/tutorial/t-stage0 b/tutorial/t-stage0
new file mode 100755
index 000000000..55687ee0f
--- /dev/null
+++ b/tutorial/t-stage0
@@ -0,0 +1,9 @@
+#! /bin/sh -ve
+
+ovs-ofctl add-flow br0 \
+    "table=0, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=drop"
+
+ovs-ofctl add-flow br0 \
+    "table=0, dl_dst=01:08:c2:00:00:00/ff:ff:ff:ff:ff:f0, actions=drop"
+
+ovs-ofctl add-flow br0 "table=0, priority=0, actions=resubmit(,1)"
diff --git a/tutorial/t-stage1 b/tutorial/t-stage1
new file mode 100755
index 000000000..97aed7e3e
--- /dev/null
+++ b/tutorial/t-stage1
@@ -0,0 +1,12 @@
+#! /bin/sh -ve
+
+ovs-ofctl add-flow br0 "table=1, priority=0, actions=drop"
+
+ovs-ofctl add-flow br0 \
+    "table=1, priority=99, in_port=1, actions=resubmit(,2)"
+
+ovs-ofctl add-flows br0 - <<'EOF'
+    table=1, priority=99, in_port=2, vlan_tci=0, actions=mod_vlan_vid:20, resubmit(,2)
+    table=1, priority=99, in_port=3, vlan_tci=0, actions=mod_vlan_vid:30, resubmit(,2)
+    table=1, priority=99, in_port=4, vlan_tci=0, actions=mod_vlan_vid:30, resubmit(,2)
+EOF
diff --git a/tutorial/t-stage2 b/tutorial/t-stage2
new file mode 100755
index 000000000..f0d687e68
--- /dev/null
+++ b/tutorial/t-stage2
@@ -0,0 +1,7 @@
+#! /bin/sh -ve
+
+ovs-ofctl add-flow br0 \
+    "table=2 actions=learn(table=10, NXM_OF_VLAN_TCI[0..11], \
+                           NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[], \
+                           load:NXM_OF_IN_PORT[]->NXM_NX_REG0[0..15]), \
+                     resubmit(,3)"
diff --git a/tutorial/t-stage3 b/tutorial/t-stage3
new file mode 100755
index 000000000..eb4ab3c3f
--- /dev/null
+++ b/tutorial/t-stage3
@@ -0,0 +1,8 @@
+#! /bin/sh -ve
+
+ovs-ofctl add-flow br0 \
+    "table=3 priority=50 actions=resubmit(,10), resubmit(,4)"
+
+ovs-ofctl add-flow br0 \
+    "table=3 priority=99 dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 \
+      actions=resubmit(,4)"
diff --git a/tutorial/t-stage4 b/tutorial/t-stage4
new file mode 100755
index 000000000..3f71b9228
--- /dev/null
+++ b/tutorial/t-stage4
@@ -0,0 +1,15 @@
+#! /bin/sh -ve
+
+ovs-ofctl add-flow br0 "table=4 reg0=1 actions=1"
+
+ovs-ofctl add-flows br0 - <<'EOF'
+    table=4 reg0=2 actions=strip_vlan,2
+    table=4 reg0=3 actions=strip_vlan,3
+    table=4 reg0=4 actions=strip_vlan,4
+EOF
+
+ovs-ofctl add-flows br0 - <<'EOF'
+    table=4 reg0=0 priority=99 dl_vlan=20 actions=1,strip_vlan,2
+    table=4 reg0=0 priority=99 dl_vlan=30 actions=1,strip_vlan,3,4
+    table=4 reg0=0 priority=50            actions=1
+EOF