docs: Add documentation for ovsdb relay mode.

Main documentation for the service model and tutorial with the use case
and configuration examples.

Acked-by: Mark D. Gray <mark.d.gray@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>

4 files changed, 180 insertions, 8 deletions

diff --git a/Documentation/automake.mk b/Documentation/automake.mk
index bc30f94c5..213d9c867 100644
--- a/Documentation/automake.mk
+++ b/Documentation/automake.mk
@@ -52,6 +52,7 @@ DOC_SOURCE = \
 	Documentation/topics/networking-namespaces.rst \
 	Documentation/topics/openflow.rst \
 	Documentation/topics/ovs-extensions.rst \
+	Documentation/topics/ovsdb-relay.rst \
 	Documentation/topics/ovsdb-replication.rst \
 	Documentation/topics/porting.rst \
 	Documentation/topics/record-replay.rst \
diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst
index e4f1bf766..980ba29e7 100644
--- a/Documentation/ref/ovsdb.7.rst
+++ b/Documentation/ref/ovsdb.7.rst
@@ -121,13 +121,14 @@ schema checksum from a schema or database file, respectively.
 Service Models
 ==============
 
-OVSDB supports three service models for databases: **standalone**,
-**active-backup**, and **clustered**.  The service models provide different
-compromises among consistency, availability, and partition tolerance.  They
-also differ in the number of servers required and in terms of performance.  The
-standalone and active-backup database service models share one on-disk format,
-and clustered databases use a different format, but the OVSDB programs work
-with both formats.  ``ovsdb(5)`` documents these file formats.
+OVSDB supports four service models for databases: **standalone**,
+**active-backup**, **relay** and **clustered**.  The service models provide
+different compromises among consistency, availability, and partition tolerance.
+They also differ in the number of servers required and in terms of performance.
+The standalone and active-backup database service models share one on-disk
+format, and clustered databases use a different format, but the OVSDB programs
+work with both formats.  ``ovsdb(5)`` documents these file formats.  Relay
+databases have no on-disk storage.
 
 RFC 7047, which specifies the OVSDB protocol, does not mandate or specify
 any particular service model.
@@ -406,6 +407,50 @@ following consequences:
   that the client previously read.  The OVSDB client library in Open vSwitch
   uses this feature to avoid servers with stale data.
 
+Relay Service Model
+-------------------
+
+A **relay** database is a way to scale out read-mostly access to the
+existing database working in any service model including relay.
+
+Relay database creates and maintains an OVSDB connection with another OVSDB
+server.  It uses this connection to maintain an in-memory copy of the remote
+database (a.k.a. the ``relay source``) keeping the copy up-to-date as the
+database content changes on the relay source in the real time.
+
+The purpose of relay server is to scale out the number of database clients.
+Read-only transactions and monitor requests are fully handled by the relay
+server itself.  For the transactions that request database modifications,
+relay works as a proxy between the client and the relay source, i.e. it
+forwards transactions and replies between them.
+
+Compared to the clustered and active-backup models, relay service model
+provides read and write access to the database similarly to a clustered
+database (and even more scalable), but with generally insignificant performance
+overhead of an active-backup model.  At the same time it doesn't increase
+availability that needs to be covered by the service model of the relay source.
+
+Relay database has no on-disk storage and therefore cannot be converted to
+any other service model.
+
+If there is already a database started in any service model, to start a relay
+database server use ``ovsdb-server relay:<DB_NAME>:<relay source>``, where
+``<DB_NAME>`` is the database name as specified in the schema of the database
+that existing server runs, and ``<relay source>`` is an OVSDB connection method
+(see `Connection Methods`_ below) that connects to the existing database
+server.  ``<relay source>`` could contain a comma-separated list of connection
+methods, e.g. to connect to any server of the clustered database.
+Multiple relay servers could be started for the same relay source.
+
+Since the way relays handle read and write transactions is very similar
+to the clustered model where "cluster" means "set of relay servers connected
+to the same relay source", "follower" means "relay server" and the "leader"
+means "relay source", same consistency consequences as for the clustered
+model applies to relay as well (See `Understanding Cluster Consistency`_
+above).
+
+Open vSwitch 2.16 introduced support for relay service model.
+
 Database Replication
 ====================
 
@@ -414,7 +459,8 @@ Replication, in this context, means to make, and keep up-to-date, a read-only
 copy of the contents of a database (the ``replica``).  One use of replication
 is to keep an up-to-date backup of a database.  A replica used solely for
 backup would not need to support clients of its own.  A set of replicas that do
-serve clients could be used to scale out read access to the primary database.
+serve clients could be used to scale out read access to the primary database,
+however `Relay Service Model`_ is more suitable for that purpose.
 
 A database replica is set up in the same way as a backup server in an
 active-backup pair, with the difference that the replica is never promoted to
diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst
index 0036567eb..d8ccbd757 100644
--- a/Documentation/topics/index.rst
+++ b/Documentation/topics/index.rst
@@ -44,6 +44,7 @@ OVS
    openflow
    bonding
    networking-namespaces
+   ovsdb-relay
    ovsdb-replication
    dpdk/index
    windows
diff --git a/Documentation/topics/ovsdb-relay.rst b/Documentation/topics/ovsdb-relay.rst
new file mode 100644
index 000000000..50a3c6d07
--- /dev/null
+++ b/Documentation/topics/ovsdb-relay.rst
@@ -0,0 +1,124 @@
+..
+      Copyright 2021, Red Hat, Inc.
+
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+      Convention for heading levels in Open vSwitch documentation:
+
+      =======  Heading 0 (reserved for the title in a document)
+      -------  Heading 1
+      ~~~~~~~  Heading 2
+      +++++++  Heading 3
+      '''''''  Heading 4
+
+      Avoid deeper levels because they do not render well.
+
+===============================
+Scaling OVSDB Access With Relay
+===============================
+
+Open vSwitch 2.16 introduced support for OVSDB Relay mode with the goal to
+increase database scalability for a big deployments.  Mainly, OVN (Open Virtual
+Network) Southbound Database deployments.  This document describes the main
+concept and provides the configuration examples.
+
+What is OVSDB Relay?
+--------------------
+
+Relay is a database service model in which one ``ovsdb-server`` (``relay``)
+connects to another standalone or clustered database server
+(``relay source``) and maintains in-memory copy of its data, receiving
+all the updates via this OVSDB connection.  Relay server handles all the
+read-only requests (monitors and transactions) on its own and forwards all the
+transactions that requires database modifications to the relay source.
+
+Why is this needed?
+-------------------
+
+Some OVN deployment could have hundreds or even thousands of nodes.  On each of
+these nodes there is an ovn-controller, which is connected to the
+OVN_Southbound database that is served by a standalone or clustered OVSDB.
+Standalone database is handled by a single ovsdb-server process and clustered
+could consist of 3 to 5 ovsdb-server processes.  For the clustered database,
+higher number of servers may significantly increase transaction latency due
+to necessity for these servers to reach consensus.  So, in the end limited
+number of ovsdb-server processes serves ever growing number of clients and this
+leads to performance issues.
+
+Read-only access could be scaled up with OVSDB replication on top of
+active-backup service model, but ovn-controller is a read-mostly client, not
+a read-only, i.e. it needs to execute write transactions from time to time.
+Here relay service model comes into play.
+
+2-Tier Deployment
+-----------------
+
+Solution for the scaling issue could look like a 2-tier deployment, where
+a set of relay servers is connected to the main database cluster
+(OVN_Southbound) and clients (ovn-conrtoller) connected to these relay
+servers::
+
+                                    172.16.0.1
+   +--------------------+   +----+ ovsdb-relay-1 +--+---+ client-1
+   |                    |   |                       |
+   |    Clustered       |   |                       +---+ client-2
+   |     Database       |   |                        ...
+   |                    |   |                       +---+ client-N
+   |    10.0.0.2        |   |
+   |  ovsdb-server-2    |   |       172.16.0.2
+   |   +        +       |   +----+ ovsdb-relay-2 +--+---+ client-N+1
+   |   |        |       |   |                       |
+   |   |        +       +---+                       +---+ client-N+2
+   |   |   10.0.0.1     |   |                        ...
+   |   | ovsdb-server-1 |   |                       +---+ client-2N
+   |   |        +       |   |
+   |   |        |       |   |
+   |   +        +       |   +      ... ... ... ... ...
+   |  ovsdb-server-3    |   |
+   |    10.0.0.3        |   |                       +---+ client-KN-1
+   |                    |   |       172.16.0.K      |
+   +--------------------+   +----+ ovsdb-relay-K +--+---+ client-KN
+
+In practice, the picture might look a bit more complex, because all relay
+servers might connect to any member of a main cluster and clients might
+connect to any relay server of their choice.
+
+Assuming that servers of a main cluster started like this::
+
+  $ ovsdb-server --remote=ptcp:6642:10.0.0.1 ovn-sb-1.db
+
+The same for other two servers.  In this case relay servers could be
+started like this::
+
+  $ REMOTES=tcp:10.0.0.1:6642,tcp:10.0.0.2:6642,tcp:10.0.0.3:6642
+  $ ovsdb-server --remote=ptcp:6642:172.16.0.1 relay:OVN_Southbound:$REMOTES
+  $ ...
+  $ ovsdb-server --remote=ptcp:6642:172.16.0.K relay:OVN_Southbound:$REMOTES
+
+Every relay server could connect to any of the cluster members of their choice,
+fairness of load distribution is achieved by shuffling remotes.
+
+For the actual clients, they could be configured to connect to any of the
+relay servers.  For ovn-controllers the configuration could look like this::
+
+  $ REMOTES=tcp:172.16.0.1:6642,...,tcp:172.16.0.K:6642
+  $ ovs-vsctl set Open_vSwitch . external-ids:ovn-remote=$REMOTES
+
+Setup like this allows the system to serve ``K * N`` clients while having only
+``K`` actual connections on the main clustered database keeping it in a
+stable state.
+
+It's also possible to create multi-tier deployments by connecting one set
+of relay servers to another (smaller) set of relay servers, or even create
+tree-like structures with the cost of increased latency for write transactions,
+because they will be forwarded multiple times.