diff options
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/automake.mk | 1 | ||||
-rw-r--r-- | Documentation/ref/ovsdb.7.rst | 62 | ||||
-rw-r--r-- | Documentation/topics/index.rst | 1 | ||||
-rw-r--r-- | Documentation/topics/ovsdb-relay.rst | 124 |
4 files changed, 180 insertions, 8 deletions
diff --git a/Documentation/automake.mk b/Documentation/automake.mk index bc30f94c5..213d9c867 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -52,6 +52,7 @@ DOC_SOURCE = \ Documentation/topics/networking-namespaces.rst \ Documentation/topics/openflow.rst \ Documentation/topics/ovs-extensions.rst \ + Documentation/topics/ovsdb-relay.rst \ Documentation/topics/ovsdb-replication.rst \ Documentation/topics/porting.rst \ Documentation/topics/record-replay.rst \ diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst index e4f1bf766..980ba29e7 100644 --- a/Documentation/ref/ovsdb.7.rst +++ b/Documentation/ref/ovsdb.7.rst @@ -121,13 +121,14 @@ schema checksum from a schema or database file, respectively. Service Models ============== -OVSDB supports three service models for databases: **standalone**, -**active-backup**, and **clustered**. The service models provide different -compromises among consistency, availability, and partition tolerance. They -also differ in the number of servers required and in terms of performance. The -standalone and active-backup database service models share one on-disk format, -and clustered databases use a different format, but the OVSDB programs work -with both formats. ``ovsdb(5)`` documents these file formats. +OVSDB supports four service models for databases: **standalone**, +**active-backup**, **relay** and **clustered**. The service models provide +different compromises among consistency, availability, and partition tolerance. +They also differ in the number of servers required and in terms of performance. +The standalone and active-backup database service models share one on-disk +format, and clustered databases use a different format, but the OVSDB programs +work with both formats. ``ovsdb(5)`` documents these file formats. Relay +databases have no on-disk storage. RFC 7047, which specifies the OVSDB protocol, does not mandate or specify any particular service model. @@ -406,6 +407,50 @@ following consequences: that the client previously read. The OVSDB client library in Open vSwitch uses this feature to avoid servers with stale data. +Relay Service Model +------------------- + +A **relay** database is a way to scale out read-mostly access to the +existing database working in any service model including relay. + +Relay database creates and maintains an OVSDB connection with another OVSDB +server. It uses this connection to maintain an in-memory copy of the remote +database (a.k.a. the ``relay source``) keeping the copy up-to-date as the +database content changes on the relay source in the real time. + +The purpose of relay server is to scale out the number of database clients. +Read-only transactions and monitor requests are fully handled by the relay +server itself. For the transactions that request database modifications, +relay works as a proxy between the client and the relay source, i.e. it +forwards transactions and replies between them. + +Compared to the clustered and active-backup models, relay service model +provides read and write access to the database similarly to a clustered +database (and even more scalable), but with generally insignificant performance +overhead of an active-backup model. At the same time it doesn't increase +availability that needs to be covered by the service model of the relay source. + +Relay database has no on-disk storage and therefore cannot be converted to +any other service model. + +If there is already a database started in any service model, to start a relay +database server use ``ovsdb-server relay:<DB_NAME>:<relay source>``, where +``<DB_NAME>`` is the database name as specified in the schema of the database +that existing server runs, and ``<relay source>`` is an OVSDB connection method +(see `Connection Methods`_ below) that connects to the existing database +server. ``<relay source>`` could contain a comma-separated list of connection +methods, e.g. to connect to any server of the clustered database. +Multiple relay servers could be started for the same relay source. + +Since the way relays handle read and write transactions is very similar +to the clustered model where "cluster" means "set of relay servers connected +to the same relay source", "follower" means "relay server" and the "leader" +means "relay source", same consistency consequences as for the clustered +model applies to relay as well (See `Understanding Cluster Consistency`_ +above). + +Open vSwitch 2.16 introduced support for relay service model. + Database Replication ==================== @@ -414,7 +459,8 @@ Replication, in this context, means to make, and keep up-to-date, a read-only copy of the contents of a database (the ``replica``). One use of replication is to keep an up-to-date backup of a database. A replica used solely for backup would not need to support clients of its own. A set of replicas that do -serve clients could be used to scale out read access to the primary database. +serve clients could be used to scale out read access to the primary database, +however `Relay Service Model`_ is more suitable for that purpose. A database replica is set up in the same way as a backup server in an active-backup pair, with the difference that the replica is never promoted to diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst index 0036567eb..d8ccbd757 100644 --- a/Documentation/topics/index.rst +++ b/Documentation/topics/index.rst @@ -44,6 +44,7 @@ OVS openflow bonding networking-namespaces + ovsdb-relay ovsdb-replication dpdk/index windows diff --git a/Documentation/topics/ovsdb-relay.rst b/Documentation/topics/ovsdb-relay.rst new file mode 100644 index 000000000..50a3c6d07 --- /dev/null +++ b/Documentation/topics/ovsdb-relay.rst @@ -0,0 +1,124 @@ +.. + Copyright 2021, Red Hat, Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=============================== +Scaling OVSDB Access With Relay +=============================== + +Open vSwitch 2.16 introduced support for OVSDB Relay mode with the goal to +increase database scalability for a big deployments. Mainly, OVN (Open Virtual +Network) Southbound Database deployments. This document describes the main +concept and provides the configuration examples. + +What is OVSDB Relay? +-------------------- + +Relay is a database service model in which one ``ovsdb-server`` (``relay``) +connects to another standalone or clustered database server +(``relay source``) and maintains in-memory copy of its data, receiving +all the updates via this OVSDB connection. Relay server handles all the +read-only requests (monitors and transactions) on its own and forwards all the +transactions that requires database modifications to the relay source. + +Why is this needed? +------------------- + +Some OVN deployment could have hundreds or even thousands of nodes. On each of +these nodes there is an ovn-controller, which is connected to the +OVN_Southbound database that is served by a standalone or clustered OVSDB. +Standalone database is handled by a single ovsdb-server process and clustered +could consist of 3 to 5 ovsdb-server processes. For the clustered database, +higher number of servers may significantly increase transaction latency due +to necessity for these servers to reach consensus. So, in the end limited +number of ovsdb-server processes serves ever growing number of clients and this +leads to performance issues. + +Read-only access could be scaled up with OVSDB replication on top of +active-backup service model, but ovn-controller is a read-mostly client, not +a read-only, i.e. it needs to execute write transactions from time to time. +Here relay service model comes into play. + +2-Tier Deployment +----------------- + +Solution for the scaling issue could look like a 2-tier deployment, where +a set of relay servers is connected to the main database cluster +(OVN_Southbound) and clients (ovn-conrtoller) connected to these relay +servers:: + + 172.16.0.1 + +--------------------+ +----+ ovsdb-relay-1 +--+---+ client-1 + | | | | + | Clustered | | +---+ client-2 + | Database | | ... + | | | +---+ client-N + | 10.0.0.2 | | + | ovsdb-server-2 | | 172.16.0.2 + | + + | +----+ ovsdb-relay-2 +--+---+ client-N+1 + | | | | | | + | | + +---+ +---+ client-N+2 + | | 10.0.0.1 | | ... + | | ovsdb-server-1 | | +---+ client-2N + | | + | | + | | | | | + | + + | + ... ... ... ... ... + | ovsdb-server-3 | | + | 10.0.0.3 | | +---+ client-KN-1 + | | | 172.16.0.K | + +--------------------+ +----+ ovsdb-relay-K +--+---+ client-KN + +In practice, the picture might look a bit more complex, because all relay +servers might connect to any member of a main cluster and clients might +connect to any relay server of their choice. + +Assuming that servers of a main cluster started like this:: + + $ ovsdb-server --remote=ptcp:6642:10.0.0.1 ovn-sb-1.db + +The same for other two servers. In this case relay servers could be +started like this:: + + $ REMOTES=tcp:10.0.0.1:6642,tcp:10.0.0.2:6642,tcp:10.0.0.3:6642 + $ ovsdb-server --remote=ptcp:6642:172.16.0.1 relay:OVN_Southbound:$REMOTES + $ ... + $ ovsdb-server --remote=ptcp:6642:172.16.0.K relay:OVN_Southbound:$REMOTES + +Every relay server could connect to any of the cluster members of their choice, +fairness of load distribution is achieved by shuffling remotes. + +For the actual clients, they could be configured to connect to any of the +relay servers. For ovn-controllers the configuration could look like this:: + + $ REMOTES=tcp:172.16.0.1:6642,...,tcp:172.16.0.K:6642 + $ ovs-vsctl set Open_vSwitch . external-ids:ovn-remote=$REMOTES + +Setup like this allows the system to serve ``K * N`` clients while having only +``K`` actual connections on the main clustered database keeping it in a +stable state. + +It's also possible to create multi-tier deployments by connecting one set +of relay servers to another (smaller) set of relay servers, or even create +tree-like structures with the cost of increased latency for write transactions, +because they will be forwarded multiple times. |