1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
|
<?xml version="1.0" encoding="utf-8"?>
<manpage program="ovn-architecture" section="7" title="OVN Architecture">
<h1>Name</h1>
<p>ovn-architecture -- Open Virtual Network architecture</p>
<h1>Description</h1>
<p>
OVN, the Open Virtual Network, is a system to support virtual network
abstraction. OVN complements the existing capabilities of OVS to add
native support for virtual network abstractions, such as virtual L2 and L3
overlays and security groups. Services such as DHCP are also desirable
features. Just like OVS, OVN's design goal is to have a production-quality
implementation that can operate at significant scale.
</p>
<p>
An OVN deployment consists of several components:
</p>
<ul>
<li>
<p>
A <dfn>Cloud Management System</dfn> (<dfn>CMS</dfn>), which is
OVN's ultimate client (via its users and administrators). OVN
integration requires installing a CMS-specific plugin and
related software (see below). OVN initially targets OpenStack
as CMS.
</p>
<p>
We generally speak of ``the'' CMS, but one can imagine scenarios in
which multiple CMSes manage different parts of an OVN deployment.
</p>
</li>
<li>
An OVN Database physical or virtual node (or, eventually, cluster)
installed in a central location.
</li>
<li>
One or more (usually many) <dfn>hypervisors</dfn>. Hypervisors must run
Open vSwitch and implement the interface described in
<code>IntegrationGuide.rst</code> in the OVS source tree. Any hypervisor
platform supported by Open vSwitch is acceptable.
</li>
<li>
<p>
Zero or more <dfn>gateways</dfn>. A gateway extends a tunnel-based
logical network into a physical network by bidirectionally forwarding
packets between tunnels and a physical Ethernet port. This allows
non-virtualized machines to participate in logical networks. A gateway
may be a physical host, a virtual machine, or an ASIC-based hardware
switch that supports the <code>vtep</code>(5) schema.
</p>
<p>
Hypervisors and gateways are together called <dfn>transport node</dfn>
or <dfn>chassis</dfn>.
</p>
</li>
</ul>
<p>
The diagram below shows how the major components of OVN and related
software interact. Starting at the top of the diagram, we have:
</p>
<ul>
<li>
The Cloud Management System, as defined above.
</li>
<li>
<p>
The <dfn>OVN/CMS Plugin</dfn> is the component of the CMS that
interfaces to OVN. In OpenStack, this is a Neutron plugin.
The plugin's main purpose is to translate the CMS's notion of logical
network configuration, stored in the CMS's configuration database in a
CMS-specific format, into an intermediate representation understood by
OVN.
</p>
<p>
This component is necessarily CMS-specific, so a new plugin needs to be
developed for each CMS that is integrated with OVN. All of the
components below this one in the diagram are CMS-independent.
</p>
</li>
<li>
<p>
The <dfn>OVN Northbound Database</dfn> receives the intermediate
representation of logical network configuration passed down by the
OVN/CMS Plugin. The database schema is meant to be ``impedance
matched'' with the concepts used in a CMS, so that it directly supports
notions of logical switches, routers, ACLs, and so on. See
<code>ovn-nb</code>(5) for details.
</p>
<p>
The OVN Northbound Database has only two clients: the OVN/CMS Plugin
above it and <code>ovn-northd</code> below it.
</p>
</li>
<li>
<code>ovn-northd</code>(8) connects to the OVN Northbound Database
above it and the OVN Southbound Database below it. It translates the
logical network configuration in terms of conventional network
concepts, taken from the OVN Northbound Database, into logical
datapath flows in the OVN Southbound Database below it.
</li>
<li>
<p>
The <dfn>OVN Southbound Database</dfn> is the center of the system.
Its clients are <code>ovn-northd</code>(8) above it and
<code>ovn-controller</code>(8) on every transport node below it.
</p>
<p>
The OVN Southbound Database contains three kinds of data: <dfn>Physical
Network</dfn> (PN) tables that specify how to reach hypervisor and
other nodes, <dfn>Logical Network</dfn> (LN) tables that describe the
logical network in terms of ``logical datapath flows,'' and
<dfn>Binding</dfn> tables that link logical network components'
locations to the physical network. The hypervisors populate the PN and
Port_Binding tables, whereas <code>ovn-northd</code>(8) populates the
LN tables.
</p>
<p>
OVN Southbound Database performance must scale with the number of
transport nodes. This will likely require some work on
<code>ovsdb-server</code>(1) as we encounter bottlenecks.
Clustering for availability may be needed.
</p>
</li>
</ul>
<p>
The remaining components are replicated onto each hypervisor:
</p>
<ul>
<li>
<code>ovn-controller</code>(8) is OVN's agent on each hypervisor and
software gateway. Northbound, it connects to the OVN Southbound
Database to learn about OVN configuration and status and to
populate the PN table and the <code>Chassis</code> column in
<code>Binding</code> table with the hypervisor's status.
Southbound, it connects to <code>ovs-vswitchd</code>(8) as an
OpenFlow controller, for control over network traffic, and to the
local <code>ovsdb-server</code>(1) to allow it to monitor and
control Open vSwitch configuration.
</li>
<li>
<code>ovs-vswitchd</code>(8) and <code>ovsdb-server</code>(1) are
conventional components of Open vSwitch.
</li>
</ul>
<pre fixed="yes">
CMS
|
|
+-----------|-----------+
| | |
| OVN/CMS Plugin |
| | |
| | |
| OVN Northbound DB |
| | |
| | |
| ovn-northd |
| | |
+-----------|-----------+
|
|
+-------------------+
| OVN Southbound DB |
+-------------------+
|
|
+------------------+------------------+
| | |
HV 1 | | HV n |
+---------------|---------------+ . +---------------|---------------+
| | | . | | |
| ovn-controller | . | ovn-controller |
| | | | . | | | |
| | | | | | | |
| ovs-vswitchd ovsdb-server | | ovs-vswitchd ovsdb-server |
| | | |
+-------------------------------+ +-------------------------------+
</pre>
<h2>Information Flow in OVN</h2>
<p>
Configuration data in OVN flows from north to south. The CMS, through its
OVN/CMS plugin, passes the logical network configuration to
<code>ovn-northd</code> via the northbound database. In turn,
<code>ovn-northd</code> compiles the configuration into a lower-level form
and passes it to all of the chassis via the southbound database.
</p>
<p>
Status information in OVN flows from south to north. OVN currently
provides only a few forms of status information. First,
<code>ovn-northd</code> populates the <code>up</code> column in the
northbound <code>Logical_Switch_Port</code> table: if a logical port's
<code>chassis</code> column in the southbound <code>Port_Binding</code>
table is nonempty, it sets <code>up</code> to <code>true</code>, otherwise
to <code>false</code>. This allows the CMS to detect when a VM's
networking has come up.
</p>
<p>
Second, OVN provides feedback to the CMS on the realization of its
configuration, that is, whether the configuration provided by the CMS has
taken effect. This feature requires the CMS to participate in a sequence
number protocol, which works the following way:
</p>
<ol>
<li>
When the CMS updates the configuration in the northbound database, as
part of the same transaction, it increments the value of the
<code>nb_cfg</code> column in the <code>NB_Global</code> table. (This is
only necessary if the CMS wants to know when the configuration has been
realized.)
</li>
<li>
When <code>ovn-northd</code> updates the southbound database based on a
given snapshot of the northbound database, it copies <code>nb_cfg</code>
from northbound <code>NB_Global</code> into the southbound database
<code>SB_Global</code> table, as part of the same transaction. (Thus, an
observer monitoring both databases can determine when the southbound
database is caught up with the northbound.)
</li>
<li>
After <code>ovn-northd</code> receives confirmation from the southbound
database server that its changes have committed, it updates
<code>sb_cfg</code> in the northbound <code>NB_Global</code> table to the
<code>nb_cfg</code> version that was pushed down. (Thus, the CMS or
another observer can determine when the southbound database is caught up
without a connection to the southbound database.)
</li>
<li>
The <code>ovn-controller</code> process on each chassis receives the
updated southbound database, with the updated <code>nb_cfg</code>. This
process in turn updates the physical flows installed in the chassis's
Open vSwitch instances. When it receives confirmation from Open vSwitch
that the physical flows have been updated, it updates <code>nb_cfg</code>
in its own <code>Chassis</code> record in the southbound database.
</li>
<li>
<code>ovn-northd</code> monitors the <code>nb_cfg</code> column in all of
the <code>Chassis</code> records in the southbound database. It keeps
track of the minimum value among all the records and copies it into the
<code>hv_cfg</code> column in the northbound <code>NB_Global</code>
table. (Thus, the CMS or another observer can determine when all of the
hypervisors have caught up to the northbound configuration.)
</li>
</ol>
<h2>Chassis Setup</h2>
<p>
Each chassis in an OVN deployment must be configured with an Open vSwitch
bridge dedicated for OVN's use, called the <dfn>integration bridge</dfn>.
System startup scripts may create this bridge prior to starting
<code>ovn-controller</code> if desired. If this bridge does not exist when
ovn-controller starts, it will be created automatically with the default
configuration suggested below. The ports on the integration bridge include:
</p>
<ul>
<li>
On any chassis, tunnel ports that OVN uses to maintain logical network
connectivity. <code>ovn-controller</code> adds, updates, and removes
these tunnel ports.
</li>
<li>
On a hypervisor, any VIFs that are to be attached to logical networks.
The hypervisor itself, or the integration between Open vSwitch and the
hypervisor (described in <code>IntegrationGuide.rst</code>) takes care of
this. (This is not part of OVN or new to OVN; this is pre-existing
integration work that has already been done on hypervisors that support
OVS.)
</li>
<li>
On a gateway, the physical port used for logical network connectivity.
System startup scripts add this port to the bridge prior to starting
<code>ovn-controller</code>. This can be a patch port to another bridge,
instead of a physical port, in more sophisticated setups.
</li>
</ul>
<p>
Other ports should not be attached to the integration bridge. In
particular, physical ports attached to the underlay network (as opposed to
gateway ports, which are physical ports attached to logical networks) must
not be attached to the integration bridge. Underlay physical ports should
instead be attached to a separate Open vSwitch bridge (they need not be
attached to any bridge at all, in fact).
</p>
<p>
The integration bridge should be configured as described below.
The effect of each of these settings is documented in
<code>ovs-vswitchd.conf.db</code>(5):
</p>
<!-- Keep the following in sync with create_br_int() in
ovn/controller/ovn-controller.c. -->
<dl>
<dt><code>fail-mode=secure</code></dt>
<dd>
Avoids switching packets between isolated logical networks before
<code>ovn-controller</code> starts up. See <code>Controller Failure
Settings</code> in <code>ovs-vsctl</code>(8) for more information.
</dd>
<dt><code>other-config:disable-in-band=true</code></dt>
<dd>
Suppresses in-band control flows for the integration bridge. It would be
unusual for such flows to show up anyway, because OVN uses a local
controller (over a Unix domain socket) instead of a remote controller.
It's possible, however, for some other bridge in the same system to have
an in-band remote controller, and in that case this suppresses the flows
that in-band control would ordinarily set up. Refer to the documentation
for more information.
</dd>
</dl>
<p>
The customary name for the integration bridge is <code>br-int</code>, but
another name may be used.
</p>
<h2>Logical Networks</h2>
<p>
A <dfn>logical network</dfn> implements the same concepts as physical
networks, but they are insulated from the physical network with tunnels or
other encapsulations. This allows logical networks to have separate IP and
other address spaces that overlap, without conflicting, with those used for
physical networks. Logical network topologies can be arranged without
regard for the topologies of the physical networks on which they run.
</p>
<p>
Logical network concepts in OVN include:
</p>
<ul>
<li>
<dfn>Logical switches</dfn>, the logical version of Ethernet switches.
</li>
<li>
<dfn>Logical routers</dfn>, the logical version of IP routers. Logical
switches and routers can be connected into sophisticated topologies.
</li>
<li>
<dfn>Logical datapaths</dfn> are the logical version of an OpenFlow
switch. Logical switches and routers are both implemented as logical
datapaths.
</li>
<li>
<p>
<dfn>Logical ports</dfn> represent the points of connectivity in and
out of logical switches and logical routers. Some common types of
logical ports are:
</p>
<ul>
<li>
Logical ports representing VIFs.
</li>
<li>
<dfn>Localnet ports</dfn> represent the points of connectivity
between logical switches and the physical network. They are
implemented as OVS patch ports between the integration bridge
and the separate Open vSwitch bridge that underlay physical
ports attach to.
</li>
<li>
<dfn>Logical patch ports</dfn> represent the points of
connectivity between logical switches and logical routers, and
in some cases between peer logical routers. There is a pair of
logical patch ports at each such point of connectivity, one on
each side.
</li>
<li>
<dfn>Localport ports</dfn> represent the points of local
connectivity between logical switches and VIFs. These ports are
present in every chassis (not bound to any particular one) and
traffic from them will never go through a tunnel. A
<code>localport</code> is expected to only generate traffic destined
for a local destination, typically in response to a request it
received.
One use case is how OpenStack Neutron uses a <code>localport</code>
port for serving metadata to VM's residing on every hypervisor. A
metadata proxy process is attached to this port on every host and all
VM's within the same network will reach it at the same IP/MAC address
without any traffic being sent over a tunnel. Further details can be
seen at https://docs.openstack.org/developer/networking-ovn/design/metadata_api.html.
</li>
</ul>
</li>
</ul>
<h2>Life Cycle of a VIF</h2>
<p>
Tables and their schemas presented in isolation are difficult to
understand. Here's an example.
</p>
<p>
A VIF on a hypervisor is a virtual network interface attached either
to a VM or a container running directly on that hypervisor (This is
different from the interface of a container running inside a VM).
</p>
<p>
The steps in this example refer often to details of the OVN and OVN
Northbound database schemas. Please see <code>ovn-sb</code>(5) and
<code>ovn-nb</code>(5), respectively, for the full story on these
databases.
</p>
<ol>
<li>
A VIF's life cycle begins when a CMS administrator creates a new VIF
using the CMS user interface or API and adds it to a switch (one
implemented by OVN as a logical switch). The CMS updates its own
configuration. This includes associating unique, persistent identifier
<var>vif-id</var> and Ethernet address <var>mac</var> with the VIF.
</li>
<li>
The CMS plugin updates the OVN Northbound database to include the new
VIF, by adding a row to the <code>Logical_Switch_Port</code>
table. In the new row, <code>name</code> is <var>vif-id</var>,
<code>mac</code> is <var>mac</var>, <code>switch</code> points to
the OVN logical switch's Logical_Switch record, and other columns
are initialized appropriately.
</li>
<li>
<code>ovn-northd</code> receives the OVN Northbound database update. In
turn, it makes the corresponding updates to the OVN Southbound database,
by adding rows to the OVN Southbound database <code>Logical_Flow</code>
table to reflect the new port, e.g. add a flow to recognize that packets
destined to the new port's MAC address should be delivered to it, and
update the flow that delivers broadcast and multicast packets to include
the new port. It also creates a record in the <code>Binding</code> table
and populates all its columns except the column that identifies the
<code>chassis</code>.
</li>
<li>
On every hypervisor, <code>ovn-controller</code> receives the
<code>Logical_Flow</code> table updates that <code>ovn-northd</code> made
in the previous step. As long as the VM that owns the VIF is powered
off, <code>ovn-controller</code> cannot do much; it cannot, for example,
arrange to send packets to or receive packets from the VIF, because the
VIF does not actually exist anywhere.
</li>
<li>
Eventually, a user powers on the VM that owns the VIF. On the hypervisor
where the VM is powered on, the integration between the hypervisor and
Open vSwitch (described in <code>IntegrationGuide.rst</code>) adds the VIF
to the OVN integration bridge and stores <var>vif-id</var> in
<code>external_ids</code>:<code>iface-id</code> to indicate that the
interface is an instantiation of the new VIF. (None of this code is new
in OVN; this is pre-existing integration work that has already been done
on hypervisors that support OVS.)
</li>
<li>
On the hypervisor where the VM is powered on, <code>ovn-controller</code>
notices <code>external_ids</code>:<code>iface-id</code> in the new
Interface. In response, in the OVN Southbound DB, it updates the
<code>Binding</code> table's <code>chassis</code> column for the
row that links the logical port from <code>external_ids</code>:<code>
iface-id</code> to the hypervisor. Afterward, <code>ovn-controller</code>
updates the local hypervisor's OpenFlow tables so that packets to and from
the VIF are properly handled.
</li>
<li>
Some CMS systems, including OpenStack, fully start a VM only when its
networking is ready. To support this, <code>ovn-northd</code> notices
the <code>chassis</code> column updated for the row in
<code>Binding</code> table and pushes this upward by updating the
<ref column="up" table="Logical_Switch_Port" db="OVN_NB"/> column
in the OVN Northbound database's <ref table="Logical_Switch_Port"
db="OVN_NB"/> table to indicate that the VIF is now up. The CMS,
if it uses this feature, can then react by allowing the VM's
execution to proceed.
</li>
<li>
On every hypervisor but the one where the VIF resides,
<code>ovn-controller</code> notices the completely populated row in the
<code>Binding</code> table. This provides <code>ovn-controller</code>
the physical location of the logical port, so each instance updates the
OpenFlow tables of its switch (based on logical datapath flows in the OVN
DB <code>Logical_Flow</code> table) so that packets to and from the VIF
can be properly handled via tunnels.
</li>
<li>
Eventually, a user powers off the VM that owns the VIF. On the
hypervisor where the VM was powered off, the VIF is deleted from the OVN
integration bridge.
</li>
<li>
On the hypervisor where the VM was powered off,
<code>ovn-controller</code> notices that the VIF was deleted. In
response, it removes the <code>Chassis</code> column content in the
<code>Binding</code> table for the logical port.
</li>
<li>
On every hypervisor, <code>ovn-controller</code> notices the empty
<code>Chassis</code> column in the <code>Binding</code> table's row
for the logical port. This means that <code>ovn-controller</code> no
longer knows the physical location of the logical port, so each instance
updates its OpenFlow table to reflect that.
</li>
<li>
Eventually, when the VIF (or its entire VM) is no longer needed by
anyone, an administrator deletes the VIF using the CMS user interface or
API. The CMS updates its own configuration.
</li>
<li>
The CMS plugin removes the VIF from the OVN Northbound database,
by deleting its row in the <code>Logical_Switch_Port</code> table.
</li>
<li>
<code>ovn-northd</code> receives the OVN Northbound update and in turn
updates the OVN Southbound database accordingly, by removing or updating
the rows from the OVN Southbound database <code>Logical_Flow</code> table
and <code>Binding</code> table that were related to the now-destroyed
VIF.
</li>
<li>
On every hypervisor, <code>ovn-controller</code> receives the
<code>Logical_Flow</code> table updates that <code>ovn-northd</code> made
in the previous step. <code>ovn-controller</code> updates OpenFlow
tables to reflect the update, although there may not be much to do, since
the VIF had already become unreachable when it was removed from the
<code>Binding</code> table in a previous step.
</li>
</ol>
<h2>Life Cycle of a Container Interface Inside a VM</h2>
<p>
OVN provides virtual network abstractions by converting information
written in OVN_NB database to OpenFlow flows in each hypervisor. Secure
virtual networking for multi-tenants can only be provided if OVN controller
is the only entity that can modify flows in Open vSwitch. When the
Open vSwitch integration bridge resides in the hypervisor, it is a
fair assumption to make that tenant workloads running inside VMs cannot
make any changes to Open vSwitch flows.
</p>
<p>
If the infrastructure provider trusts the applications inside the
containers not to break out and modify the Open vSwitch flows, then
containers can be run in hypervisors. This is also the case when
containers are run inside the VMs and Open vSwitch integration bridge
with flows added by OVN controller resides in the same VM. For both
the above cases, the workflow is the same as explained with an example
in the previous section ("Life Cycle of a VIF").
</p>
<p>
This section talks about the life cycle of a container interface (CIF)
when containers are created in the VMs and the Open vSwitch integration
bridge resides inside the hypervisor. In this case, even if a container
application breaks out, other tenants are not affected because the
containers running inside the VMs cannot modify the flows in the
Open vSwitch integration bridge.
</p>
<p>
When multiple containers are created inside a VM, there are multiple
CIFs associated with them. The network traffic associated with these
CIFs need to reach the Open vSwitch integration bridge running in the
hypervisor for OVN to support virtual network abstractions. OVN should
also be able to distinguish network traffic coming from different CIFs.
There are two ways to distinguish network traffic of CIFs.
</p>
<p>
One way is to provide one VIF for every CIF (1:1 model). This means that
there could be a lot of network devices in the hypervisor. This would slow
down OVS because of all the additional CPU cycles needed for the management
of all the VIFs. It would also mean that the entity creating the
containers in a VM should also be able to create the corresponding VIFs in
the hypervisor.
</p>
<p>
The second way is to provide a single VIF for all the CIFs (1:many model).
OVN could then distinguish network traffic coming from different CIFs via
a tag written in every packet. OVN uses this mechanism and uses VLAN as
the tagging mechanism.
</p>
<ol>
<li>
A CIF's life cycle begins when a container is spawned inside a VM by
the either the same CMS that created the VM or a tenant that owns that VM
or even a container Orchestration System that is different than the CMS
that initially created the VM. Whoever the entity is, it will need to
know the <var>vif-id</var> that is associated with the network interface
of the VM through which the container interface's network traffic is
expected to go through. The entity that creates the container interface
will also need to choose an unused VLAN inside that VM.
</li>
<li>
The container spawning entity (either directly or through the CMS that
manages the underlying infrastructure) updates the OVN Northbound
database to include the new CIF, by adding a row to the
<code>Logical_Switch_Port</code> table. In the new row,
<code>name</code> is any unique identifier,
<code>parent_name</code> is the <var>vif-id</var> of the VM
through which the CIF's network traffic is expected to go through
and the <code>tag</code> is the VLAN tag that identifies the
network traffic of that CIF.
</li>
<li>
<code>ovn-northd</code> receives the OVN Northbound database update. In
turn, it makes the corresponding updates to the OVN Southbound database,
by adding rows to the OVN Southbound database's <code>Logical_Flow</code>
table to reflect the new port and also by creating a new row in the
<code>Binding</code> table and populating all its columns except the
column that identifies the <code>chassis</code>.
</li>
<li>
On every hypervisor, <code>ovn-controller</code> subscribes to the
changes in the <code>Binding</code> table. When a new row is created
by <code>ovn-northd</code> that includes a value in
<code>parent_port</code> column of <code>Binding</code> table, the
<code>ovn-controller</code> in the hypervisor whose OVN integration bridge
has that same value in <var>vif-id</var> in
<code>external_ids</code>:<code>iface-id</code>
updates the local hypervisor's OpenFlow tables so that packets to and
from the VIF with the particular VLAN <code>tag</code> are properly
handled. Afterward it updates the <code>chassis</code> column of
the <code>Binding</code> to reflect the physical location.
</li>
<li>
One can only start the application inside the container after the
underlying network is ready. To support this, <code>ovn-northd</code>
notices the updated <code>chassis</code> column in <code>Binding</code>
table and updates the <ref column="up" table="Logical_Switch_Port"
db="OVN_NB"/> column in the OVN Northbound database's
<ref table="Logical_Switch_Port" db="OVN_NB"/> table to indicate that the
CIF is now up. The entity responsible to start the container application
queries this value and starts the application.
</li>
<li>
Eventually the entity that created and started the container, stops it.
The entity, through the CMS (or directly) deletes its row in the
<code>Logical_Switch_Port</code> table.
</li>
<li>
<code>ovn-northd</code> receives the OVN Northbound update and in turn
updates the OVN Southbound database accordingly, by removing or updating
the rows from the OVN Southbound database <code>Logical_Flow</code> table
that were related to the now-destroyed CIF. It also deletes the row in
the <code>Binding</code> table for that CIF.
</li>
<li>
On every hypervisor, <code>ovn-controller</code> receives the
<code>Logical_Flow</code> table updates that <code>ovn-northd</code> made
in the previous step. <code>ovn-controller</code> updates OpenFlow
tables to reflect the update.
</li>
</ol>
<h2>Architectural Physical Life Cycle of a Packet</h2>
<p>
This section describes how a packet travels from one virtual machine or
container to another through OVN. This description focuses on the physical
treatment of a packet; for a description of the logical life cycle of a
packet, please refer to the <code>Logical_Flow</code> table in
<code>ovn-sb</code>(5).
</p>
<p>
This section mentions several data and metadata fields, for clarity
summarized here:
</p>
<dl>
<dt>tunnel key</dt>
<dd>
When OVN encapsulates a packet in Geneve or another tunnel, it attaches
extra data to it to allow the receiving OVN instance to process it
correctly. This takes different forms depending on the particular
encapsulation, but in each case we refer to it here as the ``tunnel
key.'' See <code>Tunnel Encapsulations</code>, below, for details.
</dd>
<dt>logical datapath field</dt>
<dd>
A field that denotes the logical datapath through which a packet is being
processed.
<!-- Keep the following in sync with MFF_LOG_DATAPATH in
ovn/lib/logical-fields.h. -->
OVN uses the field that OpenFlow 1.1+ simply (and confusingly) calls
``metadata'' to store the logical datapath. (This field is passed across
tunnels as part of the tunnel key.)
</dd>
<dt>logical input port field</dt>
<dd>
<p>
A field that denotes the logical port from which the packet
entered the logical datapath.
<!-- Keep the following in sync with MFF_LOG_INPORT in
ovn/lib/logical-fields.h. -->
OVN stores this in Open vSwitch extension register number 14.
</p>
<p>
Geneve and STT tunnels pass this field as part of the tunnel key.
Although VXLAN tunnels do not explicitly carry a logical input port,
OVN only uses VXLAN to communicate with gateways that from OVN's
perspective consist of only a single logical port, so that OVN can set
the logical input port field to this one on ingress to the OVN logical
pipeline.
</p>
</dd>
<dt>logical output port field</dt>
<dd>
<p>
A field that denotes the logical port from which the packet will
leave the logical datapath. This is initialized to 0 at the
beginning of the logical ingress pipeline.
<!-- Keep the following in sync with MFF_LOG_OUTPORT in
ovn/lib/logical-fields.h. -->
OVN stores this in Open vSwitch extension register number 15.
</p>
<p>
Geneve and STT tunnels pass this field as part of the tunnel key.
VXLAN tunnels do not transmit the logical output port field.
Since VXLAN tunnels do not carry a logical output port field in
the tunnel key, when a packet is received from VXLAN tunnel by
an OVN hypervisor, the packet is resubmitted to table 8 to
determine the output port(s); when the packet reaches table 32,
these packets are resubmitted to table 33 for local delivery by
checking a MLF_RCV_FROM_VXLAN flag, which is set when the packet
arrives from a VXLAN tunnel.
</p>
</dd>
<dt>conntrack zone field for logical ports</dt>
<dd>
A field that denotes the connection tracking zone for logical ports.
The value only has local significance and is not meaningful between
chassis. This is initialized to 0 at the beginning of the logical
<!-- Keep the following in sync with MFF_LOG_CT_ZONE in
ovn/lib/logical-fields.h. -->
ingress pipeline. OVN stores this in Open vSwitch extension register
number 13.
</dd>
<dt>conntrack zone fields for routers</dt>
<dd>
Fields that denote the connection tracking zones for routers. These
values only have local significance and are not meaningful between
chassis. OVN stores the zone information for DNATting in Open vSwitch
<!-- Keep the following in sync with MFF_LOG_DNAT_ZONE and
MFF_LOG_SNAT_ZONE in ovn/lib/logical-fields.h. -->
extension register number 11 and zone information for SNATing in
Open vSwitch extension register number 12.
</dd>
<dt>logical flow flags</dt>
<dd>
The logical flags are intended to handle keeping context between
tables in order to decide which rules in subsequent tables are
matched. These values only have local significance and are not
meaningful between chassis. OVN stores the logical flags in
<!-- Keep the following in sync with MFF_LOG_FLAGS in
ovn/lib/logical-fields.h. -->
Open vSwitch extension register number 10.
</dd>
<dt>VLAN ID</dt>
<dd>
The VLAN ID is used as an interface between OVN and containers nested
inside a VM (see <code>Life Cycle of a container interface inside a
VM</code>, above, for more information).
</dd>
</dl>
<p>
Initially, a VM or container on the ingress hypervisor sends a packet on a
port attached to the OVN integration bridge. Then:
</p>
<ol>
<li>
<p>
OpenFlow table 0 performs physical-to-logical translation. It matches
the packet's ingress port. Its actions annotate the packet with
logical metadata, by setting the logical datapath field to identify the
logical datapath that the packet is traversing and the logical input
port field to identify the ingress port. Then it resubmits to table 8
to enter the logical ingress pipeline.
</p>
<p>
Packets that originate from a container nested within a VM are treated
in a slightly different way. The originating container can be
distinguished based on the VIF-specific VLAN ID, so the
physical-to-logical translation flows additionally match on VLAN ID and
the actions strip the VLAN header. Following this step, OVN treats
packets from containers just like any other packets.
</p>
<p>
Table 0 also processes packets that arrive from other chassis. It
distinguishes them from other packets by ingress port, which is a
tunnel. As with packets just entering the OVN pipeline, the actions
annotate these packets with logical datapath and logical ingress port
metadata. In addition, the actions set the logical output port field,
which is available because in OVN tunneling occurs after the logical
output port is known. These three pieces of information are obtained
from the tunnel encapsulation metadata (see <code>Tunnel
Encapsulations</code> for encoding details). Then the actions resubmit
to table 33 to enter the logical egress pipeline.
</p>
</li>
<li>
<p>
OpenFlow tables 8 through 31 execute the logical ingress pipeline from
the <code>Logical_Flow</code> table in the OVN Southbound database.
These tables are expressed entirely in terms of logical concepts like
logical ports and logical datapaths. A big part of
<code>ovn-controller</code>'s job is to translate them into equivalent
OpenFlow (in particular it translates the table numbers:
<code>Logical_Flow</code> tables 0 through 23 become OpenFlow tables 8
through 31).
</p>
<p>
Each logical flow maps to one or more OpenFlow flows. An actual packet
ordinarily matches only one of these, although in some cases it can
match more than one of these flows (which is not a problem because all
of them have the same actions). <code>ovn-controller</code> uses the
first 32 bits of the logical flow's UUID as the cookie for its OpenFlow
flow or flows. (This is not necessarily unique, since the first 32
bits of a logical flow's UUID is not necessarily unique.)
</p>
<p>
Some logical flows can map to the Open vSwitch ``conjunctive match''
extension (see <code>ovs-fields</code>(7)). Flows with a
<code>conjunction</code> action use an OpenFlow cookie of 0, because
they can correspond to multiple logical flows. The OpenFlow flow for a
conjunctive match includes a match on <code>conj_id</code>.
</p>
<p>
Some logical flows may not be represented in the OpenFlow tables on a
given hypervisor, if they could not be used on that hypervisor. For
example, if no VIF in a logical switch resides on a given hypervisor,
and the logical switch is not otherwise reachable on that hypervisor
(e.g. over a series of hops through logical switches and routers
starting from a VIF on the hypervisor), then the logical flow may not
be represented there.
</p>
<p>
Most OVN actions have fairly obvious implementations in OpenFlow (with
OVS extensions), e.g. <code>next;</code> is implemented as
<code>resubmit</code>, <code><var>field</var> =
<var>constant</var>;</code> as <code>set_field</code>. A few are worth
describing in more detail:
</p>
<dl>
<dt><code>output:</code></dt>
<dd>
Implemented by resubmitting the packet to table 32. If the pipeline
executes more than one <code>output</code> action, then each one is
separately resubmitted to table 32. This can be used to send
multiple copies of the packet to multiple ports. (If the packet was
not modified between the <code>output</code> actions, and some of the
copies are destined to the same hypervisor, then using a logical
multicast output port would save bandwidth between hypervisors.)
</dd>
<dt><code>get_arp(<var>P</var>, <var>A</var>);</code></dt>
<dt><code>get_nd(<var>P</var>, <var>A</var>);</code></dt>
<dd>
<p>
Implemented by storing arguments into OpenFlow fields, then
resubmitting to table 66, which <code>ovn-controller</code>
populates with flows generated from the <code>MAC_Binding</code>
table in the OVN Southbound database. If there is a match in table
66, then its actions store the bound MAC in the Ethernet
destination address field.
</p>
<p>
(The OpenFlow actions save and restore the OpenFlow fields used for
the arguments, so that the OVN actions do not have to be aware of
this temporary use.)
</p>
</dd>
<dt><code>put_arp(<var>P</var>, <var>A</var>, <var>E</var>);</code></dt>
<dt><code>put_nd(<var>P</var>, <var>A</var>, <var>E</var>);</code></dt>
<dd>
<p>
Implemented by storing the arguments into OpenFlow fields, then
outputting a packet to <code>ovn-controller</code>, which updates
the <code>MAC_Binding</code> table.
</p>
<p>
(The OpenFlow actions save and restore the OpenFlow fields used for
the arguments, so that the OVN actions do not have to be aware of
this temporary use.)
</p>
</dd>
</dl>
</li>
<li>
<p>
OpenFlow tables 32 through 47 implement the <code>output</code> action
in the logical ingress pipeline. Specifically, table 32 handles
packets to remote hypervisors, table 33 handles packets to the local
hypervisor, and table 34 checks whether packets whose logical ingress
and egress port are the same should be discarded.
</p>
<p>
Logical patch ports are a special case. Logical patch ports do not
have a physical location and effectively reside on every hypervisor.
Thus, flow table 33, for output to ports on the local hypervisor,
naturally implements output to unicast logical patch ports too.
However, applying the same logic to a logical patch port that is part
of a logical multicast group yields packet duplication, because each
hypervisor that contains a logical port in the multicast group will
also output the packet to the logical patch port. Thus, multicast
groups implement output to logical patch ports in table 32.
</p>
<p>
Each flow in table 32 matches on a logical output port for unicast or
multicast logical ports that include a logical port on a remote
hypervisor. Each flow's actions implement sending a packet to the port
it matches. For unicast logical output ports on remote hypervisors,
the actions set the tunnel key to the correct value, then send the
packet on the tunnel port to the correct hypervisor. (When the remote
hypervisor receives the packet, table 0 there will recognize it as a
tunneled packet and pass it along to table 33.) For multicast logical
output ports, the actions send one copy of the packet to each remote
hypervisor, in the same way as for unicast destinations. If a
multicast group includes a logical port or ports on the local
hypervisor, then its actions also resubmit to table 33. Table 32 also
includes:
</p>
<ul>
<li>
A higher-priority rule to match packets received from VXLAN tunnels,
based on flag MLF_RCV_FROM_VXLAN, and resubmit these packets to table
33 for local delivery. Packets received from VXLAN tunnels reach
here because of a lack of logical output port field in the tunnel key
and thus these packets needed to be submitted to table 8 to
determine the output port.
</li>
<li>
A higher-priority rule to match packets received from ports of type
<code>localport</code>, based on the logical input port, and resubmit
these packets to table 33 for local delivery. Ports of type
<code>localport</code> exist on every hypervisor and by definition
their traffic should never go out through a tunnel.
</li>
<li>
A higher-priority rule to match packets that have the MLF_LOCAL_ONLY
logical flow flag set, and whose destination is a multicast address.
This flag indicates that the packet should not be delivered to remote
hypervisors, even if the multicast destination includes ports on
remote hypervisors. This flag is used when
<code>ovn-controller</code> is the originator of the multicast packet.
Since each <code>ovn-controller</code> instance is originating these
packets, the packets only need to be delivered to local ports.
</li>
<li>
A fallback flow that resubmits to table 33 if there is no other
match.
</li>
</ul>
<p>
Flows in table 33 resemble those in table 32 but for logical ports that
reside locally rather than remotely. For unicast logical output ports
on the local hypervisor, the actions just resubmit to table 34. For
multicast output ports that include one or more logical ports on the
local hypervisor, for each such logical port <var>P</var>, the actions
change the logical output port to <var>P</var>, then resubmit to table
34.
</p>
<p>
A special case is that when a localnet port exists on the datapath,
remote port is connected by switching to the localnet port. In this
case, instead of adding a flow in table 32 to reach the remote port, a
flow is added in table 33 to switch the logical outport to the localnet
port, and resubmit to table 33 as if it were unicasted to a logical
port on the local hypervisor.
</p>
<p>
Table 34 matches and drops packets for which the logical input and
output ports are the same and the MLF_ALLOW_LOOPBACK flag is not
set. It resubmits other packets to table 40.
</p>
</li>
<li>
<p>
OpenFlow tables 40 through 63 execute the logical egress pipeline from
the <code>Logical_Flow</code> table in the OVN Southbound database.
The egress pipeline can perform a final stage of validation before
packet delivery. Eventually, it may execute an <code>output</code>
action, which <code>ovn-controller</code> implements by resubmitting to
table 64. A packet for which the pipeline never executes
<code>output</code> is effectively dropped (although it may have been
transmitted through a tunnel across a physical network).
</p>
<p>
The egress pipeline cannot change the logical output port or cause
further tunneling.
</p>
</li>
<li>
<p>
Table 64 bypasses OpenFlow loopback when MLF_ALLOW_LOOPBACK is set.
Logical loopback was handled in table 34, but OpenFlow by default also
prevents loopback to the OpenFlow ingress port. Thus, when
MLF_ALLOW_LOOPBACK is set, OpenFlow table 64 saves the OpenFlow ingress
port, sets it to zero, resubmits to table 65 for logical-to-physical
transformation, and then restores the OpenFlow ingress port,
effectively disabling OpenFlow loopback prevents. When
MLF_ALLOW_LOOPBACK is unset, table 64 flow simply resubmits to table
65.
</p>
</li>
<li>
<p>
OpenFlow table 65 performs logical-to-physical translation, the
opposite of table 0. It matches the packet's logical egress port. Its
actions output the packet to the port attached to the OVN integration
bridge that represents that logical port. If the logical egress port
is a container nested with a VM, then before sending the packet the
actions push on a VLAN header with an appropriate VLAN ID.
</p>
</li>
</ol>
<h2>Logical Routers and Logical Patch Ports</h2>
<p>
Typically logical routers and logical patch ports do not have a
physical location and effectively reside on every hypervisor. This is
the case for logical patch ports between logical routers and logical
switches behind those logical routers, to which VMs (and VIFs) attach.
</p>
<p>
Consider a packet sent from one virtual machine or container to another
VM or container that resides on a different subnet. The packet will
traverse tables 0 to 65 as described in the previous section
<code>Architectural Physical Life Cycle of a Packet</code>, using the
logical datapath representing the logical switch that the sender is
attached to. At table 32, the packet will use the fallback flow that
resubmits locally to table 33 on the same hypervisor. In this case,
all of the processing from table 0 to table 65 occurs on the hypervisor
where the sender resides.
</p>
<p>
When the packet reaches table 65, the logical egress port is a logical
patch port. The implementation in table 65 differs depending on the OVS
version, although the observed behavior is meant to be the same:
</p>
<ul>
<li>
In OVS versions 2.6 and earlier, table 65 outputs to an OVS patch
port that represents the logical patch port. The packet re-enters
the OpenFlow flow table from the OVS patch port's peer in table 0,
which identifies the logical datapath and logical input port based
on the OVS patch port's OpenFlow port number.
</li>
<li>
In OVS versions 2.7 and later, the packet is cloned and resubmitted
directly to the first OpenFlow flow table in the ingress pipeline,
setting the logical ingress port to the peer logical patch port, and
using the peer logical patch port's logical datapath (that
represents the logical router).
</li>
</ul>
<p>
The packet re-enters the ingress pipeline in order to traverse tables
8 to 65 again, this time using the logical datapath representing the
logical router. The processing continues as described in the previous
section <code>Architectural Physical Life Cycle of a Packet</code>.
When the packet reachs table 65, the logical egress port will once
again be a logical patch port. In the same manner as described above,
this logical patch port will cause the packet to be resubmitted to
OpenFlow tables 8 to 65, this time using the logical datapath
representing the logical switch that the destination VM or container
is attached to.
</p>
<p>
The packet traverses tables 8 to 65 a third and final time. If the
destination VM or container resides on a remote hypervisor, then table
32 will send the packet on a tunnel port from the sender's hypervisor
to the remote hypervisor. Finally table 65 will output the packet
directly to the destination VM or container.
</p>
<p>
The following sections describe two exceptions, where logical routers
and/or logical patch ports are associated with a physical location.
</p>
<h3>Gateway Routers</h3>
<p>
A <dfn>gateway router</dfn> is a logical router that is bound to a
physical location. This includes all of the logical patch ports of
the logical router, as well as all of the peer logical patch ports on
logical switches. In the OVN Southbound database, the
<code>Port_Binding</code> entries for these logical patch ports use
the type <code>l3gateway</code> rather than <code>patch</code>, in
order to distinguish that these logical patch ports are bound to a
chassis.
</p>
<p>
When a hypervisor processes a packet on a logical datapath
representing a logical switch, and the logical egress port is a
<code>l3gateway</code> port representing connectivity to a gateway
router, the packet will match a flow in table 32 that sends the
packet on a tunnel port to the chassis where the gateway router
resides. This processing in table 32 is done in the same manner as
for VIFs.
</p>
<p>
Gateway routers are typically used in between distributed logical
routers and physical networks. The distributed logical router and
the logical switches behind it, to which VMs and containers attach,
effectively reside on each hypervisor. The distributed router and
the gateway router are connected by another logical switch, sometimes
referred to as a <code>join</code> logical switch. On the other
side, the gateway router connects to another logical switch that has
a localnet port connecting to the physical network.
</p>
<p>
When using gateway routers, DNAT and SNAT rules are associated with
the gateway router, which provides a central location that can handle
one-to-many SNAT (aka IP masquerading).
</p>
<h3>Distributed Gateway Ports</h3>
<p>
<dfn>Distributed gateway ports</dfn> are logical router patch ports
that directly connect distributed logical routers to logical
switches with localnet ports.
</p>
<p>
The primary design goal of distributed gateway ports is to allow as
much traffic as possible to be handled locally on the hypervisor
where a VM or container resides. Whenever possible, packets from
the VM or container to the outside world should be processed
completely on that VM's or container's hypervisor, eventually
traversing a localnet port instance on that hypervisor to the
physical network. Whenever possible, packets from the outside
world to a VM or container should be directed through the physical
network directly to the VM's or container's hypervisor, where the
packet will enter the integration bridge through a localnet port.
</p>
<p>
In order to allow for the distributed processing of packets
described in the paragraph above, distributed gateway ports need to
be logical patch ports that effectively reside on every hypervisor,
rather than <code>l3gateway</code> ports that are bound to a
particular chassis. However, the flows associated with distributed
gateway ports often need to be associated with physical locations,
for the following reasons:
</p>
<ul>
<li>
<p>
The physical network that the localnet port is attached to
typically uses L2 learning. Any Ethernet address used over the
distributed gateway port must be restricted to a single physical
location so that upstream L2 learning is not confused. Traffic
sent out the distributed gateway port towards the localnet port
with a specific Ethernet address must be sent out one specific
instance of the distributed gateway port on one specific
chassis. Traffic received from the localnet port (or from a VIF
on the same logical switch as the localnet port) with a specific
Ethernet address must be directed to the logical switch's patch
port instance on that specific chassis.
</p>
<p>
Due to the implications of L2 learning, the Ethernet address and
IP address of the distributed gateway port need to be restricted
to a single physical location. For this reason, the user must
specify one chassis associated with the distributed gateway
port. Note that traffic traversing the distributed gateway port
using other Ethernet addresses and IP addresses (e.g. one-to-one
NAT) is not restricted to this chassis.
</p>
<p>
Replies to ARP and ND requests must be restricted to a single
physical location, where the Ethernet address in the reply
resides. This includes ARP and ND replies for the IP address
of the distributed gateway port, which are restricted to the
chassis that the user associated with the distributed gateway
port.
</p>
</li>
<li>
In order to support one-to-many SNAT (aka IP masquerading), where
multiple logical IP addresses spread across multiple chassis are
mapped to a single external IP address, it will be necessary to
handle some of the logical router processing on a specific chassis
in a centralized manner. Since the SNAT external IP address is
typically the distributed gateway port IP address, and for
simplicity, the same chassis associated with the distributed
gateway port is used.
</li>
</ul>
<p>
The details of flow restrictions to specific chassis are described
in the <code>ovn-northd</code> documentation.
</p>
<p>
While most of the physical location dependent aspects of distributed
gateway ports can be handled by restricting some flows to specific
chassis, one additional mechanism is required. When a packet
leaves the ingress pipeline and the logical egress port is the
distributed gateway port, one of two different sets of actions is
required at table 32:
</p>
<ul>
<li>
If the packet can be handled locally on the sender's hypervisor
(e.g. one-to-one NAT traffic), then the packet should just be
resubmitted locally to table 33, in the normal manner for
distributed logical patch ports.
</li>
<li>
However, if the packet needs to be handled on the chassis
associated with the distributed gateway port (e.g. one-to-many
SNAT traffic or non-NAT traffic), then table 32 must send the
packet on a tunnel port to that chassis.
</li>
</ul>
<p>
In order to trigger the second set of actions, the
<code>chassisredirect</code> type of southbound
<code>Port_Binding</code> has been added. Setting the logical
egress port to the type <code>chassisredirect</code> logical port is
simply a way to indicate that although the packet is destined for
the distributed gateway port, it needs to be redirected to a
different chassis. At table 32, packets with this logical egress
port are sent to a specific chassis, in the same way that table 32
directs packets whose logical egress port is a VIF or a type
<code>l3gateway</code> port to different chassis. Once the packet
arrives at that chassis, table 33 resets the logical egress port to
the value representing the distributed gateway port. For each
distributed gateway port, there is one type
<code>chassisredirect</code> port, in addition to the distributed
logical patch port representing the distributed gateway port.
</p>
<h3>High Availability for Distributed Gateway Ports</h3>
<p>
OVN allows you to specify a prioritized list of chassis for a distributed
gateway port. This is done by associating multiple
<code>Gateway_Chassis</code> rows with a <code>Logical_Router_Port</code>
in the <code>OVN_Northbound</code> database.
</p>
<p>
When multiple chassis have been specified for a gateway, all chassis that
may send packets to that gateway will enable BFD on tunnels to all
configured gateway chassis. The current master chassis for the gateway
is the highest priority gateway chassis that is currently viewed as
active based on BFD status.
</p>
<p>
For more information on L3 gateway high availability, please refer to
http://docs.openvswitch.org/en/latest/topics/high-availability.
</p>
<h2>Multiple localnet logical switches connected to a Logical Router</h2>
<p>
It is possible to have multiple logical switches each with a localnet port
(representing physical networks) connected to a logical router, in which
one localnet logical switch may provide the external connectivity via a
distributed gateway port and rest of the localnet logical switches use
VLAN tagging in the physical network. It is expected that
<code>ovn-bridge-mappings</code> is configured appropriately on the
chassis for all these localnet networks.
</p>
<h3>East West routing</h3>
<p>
East-West routing between these localnet VLAN tagged logical switches
work almost the same way as normal logical switches. When the VM sends
such a packet, then:
</p>
<ol>
<li>
It first enters the ingress pipeline, and then egress pipeline of the
source localnet logical switch datapath. It then enters the ingress
pipeline of the logical router datapath via the logical router port in
the source chassis.
</li>
<li>
Routing decision is taken.
</li>
<li>
From the router datapath, packet enters the ingress pipeline and then
egress pipeline of the destination localnet logical switch datapath
and goes out of the integration bridge to the provider bridge (
belonging to the destination logical switch) via the localnet port.
</li>
<li>
The destination chassis receives the packet via the localnet port and
sends it to the integration bridge. The packet enters the
ingress pipeline and then egress pipeline of the destination localnet
logical switch and finally gets delivered to the destination VM port.
</li>
</ol>
<h3>External traffic</h3>
<p>
The following happens when a VM sends an external traffic (which requires
NATting) and the chassis hosting the VM doesn't have a distributed gateway
port.
</p>
<ol>
<li>
The packet first enters the ingress pipeline, and then egress pipeline of
the source localnet logical switch datapath. It then enters the ingress
pipeline of the logical router datapath via the logical router port in
the source chassis.
</li>
<li>
Routing decision is taken. Since the gateway router or the distributed
gateway port doesn't reside in the source chassis, the traffic is
redirected to the gateway chassis via the tunnel port.
</li>
<li>
The gateway chassis receives the packet via the tunnel port and the
packet enters the egress pipeline of the logical router datapath. NAT
rules are applied here. The packet then enters the ingress pipeline and
then egress pipeline of the localnet logical switch datapath which
provides external connectivity and finally goes out via the localnet
port of the logical switch which provides external connectivity.
</li>
</ol>
<p>
Although this works, the VM traffic is tunnelled when sent from the compute
chassis to the gateway chassis. In order for it to work properly, the MTU
of the localnet logical switches must be lowered to account for the tunnel
encapsulation.
</p>
<h2>
Centralized routing for localnet VLAN tagged logical switches connected
to a Logical Router
</h2>
<p>
To overcome the tunnel encapsulation problem described in the previous
section, <code>OVN</code> supports the option of enabling centralized
routing for localnet VLAN tagged logical switches. CMS can configure the
option <ref column="options:reside-on-redirect-chassis"
table="Logical_Router_Port" db="OVN_NB"/> to <code>true</code> for each
<ref table="Logical_Router_Port" db="OVN_NB"/> which connects to the
localnet VLAN tagged logical switches. This causes the gateway
chassis (hosting the distributed gateway port) to handle all the
routing for these networks, making it centralized. It will reply to
the ARP requests for the logical router port IPs.
</p>
<p>
If the logical router doesn't have a distributed gateway port connecting
to the localnet logical switch which provides external connectivity,
then this option is ignored by <code>OVN</code>.
</p>
<p>
The following happens when a VM sends an east-west traffic which needs to
be routed:
</p>
<ol>
<li>
The packet first enters the ingress pipeline, and then egress pipeline of
the source localnet logical switch datapath and is sent out via the
localnet port of the source localnet logical switch (instead of sending
it to router pipeline).
</li>
<li>
The gateway chassis receives the packet via the localnet port of the
source localnet logical switch and sends it to the integration bridge.
The packet then enters the ingress pipeline, and then egress pipeline of
the source localnet logical switch datapath and enters the ingress
pipeline of the logical router datapath.
</li>
<li>
Routing decision is taken.
</li>
<li>
From the router datapath, packet enters the ingress pipeline and then
egress pipeline of the destination localnet logical switch datapath.
It then goes out of the integration bridge to the provider bridge (
belonging to the destination logical switch) via the localnet port.
</li>
<li>
The destination chassis receives the packet via the localnet port and
sends it to the integration bridge. The packet enters the
ingress pipeline and then egress pipeline of the destination localnet
logical switch and finally delivered to the destination VM port.
</li>
</ol>
<p>
The following happens when a VM sends an external traffic which requires
NATting:
</p>
<ol>
<li>
The packet first enters the ingress pipeline, and then egress pipeline of
the source localnet logical switch datapath and is sent out via the
localnet port of the source localnet logical switch (instead of sending
it to router pipeline).
</li>
<li>
The gateway chassis receives the packet via the localnet port of the
source localnet logical switch and sends it to the integration bridge.
The packet then enters the ingress pipeline, and then egress pipeline of
the source localnet logical switch datapath and enters the ingress
pipeline of the logical router datapath.
</li>
<li>
Routing decision is taken and NAT rules are applied.
</li>
<li>
From the router datapath, packet enters the ingress pipeline and then
egress pipeline of the localnet logical switch datapath which provides
external connectivity. It then goes out of the integration bridge to the
provider bridge (belonging to the logical switch which provides external
connectivity) via the localnet port.
</li>
</ol>
<p>
The following happens for the reverse external traffic.
</p>
<ol>
<li>
The gateway chassis receives the packet from the localnet port of
the logical switch which provides external connectivity. The packet then
enters the ingress pipeline and then egress pipeline of the localnet
logical switch (which provides external connectivity). The packet then
enters the ingress pipeline of the logical router datapath.
</li>
<li>
The ingress pipeline of the logical router datapath applies the unNATting
rules. The packet then enters the ingress pipeline and then egress
pipeline of the source localnet logical switch. Since the source VM
doesn't reside in the gateway chassis, the packet is sent out via the
localnet port of the source logical switch.
</li>
<li>
The source chassis receives the packet via the localnet port and
sends it to the integration bridge. The packet enters the
ingress pipeline and then egress pipeline of the source localnet
logical switch and finally gets delivered to the source VM port.
</li>
</ol>
<h2>Life Cycle of a VTEP gateway</h2>
<p>
A gateway is a chassis that forwards traffic between the OVN-managed
part of a logical network and a physical VLAN, extending a
tunnel-based logical network into a physical network.
</p>
<p>
The steps below refer often to details of the OVN and VTEP database
schemas. Please see <code>ovn-sb</code>(5), <code>ovn-nb</code>(5)
and <code>vtep</code>(5), respectively, for the full story on these
databases.
</p>
<ol>
<li>
A VTEP gateway's life cycle begins with the administrator registering
the VTEP gateway as a <code>Physical_Switch</code> table entry in the
<code>VTEP</code> database. The <code>ovn-controller-vtep</code>
connected to this VTEP database, will recognize the new VTEP gateway
and create a new <code>Chassis</code> table entry for it in the
<code>OVN_Southbound</code> database.
</li>
<li>
The administrator can then create a new <code>Logical_Switch</code>
table entry, and bind a particular vlan on a VTEP gateway's port to
any VTEP logical switch. Once a VTEP logical switch is bound to
a VTEP gateway, the <code>ovn-controller-vtep</code> will detect
it and add its name to the <var>vtep_logical_switches</var>
column of the <code>Chassis</code> table in the <code>
OVN_Southbound</code> database. Note, the <var>tunnel_key</var>
column of VTEP logical switch is not filled at creation. The
<code>ovn-controller-vtep</code> will set the column when the
correponding vtep logical switch is bound to an OVN logical network.
</li>
<li>
Now, the administrator can use the CMS to add a VTEP logical switch
to the OVN logical network. To do that, the CMS must first create a
new <code>Logical_Switch_Port</code> table entry in the <code>
OVN_Northbound</code> database. Then, the <var>type</var> column
of this entry must be set to "vtep". Next, the <var>
vtep-logical-switch</var> and <var>vtep-physical-switch</var> keys
in the <var>options</var> column must also be specified, since
multiple VTEP gateways can attach to the same VTEP logical switch.
</li>
<li>
The newly created logical port in the <code>OVN_Northbound</code>
database and its configuration will be passed down to the <code>
OVN_Southbound</code> database as a new <code>Port_Binding</code>
table entry. The <code>ovn-controller-vtep</code> will recognize the
change and bind the logical port to the corresponding VTEP gateway
chassis. Configuration of binding the same VTEP logical switch to
a different OVN logical networks is not allowed and a warning will be
generated in the log.
</li>
<li>
Beside binding to the VTEP gateway chassis, the <code>
ovn-controller-vtep</code> will update the <var>tunnel_key</var>
column of the VTEP logical switch to the corresponding <code>
Datapath_Binding</code> table entry's <var>tunnel_key</var> for the
bound OVN logical network.
</li>
<li>
Next, the <code>ovn-controller-vtep</code> will keep reacting to the
configuration change in the <code>Port_Binding</code> in the
<code>OVN_Northbound</code> database, and updating the
<code>Ucast_Macs_Remote</code> table in the <code>VTEP</code> database.
This allows the VTEP gateway to understand where to forward the unicast
traffic coming from the extended external network.
</li>
<li>
Eventually, the VTEP gateway's life cycle ends when the administrator
unregisters the VTEP gateway from the <code>VTEP</code> database.
The <code>ovn-controller-vtep</code> will recognize the event and
remove all related configurations (<code>Chassis</code> table entry
and port bindings) in the <code>OVN_Southbound</code> database.
</li>
<li>
When the <code>ovn-controller-vtep</code> is terminated, all related
configurations in the <code>OVN_Southbound</code> database and
the <code>VTEP</code> database will be cleaned, including
<code>Chassis</code> table entries for all registered VTEP gateways
and their port bindings, and all <code>Ucast_Macs_Remote</code> table
entries and the <code>Logical_Switch</code> tunnel keys.
</li>
</ol>
<h2>Native OVN services for external logical ports</h2>
<p>
To support OVN native services (like DHCP/IPv6 RA/DNS lookup) to the
cloud resources which are external, OVN supports <code>external</code>
logical ports.
</p>
<p>
Below are some of the use cases where <code>external</code> ports can be
used.
</p>
<ul>
<li>
VMs connected to SR-IOV nics - Traffic from these VMs by passes the
kernel stack and local <code>ovn-controller</code> do not bind these
ports and cannot serve the native services.
</li>
<li>
When CMS supports provisioning baremetal servers.
</li>
</ul>
<p>
OVN will provide the native services if CMS has done the below
configuration in the <dfn>OVN Northbound Database</dfn>.
</p>
<ul>
<li>
A row is created in <code>Logical_Switch_Port</code>, configuring the
<ref column="addresses" table="Logical_Switch_Port" db="OVN_NB"/> column
and setting the <ref column="type" table="Logical_Switch_Port"
db="OVN_NB"/> to <code>external</code>.
</li>
<li>
<ref column="ha_chassis_group" table="Logical_Switch_Port"
db="OVN_NB"/> column is configured.
</li>
<li>
The HA chassis which belongs to the HA chassis group has the
<code>ovn-bridge-mappings</code> configured and has proper L2
connectivity so that it can receive the DHCP and other related request
packets from these external resources.
</li>
<li>
The Logical_Switch of this port has a <code>localnet</code> port.
</li>
<li>
Native OVN services are enabled by configuring the DHCP and other
options like the way it is done for the normal logical ports.
</li>
</ul>
<p>
It is recommended to use the same HA chassis group for all the external
ports of a logical switch. Otherwise, the physical switch might see MAC
flap issue when different chassis provide the native services. For
example when supporting native DHCPv4 service, DHCPv4 server mac
(configured in <ref column="options:server_mac" table="DHCP_Options"
db="OVN_NB"/> column in table <ref table="DHCP_Options"/>) originating
from different ports can cause MAC flap issue.
The MAC of the logical router IP(s) can also flap if the same HA chassis
group is not set for all the external ports of a logical switch.
</p>
<h1>Security</h1>
<h2>Role-Based Access Controls for the Soutbound DB</h2>
<p>
In order to provide additional security against the possibility of an OVN
chassis becoming compromised in such a way as to allow rogue software to
make arbitrary modifications to the southbound database state and thus
disrupt the OVN network, role-based access controls (see
<code>ovsdb-server(1)</code> for additional details) are provided for the
southbound database.
</p>
<p>
The implementation of role-based access controls (RBAC) requires the
addition of two tables to an OVSDB schema: the <code>RBAC_Role</code>
table, which is indexed by role name and maps the the names of the various
tables that may be modifiable for a given role to individual rows in a
permissions table containing detailed permission information for that role,
and the permission table itself which consists of rows containing the
following information:
</p>
<dl>
<dt><code>Table Name</code></dt>
<dd>
The name of the associated table. This column exists primarily as an
aid for humans reading the contents of this table.
</dd>
<dt><code>Auth Criteria</code></dt>
<dd>
A set of strings containing the names of columns (or column:key pairs
for columns containing string:string maps). The contents of at least
one of the columns or column:key values in a row to be modified,
inserted, or deleted must be equal to the ID of the client attempting
to act on the row in order for the authorization check to pass. If the
authorization criteria is empty, authorization checking is disabled and
all clients for the role will be treated as authorized.
</dd>
<dt><code>Insert/Delete</code></dt>
<dd>
Row insertion/deletion permission; boolean value indicating whether
insertion and deletion of rows is allowed for the associated table.
If true, insertion and deletion of rows is allowed for authorized
clients.
</dd>
<dt><code>Updatable Columns</code></dt>
<dd>
A set of strings containing the names of columns or column:key pairs
that may be updated or mutated by authorized clients. Modifications to
columns within a row are only permitted when the authorization check
for the client passes and all columns to be modified are included in
this set of modifiable columns.
</dd>
</dl>
<p>
RBAC configuration for the OVN southbound database is maintained by
ovn-northd. With RBAC enabled, modifications are only permitted for the
<code>Chassis</code>, <code>Encap</code>, <code>Port_Binding</code>, and
<code>MAC_Binding</code> tables, and are resstricted as follows:
</p>
<dl>
<dt><code>Chassis</code></dt>
<dd>
<p>
<code>Authorization</code>: client ID must match the chassis name.
</p>
<p>
<code>Insert/Delete</code>: authorized row insertion and deletion
are permitted.
</p>
<p>
<code>Update</code>: The columns <code>nb_cfg</code>,
<code>external_ids</code>, <code>encaps</code>, and
<code>vtep_logical_switches</code> may be modified when authorized.
</p>
</dd>
<dt><code>Encap</code></dt>
<dd>
<p>
<code>Authorization</code>: client ID must match the chassis name.
</p>
<p>
<code>Insert/Delete</code>: row insertion and row deletion
are permitted.
</p>
<p>
<code>Update</code>: The columns <code>type</code>,
<code>options</code>, and <code>ip</code> can be modified.
</p>
</dd>
<dt><code>Port_Binding</code></dt>
<dd>
<p>
<code>Authorization</code>: disabled (all clients are considered
authorized. A future enhancement may add columns (or keys to
<code>external_ids</code>) in order to control which chassis are
allowed to bind each port.
</p>
<p>
<code>Insert/Delete</code>: row insertion/deletion are not permitted
(ovn-northd maintains rows in this table.
</p>
<p>
<code>Update</code>: Only modifications to the <code>chassis</code>
column are permitted.
</p>
</dd>
<dt><code>MAC_Binding</code></dt>
<dd>
<p>
<code>Authorization</code>: disabled (all clients are considered
to be authorized).
</p>
<p>
<code>Insert/Delete</code>: row insertion/deletion are permitted.
</p>
<p>
<code>Update</code>: The columns <code>logical_port</code>,
<code>ip</code>, <code>mac</code>, and <code>datapath</code> may be
modified by ovn-controller.
</p>
</dd>
</dl>
<p>
Enabling RBAC for ovn-controller connections to the southbound database
requires the following steps:
</p>
<ol>
<li>
Creating SSL certificates for each chassis with the certificate CN field
set to the chassis name (e.g. for a chassis with
<code>external-ids:system-id=chassis-1</code>, via the command
"<code>ovs-pki -u req+sign chassis-1 switch</code>").
</li>
<li>
Configuring each ovn-controller to use SSL when connecting to the
southbound database (e.g. via "<code>ovs-vsctl set open .
external-ids:ovn-remote=ssl:x.x.x.x:6642</code>").
</li>
<li>
Configuring a southbound database SSL remote with "ovn-controller" role
(e.g. via "<code>ovn-sbctl set-connection role=ovn-controller
pssl:6642</code>").
</li>
</ol>
<h2>Encrypt Tunnel Traffic with IPsec</h2>
<p>
OVN tunnel traffic goes through physical routers and switches. These
physical devices could be untrusted (devices in public network) or might be
compromised. Enabling encryption to the tunnel traffic can prevent the
traffic data from being monitored and manipulated.
</p>
<p>
The tunnel traffic is encrypted with IPsec. The CMS sets the
<code>ipsec</code> column in the northbound <code>NB_Global</code> table to
enable or disable IPsec encrytion. If <code>ipsec</code> is true, all OVN
tunnels will be encrypted. If <code>ipsec</code> is false, no OVN tunnels
will be encrypted.
</p>
<p>
When CMS updates the <code>ipsec</code> column in the northbound
<code>NB_Global</code> table, <code>ovn-northd</code> copies the value to
the <code>ipsec</code> column in the southbound <code>SB_Global</code>
table. <code>ovn-controller</code> in each chassis monitors the southbound
database and sets the options of the OVS tunnel interface accordingly. OVS
tunnel interface options are monitored by the
<code>ovs-monitor-ipsec</code> daemon which configures IKE daemon to set up
IPsec connections.
</p>
<p>
Chassis authenticates each other by using certificate. The authentication
succeeds if the other end in tunnel presents a certificate signed by a
trusted CA and the common name (CN) matches the expected chassis name. The
SSL certificates used in role-based access controls (RBAC) can be used in
IPsec. Or use <code>ovs-pki</code> to create different certificates. The
certificate is required to be x.509 version 3, and with CN field and
subjectAltName field being set to the chassis name.
</p>
<p>
The CA certificate, chassis certificate and private key are required to be
installed in each chassis before enabling IPsec. Please see
<code>ovs-vswitchd.conf.db</code>(5) for setting up CA based IPsec
authentication.
</p>
<h1>Design Decisions</h1>
<h2>Tunnel Encapsulations</h2>
<p>
OVN annotates logical network packets that it sends from one hypervisor to
another with the following three pieces of metadata, which are encoded in
an encapsulation-specific fashion:
</p>
<ul>
<li>
24-bit logical datapath identifier, from the <code>tunnel_key</code>
column in the OVN Southbound <code>Datapath_Binding</code> table.
</li>
<li>
15-bit logical ingress port identifier. ID 0 is reserved for internal
use within OVN. IDs 1 through 32767, inclusive, may be assigned to
logical ports (see the <code>tunnel_key</code> column in the OVN
Southbound <code>Port_Binding</code> table).
</li>
<li>
16-bit logical egress port identifier. IDs 0 through 32767 have the same
meaning as for logical ingress ports. IDs 32768 through 65535,
inclusive, may be assigned to logical multicast groups (see the
<code>tunnel_key</code> column in the OVN Southbound
<code>Multicast_Group</code> table).
</li>
</ul>
<p>
For hypervisor-to-hypervisor traffic, OVN supports only Geneve and STT
encapsulations, for the following reasons:
</p>
<ul>
<li>
Only STT and Geneve support the large amounts of metadata (over 32 bits
per packet) that OVN uses (as described above).
</li>
<li>
STT and Geneve use randomized UDP or TCP source ports that allows
efficient distribution among multiple paths in environments that use ECMP
in their underlay.
</li>
<li>
NICs are available to offload STT and Geneve encapsulation and
decapsulation.
</li>
</ul>
<p>
Due to its flexibility, the preferred encapsulation between hypervisors is
Geneve. For Geneve encapsulation, OVN transmits the logical datapath
identifier in the Geneve VNI.
<!-- Keep the following in sync with ovn/controller/physical.h. -->
OVN transmits the logical ingress and logical egress ports in a TLV with
class 0x0102, type 0x80, and a 32-bit value encoded as follows, from MSB to
LSB:
</p>
<diagram>
<header name="">
<bits name="rsv" above="1" below="0" width=".25"/>
<bits name="ingress port" above="15" width=".75"/>
<bits name="egress port" above="16" width=".75"/>
</header>
</diagram>
<p>
Environments whose NICs lack Geneve offload may prefer STT encapsulation
for performance reasons. For STT encapsulation, OVN encodes all three
pieces of logical metadata in the STT 64-bit tunnel ID as follows, from MSB
to LSB:
</p>
<diagram>
<header name="">
<bits name="reserved" above="9" below="0" width=".5"/>
<bits name="ingress port" above="15" width=".75"/>
<bits name="egress port" above="16" width=".75"/>
<bits name="datapath" above="24" width="1.25"/>
</header>
</diagram>
<p>
For connecting to gateways, in addition to Geneve and STT, OVN supports
VXLAN, because only VXLAN support is common on top-of-rack (ToR) switches.
Currently, gateways have a feature set that matches the capabilities as
defined by the VTEP schema, so fewer bits of metadata are necessary. In
the future, gateways that do not support encapsulations with large amounts
of metadata may continue to have a reduced feature set.
</p>
</manpage>
|