delta/openvswitch.git/lib/ovsdb-idl.h, branch master

ovsdb-idl: Provide API to disable set_db_change_aware request.

2023-01-16T19:14:10+00:00

For ovsdb clients that are short-lived, e.g. when using
ovn-nbctl/ovn-sbctl to read some metrics from the OVN NB/SB server, they
don't really need to be aware of db changes, because they exit
immediately after getting the initial response for the requested data.
In such use cases, however, the clients still send 'set_db_change_aware'
request, which results in server side error logs when the server tries
to send out the response for the 'set_db_change_aware' request, because
at the moment the client that is supposed to receive the request has
already closed the connection and exited. E.g.:

2023-01-10T18:23:29.431Z|00007|jsonrpc|WARN|unix#3: receive error: Connection reset by peer
2023-01-10T18:23:29.431Z|00008|reconnect|WARN|unix#3: connection dropped (Connection reset by peer)

To avoid such problems, this patch provides an API to allow a client to
choose to not send the 'set_db_change_aware' request.

There was an earlier attempt to fix this [0], but it was not accepted
back then as discussed in the email [1]. It was also discussed in the
emails that an alternative approach is to use notification instead of
request, but that would require protocol changes and taking backward
compatibility into consideration. So this patch takes a different
approach and tries to keep the change small.

[0] http://patchwork.ozlabs.org/project/openvswitch/patch/1594380801-32134-1-git-send-email-dceara@redhat.com/

[1] https://mail.openvswitch.org/pipermail/ovs-discuss/2021-February/050919.html

Reported-by: Girish Moodalbail 
Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2020-July/050343.html
Reported-by: Tobias Hofmann 
Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2021-February/050914.html
Acked-by: Dumitru Ceara 
Signed-off-by: Han Zhou 
Signed-off-by: Ilya Maximets

ovsdb-idl: Add the support to specify the uuid for row insert.

2022-11-30T14:15:57+00:00

ovsdb-server allows the OVSDB clients to specify the uuid for
the row inserts [1].  Both the C IDL client library and Python
IDL are missing this feature.  This patch adds this support.

In C IDL, for each schema table, a new function is generated -
insert_persistent_uuid(txn, uuid) which can
be used the clients to persist the uuid.

ovs-vsctl and other derivatives of ctl now supports the same
in the generic 'create' command with the option "--id=".

In Python IDL, the uuid to persist can be specified in
the Transaction.insert() function.

[1] - a529e3cd1f("ovsdb-server: Allow OVSDB clients to specify the UUID for inserted rows.:)

Acked-by: Adrian Moreno 
Acked-by: Han Zhou 
Acked-by: Terry Wilson 
Signed-off-by: Numan Siddique 
Signed-off-by: Ilya Maximets

ovsdb-idl: Support write-only-changed IDL monitor mode.

2022-04-28T14:57:43+00:00

At a first glance, change tracking should never be allowed for
write-only columns.  However, some clients (e.g., ovn-northd) that are
mostly exclusive writers of a database, use change tracking to avoid
duplicating the IDL row records into a local cache when implementing
incremental processing.

The default behavior of the IDL is to automatically turn a write-only
column into a read-write column whenever the client enables change
tracking for that column.

For the afore mentioned clients, this becomes a performance issue.
Commit 1cc618c32524 ("ovsdb-idl: Fix atomicity of writes that don't
change a column's value.") explains why writes that don't change a
column's value cannot be optimized out early if the column is
read/write.

Furthermore, if there is at least one record in any table that
changed during a transaction, then *all* records that have been
written are added to the transaction, even if their values didn't
change.  If there are many such rows (e.g., like in ovn-northd's
case) this incurs a significant overhead because:
a. the client has to build this large transaction
b. the transaction has to be sent over the network
c. the server needs to parse this (mostly) no-op update

We now introduce new IDL APIs allowing users to set a new monitoring
mode flag, OVSDB_IDL_WRITE_CHANGED_ONLY, to indicate to the IDL that the
atomicity constraints may be relaxed and written columns that don't
change value can be skipped from the current transaction.

We benchmarked ovn-northd performance when using this new mode
against NB and SB databases taken from ovn-kubernetes scale tests.
We noticed that when a minor change is performed to the Northbound
database (e.g., NB_Global.nb_cfg is incremented) the time it takes to
build the Southbound transaction becomes negligible (vs ~1.5 seconds
before this change).

End-to-end ovn-kubernetes scale tests on 120-node clusters also show
significant reduction of latency to bring up pods; both average and P99
latency decreased by ~30%.

Acked-by: Han Zhou 
Signed-off-by: Dumitru Ceara 
Signed-off-by: Ilya Maximets

ovsdb-idl: Add memory report function.

2021-11-04T22:13:13+00:00

Added new function to return memory usage statistics for database
objects inside IDL.  Statistics similar to what ovsdb-server reports.
Not counting _Server database as it should be small, hence doesn't
worth adding extra code to the ovsdb-cs module.  Can be added later
if needed.

ovs-vswitchd is a user in OVS, but this API will be mostly useful for
OVN daemons.

Signed-off-by: Ilya Maximets 
Acked-by: Han Zhou 
Acked-by: Dumitru Ceara

ovsdb-idl: Add APIs to query if a table and a column is present.

2021-08-28T00:59:04+00:00

This patch adds 2 new APIs in the ovsdb-idl client library
 - ovsdb_idl_server_has_table() and ovsdb_idl_server_has_column() to
query if a table and a column is present in the IDL or not.  This
patch also adds IDL helper functions which are auto generated from
the schema which makes it easier for the clients.

These APIs are required for scenarios where the server schema is old and
missing a table or column and the client (built with a new schema
version) does a transaction with the missing table or column.  This
results in a continuous loop of transaction failures.

Related-Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1992705
Signed-off-by: Numan Siddique 
Signed-off-by: Ilya Maximets

ovsdb-idl.at: Make test outputs more predictable.

2021-04-01T11:53:20+00:00

IDL tests need predictable output from test-ovsdb.

This used to be done by first sorting the output of test-ovsdb and then
applying uuidfilt to predictably translate UUIDs.  This was not
reliable enough in case test-ovsdb processes two or more insert/delete
operations in the same iteration because the order of lines in the
output depends on the automatically generated UUID values.

To fix this we change the way test-ovsdb and test-ovsdb.py generate
outputs and prepend the table name and tracking information before
printing the contents of a row.

All existing ovsdb-idl.at and ovsdb-cluster.at tests are updated to
expect the new output format.

Signed-off-by: Dumitru Ceara 
Acked-by: Han Zhou 
Signed-off-by: Ilya Maximets

ovsdb-idl: Improve prototypes.

2020-12-19T02:32:14+00:00

Adding parameter names makes these prototypes clearer.

Signed-off-by: Ben Pfaff 
Acked-by: Ilya Maximets

ovsdb-idl: Remove prototype for function that is not defined or used.

2020-12-19T02:32:10+00:00

Signed-off-by: Ben Pfaff 
Acked-by: Ilya Maximets

Eliminate "whitelist" and "blacklist" terms.

2020-10-17T02:22:24+00:00

There is one remaining use under datapath.  That change should happen
upstream in Linux first according to our usual policy.

Signed-off-by: Ben Pfaff 
Acked-by: Alin Gabriel Serdean

ovsdb-idl: Add function to reset min_index.

2020-05-28T16:36:48+00:00

If an administrator removes all of the databases in a cluster from
disk, then ovsdb IDL clients will have a problem. The databases will all
reset their stored indexes to 0, so The IDL client's min_index will be
higher than the indexes of all databases in the cluster. This results in
the client constantly connecting to databases, detecting the data as
"stale", and then attempting to connect to another.

This function provides a way to reset the IDL to an initial state with
min_index of 0. This way, the client will not wrongly detect the
database data as stale and will recover properly.

Notice that this function is not actually used anywhere in this patch.
This will be used by OVN, though, since OVN is the primary user of
clustered OVSDB.

Signed-off-by: Mark Michelson 
Acked-by: Han Zhou 
Signed-off-by: Ilya Maximets