delta/openvswitch.git/tests/test-ovsdb.py, branch master

ovsdb-cs: Consider default conditions implicitly acked.

2022-12-13T17:52:10+00:00

When initializing a monitor table the default monitor condition is
[True] which matches the behavior of the server (to send all rows of
that table).  There's no need to include this default condition in the
initial monitor request so we can consider it implicitly acked by the
server.

This fixes the incorrect (one too large) expected condition sequence
number reported by ovsdb_idl_set_condition() when application is
trying to set a [True] condition for a new table.

Reported-by: Numan Siddique 
Suggested-by: Ilya Maximets 
Signed-off-by: Dumitru Ceara 
Signed-off-by: Ilya Maximets

ovsdb-idl: Add the support to specify the uuid for row insert.

2022-11-30T14:15:57+00:00

ovsdb-server allows the OVSDB clients to specify the uuid for
the row inserts [1].  Both the C IDL client library and Python
IDL are missing this feature.  This patch adds this support.

In C IDL, for each schema table, a new function is generated -
insert_persistent_uuid(txn, uuid) which can
be used the clients to persist the uuid.

ovs-vsctl and other derivatives of ctl now supports the same
in the generic 'create' command with the option "--id=".

In Python IDL, the uuid to persist can be specified in
the Transaction.insert() function.

[1] - a529e3cd1f("ovsdb-server: Allow OVSDB clients to specify the UUID for inserted rows.:)

Acked-by: Adrian Moreno 
Acked-by: Han Zhou 
Acked-by: Terry Wilson 
Signed-off-by: Numan Siddique 
Signed-off-by: Ilya Maximets

python: Fix E275 missing whitespace after keyword.

2022-08-04T16:13:20+00:00

With just released flake8 5.0 we're getting a bunch of E275 errors:

 utilities/bugtool/ovs-bugtool.in:959:23: E275 missing whitespace after keyword
 tests/test-ovsdb.py:623:11: E275 missing whitespace after keyword
 python/setup.py:105:8: E275 missing whitespace after keyword
 python/setup.py:106:8: E275 missing whitespace after keyword
 python/ovs/db/idl.py:145:15: E275 missing whitespace after keyword
 python/ovs/db/idl.py:167:15: E275 missing whitespace after keyword
 make[2]: *** [flake8-check] Error 1

This breaks CI on branches below 2.16.  We don't see a problem right
now on newer branches because we're installing extra dependencies
that backtrack flake8 down to 4.1 or even 3.9.

Acked-by: Mike Pattrick 
Acked-by: Dumitru Ceara 
Signed-off-by: Ilya Maximets

ovsdb-cs: Clear last_id on reconnect if condition changes in-flight.

2022-01-31T20:23:47+00:00

When reconnecting, if there are condition changes already sent to the
server but not yet acked, reset the db's 'last-id', esentially clearing
the local cache after reconnect.

This is needed because the client cannot easily differentiate between
the following cases:
a. either the server already processed the requested monitor
   condition change but the FSM was restarted before the
   client was notified.  In this case the client should
   clear its local cache because it's out of sync with the
   monitor view on the server side.
b. OR the server hasn't processed the requested monitor
   condition change yet.

Conditions changing at the same time with a reconnection happening are
rare so the performance impact of this patch should be minimal.

Also, the tests are updated to cover the fact that we cannot control
which of the two scenarios ("a" and "b" above) are hit during the test.

Reported-by: Maxime Coquelin 
Signed-off-by: Dumitru Ceara 
Acked-by: Han Zhou 
Signed-off-by: Ilya Maximets

python: idl: Allow retry even when using a single remote.

2021-06-10T23:11:57+00:00

As described in commit [1], it's possible that remote IP is backed by
a load-balancer and re-connection to this same IP will lead to
connection to a different server.  This case is supported for C version
of IDL and should be supported in a same way for python implementation.

[1] ca367fa5f8bb ("ovsdb-idl.c: Allows retry even when using a single remote.")

Signed-off-by: Ilya Maximets 
Acked-by: Dumitru Ceara

ovsdb-idl: Preserve references for deleted rows.

2021-04-01T12:15:49+00:00

Considering two DB rows, 'a' from table A and 'b' from table B (with
column 'ref_a' a reference to table A):
a = {A._uuid=}
b = {B._uuid=, B.ref_a=}

Assuming both records are present in the IDL client's in-memory view of
the database, depending whether row 'b' is also deleted in the same
transaction or not, deletion of row 'a' should generate the following
tracked changes:

1. only row 'a' is deleted:
- for table A:
  - deleted records: a = {A._uuid=}
- for table B:
  - updated records: b = {B._uuid=, B.ref_a=[]}

2. row 'a' and row 'b' are deleted in the same update:
- for table A:
  - deleted records: a = {A._uuid=}
- for table B:
  - deleted records: b = {B._uuid=, B.ref_a=}

To ensure this, we now delay reparsing row backrefs for deleted rows
until all updates in the current run have been processed.

Without this change, in scenario 2 above, the tracked changes for table
B would be:
- deleted records: b = {B._uuid=, B.ref_a=[]}

In particular, for strong references, row 'a' can never be deleted in
a transaction that happens strictly before row 'b' is deleted.  In some
cases [0] both rows are deleted in the same transaction and having
B.ref_a=[] would violate the integrity of the database from client
perspective.  This would force the client to always validate that
strong reference fields are non-NULL.  This is not really an option
because the information in the original reference is required for
incrementally processing the record deletion.

[0] with ovn-monitor-all=true, the following command triggers a crash
    in ovn-controller because a strong reference field becomes NULL:
    $ ovn-nbctl --wait=hv -- lr-add r -- lrp-add r rp 00:00:00:00:00:01 1.0.0.1/24
    $ ovn-nbctl lr-del r

Reported-at: https://bugzilla.redhat.com/1932642
Fixes: 72aeb243a52a ("ovsdb-idl: Tracking - preserve data for deleted rows.")
Signed-off-by: Dumitru Ceara 
Acked-by: Han Zhou 
Signed-off-by: Ilya Maximets

ovsdb-idl.at: Make test outputs more predictable.

2021-04-01T11:53:20+00:00

IDL tests need predictable output from test-ovsdb.

This used to be done by first sorting the output of test-ovsdb and then
applying uuidfilt to predictably translate UUIDs.  This was not
reliable enough in case test-ovsdb processes two or more insert/delete
operations in the same iteration because the order of lines in the
output depends on the automatically generated UUID values.

To fix this we change the way test-ovsdb and test-ovsdb.py generate
outputs and prepend the table name and tracking information before
printing the contents of a row.

All existing ovsdb-idl.at and ovsdb-cluster.at tests are updated to
expect the new output format.

Signed-off-by: Dumitru Ceara 
Acked-by: Han Zhou 
Signed-off-by: Ilya Maximets

python: Send notifications after the transaction ends.

2021-03-15T15:37:20+00:00

The Python IDL notification mechanism was sending a notification
for each processed update in a transaction as it was processed.
This causes issues with multi-row changes that contain references
to each other.

For example, if a Logical_Router_Port is created along with a
Gateway_Chassis, and the LRP.gateway_chassis set to that GC, then
when the notify() passes the CREATE event for the LRP, the GC will
not yet have been processed, so __getattr__ when _uuid_to_row fails
to find the GC, will return the default value for LRP.gateway_chassis
which is [].

This patch has the process_update methods return the notifications
that would be produced when a row changes, so they can be queued
and sent after all rows have been processed.

Fixes: d7d417fcddf9 ("Allow subclasses of Idl to define a notification hook")
Signed-off-by: Terry Wilson 
Acked-by: Brian Haley 
Acked-by: Dumitru Ceara 
Tested-by: Flavio Fernandes 
Signed-off-by: Ilya Maximets

Handle refTable values with setkey()

2020-03-20T15:47:53+00:00

For columns like QoS.queues where we have a map containing refTable
values, assigning w/ __setattr__ e.g. qos.queues={1: $queue_row}
works, but using using qos.setkey('queues', 1, $queue_row) results
in an Exception. The opdat argument can essentially just be the
JSON representation of the map column instead of trying to build
it.

Signed-off-by: Terry Wilson 
Signed-off-by: Ben Pfaff

Remove dependency on python3-six

2019-12-20T20:23:06+00:00

Since Python 2 support was removed in 1ca0323e7c29 ("Require Python 3 and
remove support for Python 2."), python3-six is not needed anymore.

Moreover python3-six is not available on RHEL/CentOS7 without using EPEL
and so this patch is needed in order to release OVS 2.13 on RHEL7.

Signed-off-by: Timothy Redaelli 
Signed-off-by: Ben Pfaff