diff options
author | Oran Agra <oran@redislabs.com> | 2022-01-30 11:30:19 +0200 |
---|---|---|
committer | GitHub <noreply@github.com> | 2022-01-30 11:30:19 +0200 |
commit | be0d2933545354f4868f8e4807a11f8e79c03736 (patch) | |
tree | 8cb6d6a774e1f15d47263d64b2cbc2805c5bd83c /tests/instances.tcl | |
parent | 21135471a6bb25c4c895c26ae1afddad53e83f29 (diff) | |
download | redis-be0d2933545354f4868f8e4807a11f8e79c03736.tar.gz |
fix cluster rebalance test race (#10207)
Try to fix the rebalance cluster test that's failing with ASAN daily:
Looks like `redis-cli --cluster rebalance` gets `ERR Please use SETSLOT only with masters` in `clusterManagerMoveSlot()`.
it happens when `12-replica-migration-2.tcl` is run with ASAN in GH Actions.
in `Resharding all the master #0 slots away from it`
So the fix (assuming i got it right) is to call `redis-cli --cluster check` before `--cluster rebalance`.
p.s. it looks like a few other checks in these tests needed that wait, added them too.
Other changes:
* in instances.tcl, make sure to catch tcl test crashes and let the rest of the code proceed, so that if there was
a redis crash, we'll find it and print it too.
* redis-cli, try to make sure it prints an error instead of silently exiting.
specifically about redis-cli:
1. clusterManagerMoveSlot used to print an error, only if the caller also asked for it (should be the other way around).
2. clusterManagerCommandReshard asked for an error, but didn't use it (probably tried to avoid the double print).
3. clusterManagerCommandRebalance didn't ask for the error, now it does.
4. making sure that other places in clusterManagerCommandRebalance print something before exiting with an error.
Diffstat (limited to 'tests/instances.tcl')
-rw-r--r-- | tests/instances.tcl | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/tests/instances.tcl b/tests/instances.tcl index 1d2cbed49..cf0e80f6c 100644 --- a/tests/instances.tcl +++ b/tests/instances.tcl @@ -446,7 +446,12 @@ proc run_tests {} { } if {[file isdirectory $test]} continue puts [colorstr yellow "Testing unit: [lindex [file split $test] end]"] - source $test + if {[catch { source $test } err]} { + puts "FAILED: caught an error in the test $err" + puts $::errorInfo + incr ::failed + # letting the tests resume, so we'll eventually reach the cleanup and report crashes + } check_leaks {redis sentinel} # Check if a leaked fds file was created and abort the test. |