Skip to content

Commit

Permalink
Avoid false positive in election tests (valkey-io#984)
Browse files Browse the repository at this point in the history
The node may not be able to initiate an election in time due to
problems with cluster communication. If an election is initiated,
make sure its offset is 0.

Closes valkey-io#967.

Signed-off-by: Binbin <binloveplay1314@qq.com>
Signed-off-by: Ping Xie <pingxie@google.com>
  • Loading branch information
enjoy-binbin authored and PingXie committed Sep 14, 2024
1 parent cecec4e commit bbe7c5b
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 8 deletions.
9 changes: 9 additions & 0 deletions tests/support/cluster_util.tcl
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,15 @@ proc cluster_setup {masters replicas node_count slot_allocator replica_allocator
# Setup master/replica relationships
$replica_allocator $masters $replicas

# A helper debug log that can print the server id in the server logs.
# This can help us locate the corresponding server in the log file.
for {set i 0} {$i < $masters} {incr i} {
R $i DEBUG LOG "========== I am primary $i =========="
}
for {set i $i} {$i < [expr $masters+$replicas]} {incr i} {
R $i DEBUG LOG "========== I am replica $i =========="
}

wait_for_cluster_propagation
wait_for_cluster_state "ok"

Expand Down
33 changes: 25 additions & 8 deletions tests/unit/cluster/replica-migration.tcl
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,15 @@ proc test_migrated_replica {type} {
fail "Failover does not happened"
}

# Make sure the offset of server 3 / 7 is 0.
verify_log_message -3 "*Start of election*offset 0*" 0
verify_log_message -7 "*Start of election*offset 0*" 0
# The node may not be able to initiate an election in time due to
# problems with cluster communication. If an election is initiated,
# we make sure the offset of server 3 / 7 is 0.
if {[count_log_message -3 "Start of election"] != 0} {
verify_log_message -3 "*Start of election*offset 0*" 0
}
if {[count_log_message -7 "Start of election"] != 0} {
verify_log_message -7 "*Start of election*offset 0*" 0
}

# Make sure the right replica gets the higher rank.
verify_log_message -4 "*Start of election*rank #0*" 0
Expand Down Expand Up @@ -170,9 +176,14 @@ proc test_nonempty_replica {type} {
fail "Failover does not happened"
}

# Make sure server 7 gets the lower rank and it's offset is 0.
verify_log_message -4 "*Start of election*rank #0*" 0
verify_log_message -7 "*Start of election*offset 0*" 0

# The node may not be able to initiate an election in time due to
# problems with cluster communication. If an election is initiated,
# we make sure server 7 gets the lower rank and it's offset is 0.
if {[count_log_message -7 "Start of election"] != 0} {
verify_log_message -7 "*Start of election*offset 0*" 0
}

# Wait for the cluster to be ok.
wait_for_condition 1000 50 {
Expand Down Expand Up @@ -283,9 +294,15 @@ proc test_sub_replica {type} {
fail "Failover does not happened"
}

# Make sure the offset of server 3 / 7 is 0.
verify_log_message -3 "*Start of election*offset 0*" 0
verify_log_message -7 "*Start of election*offset 0*" 0
# The node may not be able to initiate an election in time due to
# problems with cluster communication. If an election is initiated,
# we make sure the offset of server 3 / 7 is 0.
if {[count_log_message -3 "Start of election"] != 0} {
verify_log_message -3 "*Start of election*offset 0*" 0
}
if {[count_log_message -7 "Start of election"] != 0} {
verify_log_message -7 "*Start of election*offset 0*" 0
}

# Wait for the cluster to be ok.
wait_for_condition 1000 50 {
Expand Down

0 comments on commit bbe7c5b

Please sign in to comment.