Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mitigate missing inbound observations: stop scanning external chains if zetacored node is halted #3115

Open
Tracked by #1611
brewmaster012 opened this issue Nov 6, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@brewmaster012
Copy link
Collaborator

brewmaster012 commented Nov 6, 2024

Describe the Bug
During the chain halt event on Oct 30, 2024 which lasts around 6h, there were dozens of
inbound transactions that were not observed or reported to zetacored, likely because the
network is down but zetaclient keep moving and the observation transaction accumulated
in mempool and got dropped eventually.

This results in missed inbound observations that required manual intervention via the InboundTracker
mechanism.

To Reproduce
(I have not tried this)
Supposedly we may be able to reproduce this in the performance test where we induce
a zetachain halt and keep inbound transaction going.

Expected Behavior
A better behavior is for zetaclient to receive feedback from zetacored that the zetacored
block height is not progressing (either the network is down, or the particular zetacored node
loses connectivity, database corrupt, consensus failure etc), then the zetaclient should
stop scanning external chains.

Zetaclient persists progress of external chain observation and is expected to pick up
from where it stopped, when the zetacored issue is fixed.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS
  • Version used
@brewmaster012 brewmaster012 added the bug Something isn't working label Nov 6, 2024
@brewmaster012 brewmaster012 changed the title mitigate missing inbound observations: stop scanning external chains if zetachain is halted mitigate missing inbound observations: stop scanning external chains if zetacored node is halted Nov 6, 2024
@ws4charlie
Copy link
Contributor

It seems we need

  1. Determine if the the zetacore node is moving (height increase) or not.
  2. If zetacore stops, zetaclient disable inbound?

@brewmaster012
Copy link
Collaborator Author

Yeah if the zetacored block is not moving, observation should be stopped because any reporting tx is unlikely to succeed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants