Relay pending packets in the relayer loop

[ancazamfir]

Closes: #561

Description

• added the new relay_pending_packets() that queries the chains for the unreceived packets and acks, builds and sends transactions. This function uses the exact methods used by the corresponding CLIs. It adds a retry around these.
• changed relay_from_events() to perform relaying of pending packets after events subscription but before the first event is processed. The height for queries and client updates must be the height of the first event minus one.
• noticed that the signer for msg timeout is wrong in master after my last commit and fixed that. We need to add a timeout test to the CI (Add test for MsgTimeout in CI #707)
• also opened Add CI test for the relayer loop #708 for relayer loop CI

---

For contributor use:

• Updated the Unreleased section of CHANGELOG.md with the issue.
• If applicable: Unit tests written, added test to CI.
• Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
• Updated relevant documentation (docs/) and code comments.
• Re-reviewed Files changed in the Github PR explorer.

[codecov-io]

Codecov Report

Merging #709 (b2d3d7f) into master (b1b37f5) will increase coverage by 30.2%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #709      +/-   ##
=========================================
+ Coverage    13.6%   43.8%   +30.2%     
=========================================
  Files          69     150      +81     
  Lines        3752    9821    +6069     
  Branches     1374       0    -1374     
=========================================
+ Hits          513    4309    +3796     
- Misses       2618    5512    +2894     
+ Partials      621       0     -621     

Impacted Files	Coverage Δ
modules/src/address.rs	100.0% <ø> (ø)
...application/ics20_fungible_token_transfer/error.rs	0.0% <ø> (ø)
...ion/ics20_fungible_token_transfer/msgs/transfer.rs	0.0% <ø> (ø)
modules/src/events.rs	0.0% <ø> (ø)
modules/src/handler.rs	100.0% <ø> (ø)
modules/src/ics02_client/client_def.rs	48.3% <ø> (ø)
modules/src/ics02_client/client_type.rs	79.1% <ø> (+31.5%)	⬆️
modules/src/ics02_client/context.rs	100.0% <ø> (ø)
modules/src/ics02_client/error.rs	100.0% <ø> (ø)
modules/src/ics02_client/events.rs	15.7% <ø> (+15.7%)	⬆️
... and 255 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7011aa8...ff602fc. Read the comment docs.

[romac]

Wow, that's much less code than I thought it would take :)

Left a suggestion inline, but feel free to ignore.

[romac]

More general question, which does not be addressed in this PR: would it be sensible to introduce a delay or even implement some kind of exponential backoff strategy when retrying an operation which has failed (in this case, relaying the pending packets)?

[ancazamfir]

Yeh, we've been avoiding the whole error handling for some time. We have this pattern everywhere! We blindly retry in a loop for all errors. We need to sit down and see how to deal with each error. We have RPC, light client, on-chain and internal errors. The most complex ones are the on-chain errors that are a bit more subtle (leaving aside errors in the relayer code) and also they are not typed (we just get an error message and the reason is embedded in a string). The retry with exponential backoff may apply for some cases, in some cases we should not retry (e.g. a transaction has already been submitted by another relayer), in other cases we should rebuild stuff, etc. I think we should start a little doc and try to cover all errors and what actions we should take.
Because the problem is so broad I would leave this handling here for later.

[ancazamfir]

Opened #712 for error handling.