Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve websockets reconnects #170

Merged
merged 43 commits into from
Sep 4, 2023

Conversation

Lohann
Copy link
Collaborator

@Lohann Lohann commented Aug 31, 2023

Description

This PR implements a more robust reconnect strategy for Websocket connections, also replaces the default ethere's WS provider by jsonrpsee, which is the RPC Client used by Subxt and Substrate, this also allows Astar connector to use the same websocket connection both for ethereum and substrate calls.

  • Create an adapter between jsonrpsee and ethers.
  • Supports eth subscriptions
  • Replace WS by jsonrpsee.
  • Implement a more robust reconnect strategy
  • Implement and test the strategy on Astar and Ethereum connectors
  • Unit tests

Adapters

Most of the code on this PR are adapters that allow us to use the same client for Ethereum and Substrate interchangeably.

Adapter JsonRpsee Trait Ethers Trait
EthClientAdapter ClientT JsonRpcClient
EthPubsubAdapter SubscriptionT PubsubClient
EthSubscription Subscription NotificationStream
EthError Error RpcError
EthRpcParams ToRpcParams Serialize

Reconnect Strategy:

  • When the client returns a RestartNeeded error, the strategy will try to reconnect
  • One single task/thread is allowed to reconnect at a given moment
  • While reconnecting, new requests will be pending until the reconnect finishes
  • When the reconnect fails, all pending requests fails with the same error message
  • When the reconnect succeed, all pending requests are processed

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Code review prechecks:

  • Code follows the style guidelines of this project
  • Code has been self-reviewed
  • Inline comments have been added for each method
  • I have made corresponding changes to the documentation
  • Code changes introduces no new problems or warnings
  • Test cases have been added
  • Dependent changes have been merged and published in downstream modules

@Lohann Lohann requested a review from dvc94ch August 31, 2023 12:09
@Lohann Lohann changed the title WIP: Feature/reuse ws connection for astar connector WIP: Improve websockets reconnects Aug 31, 2023
Copy link
Contributor

@dvc94ch dvc94ch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have the bandwidth to do this PR justice. Looks good, although not sure why we need such a complicated solution. Sorry you're not getting enough appreciation for your work. I'll go ahead and approve...

@Lohann
Copy link
Collaborator Author

Lohann commented Sep 1, 2023

Yeah this PR is big and just look complex because of the number of third-part traits and adapters I had to implement:

Adapter JsonRpsee Trait Ethers Trait
EthClientAdapter ClientT JsonRpcClient
EthPubsubAdapter SubscriptionT PubsubClient
EthSubscription Subscription NotificationStream
EthError Error RpcError
EthRpcParams ToRpcParams Serialize

Our RPC client had to implement all those traits, this ended up bloating the code, besides that the actual reconnect logic is in the reconnect.rs and reconnect_impl.rs file, all the rest of the code are just implementation of those traits.

@Lohann Lohann changed the title WIP: Improve websockets reconnects Improve websockets reconnects Sep 4, 2023
@Lohann Lohann marked this pull request as ready for review September 4, 2023 19:19
@Lohann Lohann merged commit 8dc29a5 into master Sep 4, 2023
@dvc94ch dvc94ch deleted the feature/reuse-ws-connection-for-astar-connector branch September 7, 2023 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants