Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: move net2 dependency out of tendermint crate #338

Merged
merged 7 commits into from
Jun 17, 2020

Conversation

xla
Copy link
Contributor

@xla xla commented Jun 17, 2020

This is isolates the code paths which depend on network and http functionality by moving the rpc module into its own crate (tendermint-rpc). The expected outcome is that third-parties which depend on the tendermint crate are not forced to pull in unnecessary dependencies. Furthermore it will isolate any potential crates used to drive an rpc server implementation going forward.

Controversially this also removes the dependency on tokio as it was mostly used for async test execution, which was achieved by making extensive use of futures::executor::block_on - we've come full circle. Eager to hear opinions and alternatives to that.
Update: now a dev-dependency only instead of the above.

Follow-up to #7
Preparation for #219
Closes #289


  • Referenced an issue explaining the need for the change
  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Updated CHANGES.md

@xla xla added enhancement New feature or request rpc labels Jun 17, 2020
@xla xla self-assigned this Jun 17, 2020
@xla xla mentioned this pull request Jun 17, 2020
rpc/src/endpoint/block.rs Outdated Show resolved Hide resolved
rpc/Cargo.toml Show resolved Hide resolved
use tendermint::abci::{Code, Log, Path};
use tendermint::block;
use tendermint::merkle::proof::Proof;
use tendermint::serializers;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just out of curiosity: are all these import changes due to a different formatter? Is there a re reason why all the use crate:: ...s are replaced in that fashion?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personal touch really, optimised for editability and tracebility of dependencies. I usually one liner per top-level module in a crate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok for me. Ideally, these should be consistent in all our crates though.

rpc/src/lib.rs Outdated Show resolved Hide resolved
pub(crate) use custom::parse_non_empty_block_id;
pub(crate) use custom::parse_non_empty_hash;
pub use custom::parse_non_empty_block_id;
pub use custom::parse_non_empty_hash;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to publish these beyond the crate? Probably, yes as some might be used in the rpc crate now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, the rpc crate depends on a lot of these serialisers. Not happy with it. Also found the excessive use of pub(crate) is unnecessary as the top-level lib determines visibility beyond crate boundaries. Eager to hear how we can address this issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are only used in the rpc lib, they could actually live there. Not sure yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are also used in tendermint::block.

Copy link
Member

@liamsi liamsi Jun 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe removing the net dependency outweighs this now public modules/methods (serializers).

I might have mentioned that somewhere else before but ideally, the serialzation code would simply be abstracted away in the sense that anyone using the tendermint crate can e.g. simply call block::deserialize (or deserialize_json and deserialize_binary) and the that could internally use a dedicated serialization type serialize::block (and impl Froms/Intos between both) which uses methods / helpers like the above. Anyone (like the rpc crate) could simply use the core types and wouldn't need to care about serialization specific stuff. Does that make sense? Probably worth another discussion and later another PR.

Copy link
Member

@liamsi liamsi Jun 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now: maybe the serializers module should include a comment that we don't guarantee any backwards compatibility on the methods in the module between any releases and consider them an internal implementation detail (use at you own risk alike).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to lay out an explicit semantic versioning scheme and declare which modules we're covering and which we're not. Good to start tracking like this though thanks!

liamsi
liamsi previously requested changes Jun 17, 2020
Copy link
Member

@liamsi liamsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome. Thanks @xla! Left some comments (looks like s/rpc/crate/g changed some comments to sound funny, e.g. "request json crate").

But more relevant is that some of the changes seem to have messed with the integration tests:
thread 'rpc::abci_info' panicked at 'there is no reactor running, must be called from the context of Tokio runtime',

e.g.: rpc::abci_info

@@ -0,0 +1,20 @@
[package]
name = "tendermint-rpc"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably deserves a short readme (can be done in a followup PR).

serde = { version = "1", features = [ "derive" ] }
serde_bytes = "0.11"
serde_json = "1"
tendermint = { version = "0.13.0", path = "../tendermint" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm always sceptical of circular dependencies (even if it's just a dev-dependency). Are we confident that this does cannot have weird side-effects?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not expect any, but truth be told this was the solution with the least amount of opinionated changes. Would like to get rid of this circle as well, but afraid this goes too far for this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't think of any problems either 🍀 🤞

@xla
Copy link
Contributor Author

xla commented Jun 17, 2020

But more relevant is that some of the changes seem to have messed with the integration tests:
thread 'rpc::abci_info' panicked at 'there is no reactor running, must be called from the context of Tokio runtime',

On second thought it might be worthwhile to keep tokio in dev-dependencies.

@xla xla dismissed liamsi’s stale review June 17, 2020 19:02

Major points addressed, curious to get another pass.

@xla xla requested a review from liamsi June 17, 2020 19:02
Copy link
Member

@liamsi liamsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -2,6 +2,10 @@
//!
//! Serializers and deserializers for a transparent developer experience.
//!
//! CAUTION: There are no guarantees for backwards compatibility, this module should be considered
//! an internal implementation detail which can vanish without further warning. Use at your own
//! risk.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@liamsi liamsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤸🏿‍♀️🙌🏽🤾🏼‍♂️

@xla xla mentioned this pull request Jun 17, 2020
3 tasks
@liamsi liamsi merged commit e2335c4 into master Jun 17, 2020
@liamsi liamsi deleted the xla/289-extract-networking branch June 17, 2020 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request rpc
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature guard the networking stuff
3 participants