Add RPC support for secure connections and HTTP proxies (#820)

* Refactor to add HTTPS client

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Switch to Rustls

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add TLS support for WebSocket connections

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add support for secure WebSocket connections

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update docs and exports

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update docs to use new link format

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove unused file

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add required attributes to WebSocket client markers

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor out constructor common bits

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add tendermint-rpc binary

This is unfortunately a big commit. It primarily adds an optional binary
application (behind the "cli" feature flag) that allows you to easily
perform queries against a Tendermint endpoint via several protocols
(http, https, ws, wss).

It only contains support at present for Client queries, and
SubscriptionClient queries (i.e. subscriptions) are planned for
subsequent commits.

Some of the bulk of this commit is due to the fact that we need to be
able to parse queries (e.g. from tx_search), so I added a PEG-based
parser for queries along with some tests.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add support for subscription to RPC CLI

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add support for parsing floating point numbers in RPC queries

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add termination option for subscriptions in CLI app

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update docs to describe CLI tool

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Refactor RPC to allow for generic URLs

This commit refactors the RPC interface, while trying to reduce the
amount of downstream impact as far as possible, with the aim of allowing
the user to specify a generic URL as input to the HTTP and WebSocket
clients.

This allows the client to infer whether or not to use a secure
connection based on the supplied URL.

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix version requirement of tendermint-proto

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update client usage to show simplified parameters

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Use tracing instead of log to show comprehensive debug information during testing

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update docs

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add query parsing examples to crate docs

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Rename ambiguous module url to rpc_url and ensure it is only exported when a client is enabled

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Improve environment variable handling for proxies

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add examples for using proxy to CLI usage docs

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Fix some README typos

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Export HTTP and WebSocket URL types to surface documentation

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add CHANGELOG entries

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Clarify directory for tendermint-rpc CLI

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Add HTTP/2 support for HTTP client

Signed-off-by: Thane Thomson <connect@thanethomson.com>

CHANGELOG.md

@@ -1,5 +1,17 @@
 ## Unreleased
 
+### BREAKING CHANGES
+
+* `[tendermint-rpc]` The `SubscriptionClient` trait now requires a `close`
+  method, since it assumes that subscription clients will, in general, use
+  long-running connections. This should not, however, break any downstream
+  usage of the clients ([#820])
+* `[tendermint-rpc]` The `HttpClient` and `WebSocketClient` constructors now
+  take any input that can be converted to a `tendermint_rpc::Url`. This should
+  hopefully have minimal impact on projects using the code, but it might
+  require some minor code changes in some cases - see the crate docs for more
+  details ([#820])
+
 ### FEATURES
 
 * `[tendermint-abci]` Release minimal framework for building ABCI applications
@@ -10,9 +22,15 @@
   method at present, exclusively provides access to block verification. This
   does not include network access or the Light Client's bisection algorithm
   ([#812])
+* `[tendermint-rpc]` Support for secure connections (`https://` and `wss://`)
+  has been added to the Tendermint RPC clients, as well as support for HTTP
+  proxies for HTTP clients ([#820])
+* `[tendermint-rpc]` A `tendermint-rpc` CLI has been added to simplify
+  interaction with RPC endpoints from the command line ([#820])
 
 [#794]: https://github.com/informalsystems/tendermint-rs/pull/794
 [#812]: https://github.com/informalsystems/tendermint-rs/pull/812
+[#820]: https://github.com/informalsystems/tendermint-rs/pull/820
 
 ## v0.18.1
 

abci/Cargo.toml

@@ -25,7 +25,7 @@ binary = [ "structopt", "tracing-subscriber" ]
 bytes = "1.0"
 eyre = "0.6"
 prost = "0.7"
-tendermint-proto = { version = "0.18.0", path = "../proto" }
+tendermint-proto = { version = "0.18.1", path = "../proto" }
 thiserror = "1.0"
 tracing = "0.1"
 

rpc/Cargo.toml

@@ -23,16 +23,30 @@ description = """
 [package.metadata.docs.rs]
 all-features = true
 
+[[bin]]
+name = "tendermint-rpc"
+path = "src/client/bin/main.rs"
+required-features = [ "cli" ]
+
 [features]
 default = []
+cli = [
+  "http-client",
+  "structopt",
+  "tracing-subscriber",
+  "websocket-client"
+]
 http-client = [
   "async-trait",
   "futures",
   "http",
   "hyper",
+  "hyper-proxy",
+  "hyper-rustls",
   "tokio/fs",
   "tokio/macros",
-  "tracing"
+  "tracing",
+  "url"
 ]
 secp256k1 = [ "tendermint/secp256k1" ]
 websocket-client = [
@@ -44,13 +58,18 @@ websocket-client = [
   "tokio/macros",
   "tokio/sync",
   "tokio/time",
-  "tracing"
+  "tracing",
+  "url"
 ]
 
 [dependencies]
 bytes = "1.0"
 chrono = "0.4"
 getrandom = "0.1"
+# TODO(thane): Use a released version once support for inverted patterns is released.
+#              See https://github.com/kevinmehall/rust-peg/pull/245
+peg = { git = "https://github.com/kevinmehall/rust-peg.git", rev = "ba6019539b2cf80289190cbb9537c94113b6b7d1" }
+pin-project = "1.0.1"
 serde = { version = "1", features = [ "derive" ] }
 serde_bytes = "0.11"
 serde_json = "1"
@@ -62,10 +81,14 @@ subtle-encoding = { version = "0.5", features = ["bech32-preview"] }
 walkdir = "2.3"
 
 async-trait = { version = "0.1", optional = true }
-async-tungstenite = { version = "0.12", features = ["tokio-runtime"], optional = true }
+async-tungstenite = { version = "0.12", features = ["tokio-runtime", "tokio-rustls"], optional = true }
 futures = { version = "0.3", optional = true }
 http = { version = "0.2", optional = true }
-hyper = { version = "0.14", optional = true, features = ["client", "http1", "tcp"] }
+hyper = { version = "0.14", optional = true, features = ["client", "http1", "http2", "tcp"] }
+hyper-proxy = { version = "0.9", optional = true }
+hyper-rustls = { version = "0.22.1", optional = true }
+structopt = { version = "0.3", optional = true }
 tokio = { version = "1.0", optional = true }
 tracing = { version = "0.1", optional = true }
-pin-project = "1.0.1"
+tracing-subscriber = { version = "0.2", optional = true }
+url = { version = "2.2", optional = true }

rpc/README.md

@@ -21,16 +21,72 @@ This crate optionally provides access to different types of RPC client
 functionality and different client transports based on which features you
 select when using it.
 
-Two features are provided at present.
+Several client-related features are provided at present:
 
 * `http-client` - Provides `HttpClient`, which is a basic RPC client that
-  interacts with remote Tendermint nodes via **JSON-RPC over HTTP**. This
-  client does not provide `Event` subscription functionality. See the
-  [Tendermint RPC] for more details.
+  interacts with remote Tendermint nodes via **JSON-RPC over HTTP or
+  HTTPS**. This client does not provide `Event` subscription
+  functionality. See the [Tendermint RPC] for more details.
 * `websocket-client` - Provides `WebSocketClient`, which provides full
-  client functionality, including general RPC functionality (such as that
-  provided by `HttpClient`) as well as `Event` subscription
-  functionality.
+  client functionality, including general RPC functionality as well as
+  `Event`] subscription functionality. Can be used over secure
+  (`wss://`) and unsecure (`ws://`) connections.
+
+### CLI
+
+A `tendermint-rpc` console application is provided for testing/experimentation
+purposes. To build this application:
+
+```bash
+# From the tendermint-rpc crate's directory
+cd rpc
+cargo build --bin tendermint-rpc --features cli
+
+# To run directly and show usage information
+cargo run --bin tendermint-rpc --features cli -- --help
+
+# To install the binary to your Cargo binaries path
+# (should be globally accessible)
+cargo install --bin tendermint-rpc --features cli --path .
+```
+
+The application sends its logs to **stderr** and its output to **stdout**, so
+it's relatively easy to capture RPC output.
+
+**Usage examples:** (assuming you've installed the binary)
+
+```bash
+# Check which RPC commands/endpoints are supported.
+tendermint-rpc --help
+
+# Query the status of the Tendermint node bound to tcp://127.0.0.1:26657
+tendermint-rpc status
+
+# Submit a transaction to the key/value store ABCI app via a Tendermint node
+# bound to tcp://127.0.0.1:26657
+tendermint-rpc broadcast-tx-async somekey=somevalue
+
+# Query the value associated with key "somekey" (still assuming a key/value
+# store ABCI app)
+tendermint-rpc abci-query somekey
+
+# To use an HTTP/S proxy to access your RPC endpoint
+tendermint-rpc --proxy-url http://yourproxy:8080 abci-query somekey
+
+# To set your HTTP/S proxy for multiple subsequent queries
+export HTTP_PROXY=http://yourproxy:8080
+tendermint-rpc abci-query somekey
+
+# Subscribe to receive new blocks (must use the WebSocket endpoint)
+# Prints out all incoming events
+tendermint-rpc -u ws://127.0.0.1:26657/websocket subscribe "tm.event='NewBlock'"
+
+# If you want to execute a number of queries against a specific endpoint and
+# don't feel like re-typing the URL over and over again, just set the
+# TENDERMINT_RPC_URL environment variable
+export TENDERMINT_RPC_URL=ws://127.0.0.1:26657/websocket
+tendermint-rpc subscribe "tm.event='Tx'"
+```
 
 ### Mock Clients
 

rpc/src/client.rs

@@ -8,9 +8,9 @@ mod transport;
 pub use transport::mock::{MockClient, MockRequestMatcher, MockRequestMethodMatcher};
 
 #[cfg(feature = "http-client")]
-pub use transport::http::HttpClient;
+pub use transport::http::{HttpClient, HttpClientUrl};
 #[cfg(feature = "websocket-client")]
-pub use transport::websocket::{WebSocketClient, WebSocketClientDriver};
+pub use transport::websocket::{WebSocketClient, WebSocketClientDriver, WebSocketClientUrl};
 
 use crate::endpoint::*;
 use crate::query::Query;
@@ -104,8 +104,8 @@ pub trait Client {
         self.perform(broadcast::tx_sync::Request::new(tx)).await
     }
 
-    /// `/broadcast_tx_sync`: broadcast a transaction, returning the response
-    /// from `CheckTx`.
+    /// `/broadcast_tx_commit`: broadcast a transaction, returning the response
+    /// from `DeliverTx`.
     async fn broadcast_tx_commit(&self, tx: Transaction) -> Result<broadcast::tx_commit::Response> {
         self.perform(broadcast::tx_commit::Request::new(tx)).await
     }