Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add grpc & rest endpoints overview #8064

Merged
merged 13 commits into from
Dec 14, 2020
6 changes: 4 additions & 2 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,17 @@ aside: false
## Get Started

- **[SDK Intro](./intro/overview.md)**: High-level overview of the Cosmos SDK.
- **[Quick Start Guide](./using-the-sdk/quick-start.md)**: Scaffold a standard Cosmos SDK app and run a node.
- **[Quick Start Guide](./using-the-sdk/quick-start.md)**: Scaffold a standard Cosmos SDK app and run a node.
- **[SDK Application Tutorial](https://github.com/cosmos/sdk-application-tutorial)**: A tutorial that showcases how to build an SDK-based blockchain from scratch and explains the basic principles of the SDK in the process.

## Reference

- **[Basics](./basics/)**: Documentation on the basic concepts of the Cosmos SDK, like the standard anatomy of an application, the transaction lifecycle and accounts management.
- **[Core](./core/)**: Documentation on the core concepts of the Cosmos SDK, like `baseapp`, the `store` or the `server`.
- **[Building Modules](./building-modules/)**: Important concepts for module developers like `message`s, `keeper`s, `handler`s and `querier`s.
- **[Interfaces](./interfaces/)**: Documentation on building interfaces for Cosmos SDK applications.
- **[IBC](./ibc/)**: Documentation for the IBC protocol integration and concepts.
- **[Running a Node, API, CLI](./run-node/)**: Documentation on how to run a node, and how to interact with it using the CLI and the API.
- **[Migrations](./migrations/)**: Migration guides for updating to Stargate.

## Other Resources

Expand Down
9 changes: 5 additions & 4 deletions docs/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ This repository contains reference documentation on the core concepts of the Cos
4. [Node Client](./node.md)
5. [Store](./store.md)
6. [Encoding](./encoding.md)
7. [Events](./events.md)
8. [Telemetry](./telemetry.md)
9. [Object-Capabilities](./ocap.md)
10. [RunTx recovery middleware](./runtx_middleware.md)
7. [gRPC, REST and Tendermint Endpoints](./grpc_rest.md)
8. [Events](./events.md)
9. [Telemetry](./telemetry.md)
10. [Object-Capabilities](./ocap.md)
11. [RunTx recovery middleware](./runtx_middleware.md)

After reading about the core concepts, check the [IBC documentation](../ibc/README.md) to learn more
about the IBC core concepts and how to integrate it to you application.
28 changes: 15 additions & 13 deletions docs/core/encoding.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,11 @@ Modules are encouraged to utilize Protobuf encoding for their respective types.
**Defining module types**

Protobuf types can be defined to encode:
- state
- [`Msg`s](../building-modules/messages-and-queries.md#messages)
- [Query services](../building-modules/query-services.md)
- [genesis](../building-modules/genesis.md)

- state
- [`Msg`s](../building-modules/messages-and-queries.md#messages)
- [Query services](../building-modules/query-services.md)
- [genesis](../building-modules/genesis.md)

**Naming and conventions**

Expand Down Expand Up @@ -112,7 +113,7 @@ The SDK `codec.Marshaler` interface provides support methods `MarshalInterface`

Module should register interfaces using `InterfaceRegistry` which provides a mechanism for registering interfaces: `RegisterInterface(protoName string, iface interface{})` and implementations: `RegisterImplementations(iface interface{}, impls ...proto.Message)` that can be safely unpacked from Any, similarly to type registration with Amino:

+++ https://github.com/cosmos/cosmos-sdk/blob/3d969a1ffdf9a80f9ee16db9c16b8a8aa1004af6/codec/types/interface_registry.go#L23-L52
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/codec/types/interface_registry.go#L25-L66

In addition, an `UnpackInterfaces` phase should be introduced to deserialization to unpack interfaces before they're needed. Protobuf types that contain a protobuf `Any` either directly or via one of their members should implement the `UnpackInterfacesMessage` interface:

Expand All @@ -125,10 +126,11 @@ type UnpackInterfacesMessage interface {
#### Guidelines for protobuf message definitions

In addition to [following official guidelines](https://developers.google.com/protocol-buffers/docs/proto3#simple), we recommend to use these annotations in .proto files when dealing with interfaces:
* fields which accept interfaces should be annotated with `cosmos_proto.accepts_interface`
using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface`
* interface implementations should be annotated with `cosmos_proto.implements_interface`
using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface`

- fields which accept interfaces should be annotated with `cosmos_proto.accepts_interface`
using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface`
- interface implementations should be annotated with `cosmos_proto.implements_interface`
using the same full-qualified name passed as `protoName` to `InterfaceRegistry.RegisterInterface`

#### Transaction Encoding

Expand All @@ -139,14 +141,14 @@ other peers. Since the underlying consensus engine is agnostic to the applicatio
it only accepts transactions in the form of raw bytes. The encoding is done by an
object called `TxEncoder` and the decoding by an object called `TxDecoder`.

+++ https://github.com/cosmos/cosmos-sdk/blob/9ae17669d6715a84c20d52e10e2232be9f467360/types/tx_msg.go#L82-L86
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/types/tx_msg.go#L83-L87

A standard implementation of both these objects can be found in the [`auth` module](https://github.com/cosmos/cosmos-sdk/blob/master/x/auth):

+++ https://github.com/cosmos/cosmos-sdk/blob/9ae17669d6715a84c20d52e10e2232be9f467360/x/auth/tx/decoder.go
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/x/auth/tx/decoder.go

+++ https://github.com/cosmos/cosmos-sdk/blob/9ae17669d6715a84c20d52e10e2232be9f467360/x/auth/tx/encoder.go
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/x/auth/tx/encoder.go

## Next {hide}

Learn about [events](./events.md) {hide}
Learn about [gRPC, REST and other endpoints](./grpc_rest.md) {hide}
2 changes: 1 addition & 1 deletion docs/core/events.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
order: 7
order: 8
-->

# Events
Expand Down
101 changes: 101 additions & 0 deletions docs/core/grpc_rest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
<!--
order: 7
-->

# gRPC, REST, and Tendermint Endpoints
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great additions on this page. 👍


This document presents an overview of all the endpoints a node exposes: gRPC, REST as well as some other endpoints. {synopsis}

## An Overview of All Endpoints

Each node exposes the following endpoints for users to interact with a node, each endpoint is served on a different port. Details on how to configure each endpoint is provided in the endpoint's own section.

- the gRPC server (default port: `9090`),
- the REST server (default port: `1317`),
- the Tendermint RPC endpoint (default port: `26657`).

::: tip
The node also exposes some other endpoints, such as the Tendermint P2P endpoint, or the [Prometheus endpoint](https://docs.tendermint.com/master/nodes/metrics.html#metrics), which are not directly related to the Cosmos SDK. Please refer to the [Tendermint documentation](https://docs.tendermint.com/master/tendermint-core/using-tendermint.html#configuration) for more information about these endpoints.
:::

## gRPC Server

Cosmos SDK v0.40 introduced Protobuf as the main [encoding](./encoding) library, and this brings a wide range of Protobuf-based tools that can be plugged into the SDK. One such tool is [gRPC](https://grpc.io), a modern open source high performance RPC framework that has decent client support in several languages.

Each module exposes [`Msg` and `Query` Protobuf services](../building-modules/messages-and-queries.md) to define state transitions and state queries. These services are hooked up to gRPC via the following function inside the application:

https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/server/types/app.go#L39-L41

The `grpc.Server` is a concrete gRPC server, which spawns and serves any gRPC requests. This server can be configured inside `$TMHOME/config/app.toml`:

- `grpc.enable = true|false` field defines if the gRPC server should be enabled. Defaults to `true`.
- `grpc.address = {string}` field defines the address (really, the port, since the host should be kept at `0.0.0.0`) the server should bind to. Defaults to `0.0.0.0:9000`.

::tip
`$TMHOME` is the directory where the node's configuration and databases are stored. By default, it's set to `~/.{app_name}`.
::

Once the gRPC server is started, you can send requests to it using a gRPC client. Some examples are given in our [Interact with the Node](../run-node/interact-node.md#using-grpc) tutorial.

An overview of all available gRPC endpoints shipped with the Cosmos SDK is [coming soon](https://github.com/cosmos/cosmos-sdk/issues/7786).
amaury1093 marked this conversation as resolved.
Show resolved Hide resolved

## REST Server

In Cosmos SDK v0.40, the node continues to serve a REST server. However, the existing routes present in version v0.39 and earlier are now marked as deprecated, and new routes have been added via gRPC-gateway.

All routes are configured under the following fields in `$TMHOME/config/app.toml`:

- `api.enable = true|false` field defines if the REST server should be enabled. Defaults to `true`.
- `api.address = {string}` field defines the address (really, the port, since the host should be kept at `0.0.0.0`) the server should bind to. Defaults to `tcp://0.0.0.0:1317`.
- some additional API configuration options are defined in `$TMHOME/config/app.toml`, along with comments, please refer to that file directly.

### gRPC-gateway REST Routes

If, for various reasons, you cannot use gRPC (for example, you are building a web application, and browsers don't support HTTP2 on which gRPC is built), then the SDK offers REST routes via gRPC-gateway.

[gRPC-gateway](https://grpc-ecosystem.github.io/grpc-gateway/) is a tool to expose gRPC endpoints as REST endpoints. For each RPC endpoint defined in a Protobuf service, the SDK offers a REST equivalent. For instance, querying a balance could be done via the `/cosmos.bank.v1beta1.Query/AllBalances` gRPC endpoint, or alternatively via the gRPC-gateway `"/cosmos/bank/v1beta1/balances/{address}"` REST endpoint: both will return the same result. For each RPC method defined in a Protobuf service, the corresponding REST endpoint is defined as an option:

+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/proto/cosmos/bank/v1beta1/query.proto#L19-L22

For application developers, gRPC-gateway REST routes needs to be wired up to the REST server, this is done by calling the `RegisterGRPCGatewayRoutes` function on the ModuleManager.

### Legacy REST API Routes

The REST routes present in Cosmos SDK v0.39 and earlier are marked as deprecated via a [HTTP deprecation header](https://tools.ietf.org/id/draft-dalal-deprecation-header-01.html). They are still maintained to keep backwards compatibility, but will be removed in v0.41. For updating from Legacy REST routes to new gRPC-gateway REST routes, please refer to our [migration guide](../migrations/rest.md).

For application developers, Legacy REST API routes needs to be wired up to the REST server, this is done by calling the `RegisterRESTRoutes` function on the ModuleManager.

### Swagger

A [Swagger](https://swagger.io/) (or OpenAPIv2) specification file is exposed under the `/swagger` route on the API server. Swagger is an open specification describing the API endpoints a server serves, including description, input arguments, return types and much more about each endpoint.

Enabling the `/swagger` endpoint is configurable inside `$TMHOME/config/app.toml` via the `api.swagger` field, which is set to true by default.

For application developers, you may want to generate your own Swagger definitions based on your custom modules. The SDK's [Swagger generation script](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/scripts/protoc-swagger-gen.sh) is a good place to start.

## Tendermint RPC

Independently from the Cosmos SDK, Tendermint also exposes a RPC server. This RPC server can be configured by tuning parameters under the `rpc` table in the `$TMHOME/config/config.toml`, the default listening address is `tcp://0.0.0.0:26657`. An OpenAPI specification of all Tendermint RPC endpoints is available [here](https://docs.tendermint.com/master/rpc/).

Some Tendermint RPC endpoints are directly related to the Cosmos SDK:

- `/abci_query`: this endpoint will query the application for state. As the `path` parameter, you can send the following strings:
- any Protobuf fully-qualified service method, such as `/cosmos.bank.v1beta1.Query/AllBalances`. The `data` field should then include the method's request parameter(s) encoded as bytes using Protobuf.
- `/app/simulate`: this will simulate a transaction, and return some information such as gas used.
- `/app/version`: this will return the application's version.
- `/store/{path}`: this will query the store directly.
- `/p2p/filter/addr/{port}`: this will return a filtered list of the node's P2P peers by address port.
- `/p2p/filter/id/{id}`: this will return a filtered list of the node's P2P peers by ID.
- `/broadcast_tx_{aync,async,commit}`: these 3 endpoint will broadcast a transaction to other peers. CLI, gRPC and REST expose [a way to broadcast transations](./transactions.md#broadcasting-the-transaction), but they all use these 3 Tendermint RPCs under the hood.

## Comparison Table

| Name | Advantages | Disadvantages |
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| gRPC | - can use code-generated stubs in various languages<br>- supports streaming and bidirectional communication (HTTP2)<br>- small wire binary sizes, faster transmission | - based on HTTP2, not available in browsers<br>- learning curve (mostly due to Protobuf) |
| REST | - ubiquitous<br>- client libraries in all languages, faster implementation<br> | - only supports unary request-response communication (HTTP1.1)<br>- bigger over-the-wire message sizes (JSON) |
| Tendermint RPC | - easy to use | - bigger over-the-wire message sizes (JSON) |

## Next {hide}

Learn about [events](./events.md) {hide}
2 changes: 1 addition & 1 deletion docs/core/node.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Upon starting, the node will bootstrap its RPC and P2P server and start dialing

## Other commands

To discover how to concretely run a node and interact with it, please refer to our [Running a Node](../run-node/README.md) guide.
To discover how to concretely run a node and interact with it, please refer to our [Running a Node, API and CLI](../run-node/README.md) guide.

## Next {hide}

Expand Down
2 changes: 1 addition & 1 deletion docs/core/ocap.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
order: 9
order: 10
-->

# Object-Capability Model
Expand Down
2 changes: 1 addition & 1 deletion docs/core/runtx_middleware.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
order: 10
order: 11
-->

# RunTx recovery middleware
Expand Down
2 changes: 1 addition & 1 deletion docs/core/telemetry.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
order: 8
order: 9
-->

# Telemetry
Expand Down
8 changes: 6 additions & 2 deletions docs/core/transactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,14 +141,18 @@ simd tx send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000stake

The `Tx` service exposes a handful of utility functions, such as simulating a transaction or querying a transaction, and also one method to broadcast transactions.

An example of broadcasting a transaction is shown in [TODO](https://github.com/cosmos/cosmos-sdk/issues/7657). Please note that the `BroadcastTx` endpoint takes `TxRaw`, not bytes.
An example of broadcasting a transaction is shown in [TODO](https://github.com/cosmos/cosmos-sdk/issues/7657).

#### REST

Each gRPC method has its corresponding REST endpoint, generated using [gRPC-gateway](https://github.com/grpc-ecosystem/grpc-gateway). Therefore, instead of using gRPC, you can also use HTTP to broadcast the same transaction, on the `POST /cosmos/tx/v1beta1/broadcast_tx` endpoint.
Each gRPC method has its corresponding REST endpoint, generated using [gRPC-gateway](https://github.com/grpc-ecosystem/grpc-gateway). Therefore, instead of using gRPC, you can also use HTTP to broadcast the same transaction, on the `POST /cosmos/tx/v1beta1/txs` endpoint.

An example can be seen [here TODO](https://github.com/cosmos/cosmos-sdk/issues/7657)

#### Tendermint RPC

The three methods presented above are actually higher abstractions over the Tendermint RPC `/broadcast_tx_{async,sync,commit}` endpoints, documented [here](https://docs.tendermint.com/master/rpc/#/Tx). This means that you can use the Tendermint RPC endpoints directly to broadcast the transaction, if you wish so.

## Next {hide}

Learn about the [context](./context.md) {hide}
4 changes: 2 additions & 2 deletions docs/interfaces/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
<!--
order: false
parent:
order: 6
order: 7
-->

# Interfaces
# Interfaces (Deprecated)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why marking this as deprecated as opposed to just removing this section entirely?

It seems like a number of these subsections are pretty much out of date and/or made redundant by the new docs proposed in this PR. Specifically:

  • "introduction to interfaces" can probably go
  • "rest inteerfaces" can probably go
  • "CLI" & "Lifecycle of a Query" sections maybe should rather be updated? Not totally sure..

Either way, i'm in favor of merging this PR as is as long as we have explicit issues / notes in existing issues to clean up the rest of these. Thoughts?

Copy link
Contributor Author

@amaury1093 amaury1093 Dec 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah exactly, i added an item in #7661 to do this exactly: remove this Interfaces section, move relevant docs to Core Concepts (when that makes sense, would need to have a more detailed look).


This repository contains documentation on interfaces for Cosmos SDK applications.

Expand Down
11 changes: 11 additions & 0 deletions docs/migrations/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!--
order: false
parent:
order: 6
-->

# Migrations

This folder contains all the migration guides to update your app and modules to Cosmos v0.40 Stargate.

1. [REST Endpoints Migration](./rest.md)
4 changes: 4 additions & 0 deletions docs/migrations/rest.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
<!--
order: 1
-->

# REST Endpoints Migration

Migrate your REST endpoints to the Stargate ones. {synopsis}
Expand Down
2 changes: 1 addition & 1 deletion docs/run-node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ parent:
order: 5
-->

# Running a Node
# Running a Node, API and CLI

This folder contains documentation on how to run a node and interact with it.

Expand Down
Loading