diff --git a/docs/basics/README.md b/docs/basics/README.md index ad54c6bee783..03934dcd5112 100644 --- a/docs/basics/README.md +++ b/docs/basics/README.md @@ -10,7 +10,8 @@ This repository contains reference documentation on the basic concepts of the Co 1. [Anatomy of an SDK Application](./app-anatomy.md) 2. [Lifecycle of a transaction](./tx-lifecycle.md) -3. [Accounts](./accounts.md) -4. [Gas and Fees](./gas-fees.md) +3. [Lifecycle of a query](./query-lifecycle.md) +4. [Accounts](./accounts.md) +5. [Gas and Fees](./gas-fees.md) After reading the basics, head on to the [Core Reference](../core/README.md) for more advanced material. diff --git a/docs/basics/accounts.md b/docs/basics/accounts.md index 930914d73501..3b50c7a145b7 100644 --- a/docs/basics/accounts.md +++ b/docs/basics/accounts.md @@ -1,5 +1,5 @@ # Accounts diff --git a/docs/basics/app-anatomy.md b/docs/basics/app-anatomy.md index 148ba9ba5da4..225da66c3c29 100644 --- a/docs/basics/app-anatomy.md +++ b/docs/basics/app-anatomy.md @@ -120,7 +120,7 @@ Here are descriptions of what each of the four fields means: - `TxConfig`: `TxConfig` defines an interface a client can utilize to generate an application-defined concrete transaction type. Currently, the SDK handles two transaction types: `SIGN_MODE_DIRECT` (which uses Protobuf binary as over-the-wire encoding) and `SIGN_MODE_LEGACY_AMINO_JSON` (which depends on Amino). Read more about transactions [here](../core/transactions.md). - `Amino`: Some legacy parts of the SDK still use Amino for backwards-compatibility. Each module exposes a `RegisterLegacyAmino` method to register the module's specific types within Amino. This `Amino` codec should not be used by app developers anymore, and will be removed in future releases. -The SDK exposes a `MakeTestEncodingConfig` function used to create a `EncodingConfig` for the app constructor (`NewApp`). It uses Protobuf as a default `Marshaler`. +The SDK exposes a `MakeTestEncodingConfig` function used to create a `EncodingConfig` for the app constructor (`NewApp`). It uses Protobuf as a default `Marshaler`. NOTE: this function is marked deprecated and should only be used to create an app or in tests. We are working on refactoring codec management in a post Stargate release. See an example of a `MakeTestEncodingConfig` from `simapp`: @@ -160,7 +160,7 @@ Each module should also implement the `RegisterServices` method as part of the [ ### gRPC `Query` Services -gRPC `Query` services are introduced in the v0.40 Stargate release. They allow users to query the state using [gRPC](https://grpc.io). They are enabled by default, and can be configued under the `grpc.enable` and `grpc.address` fields inside `app.toml`. +gRPC `Query` services are introduced in the v0.40 Stargate release. They allow users to query the state using [gRPC](https://grpc.io). They are enabled by default, and can be configued under the `grpc.enable` and `grpc.address` fields inside [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml). gRPC `Query` services are defined in the module's Protobuf definition files, specifically inside `query.proto`. The `query.proto` definition file exposes a single `Query` [Protobuf service](https://developers.google.com/protocol-buffers/docs/proto#services). Each gRPC query endpoint corresponds to a service method, starting with the `rpc` keyword, inside the `Query` service. @@ -208,7 +208,7 @@ Some external clients may not wish to use gRPC. The SDK provides in this case a The REST endpoints are defined in the Protobuf files, along with the gRPC services, using Protobuf annotations. Modules that want to expose REST queries should add `google.api.http` annotations to their `rpc` methods. By default, all REST endpoints defined in the SDK have an URL starting with the `/cosmos/` prefix. -The SDK also provides a development endpoint to generate [Swagger](https://swagger.io/) definition files for these REST endpoints. This endpoint can be enabled inside the `app.toml` config file, under the `api.swagger` key. +The SDK also provides a development endpoint to generate [Swagger](https://swagger.io/) definition files for these REST endpoints. This endpoint can be enabled inside the [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml) config file, under the `api.swagger` key. #### Legacy API REST Endpoints diff --git a/docs/basics/gas-fees.md b/docs/basics/gas-fees.md index 270066996e72..2ab586497cd9 100644 --- a/docs/basics/gas-fees.md +++ b/docs/basics/gas-fees.md @@ -1,5 +1,5 @@ # Gas and Fees diff --git a/docs/basics/query-lifecycle.md b/docs/basics/query-lifecycle.md new file mode 100644 index 000000000000..4b4c509bb354 --- /dev/null +++ b/docs/basics/query-lifecycle.md @@ -0,0 +1,140 @@ + + +# Query Lifecycle + +This document describes the lifecycle of a query in a SDK application, from the user interface to application stores and back. {synopsis} + +## Pre-requisite Readings + +- [Transaction Lifecycle](./tx-lifecycle.md) {prereq} + +## Query Creation + +A [**query**](../building-modules/messages-and-queries.md#queries) is a request for information made by end-users of applications through an interface and processed by a full-node. Users can query information about the network, the application itself, and application state directly from the application's stores or modules. Note that queries are different from [transactions](../core/transactions.md) (view the lifecycle [here](./tx-lifecycle.md)), particularly in that they do not require consensus to be processed (as they do not trigger state-transitions); they can be fully handled by one full-node. + +For the purpose of explaining the query lifecycle, let's say `MyQuery` is requesting a list of delegations made by a certain delegator address in the application called `simapp`. As to be expected, the [`staking`](../../x/staking/spec/README.md) module handles this query. But first, there are a few ways `MyQuery` can be created by users. + +### CLI + +The main interface for an application is the command-line interface. Users connect to a full-node and run the CLI directly from their machines - the CLI interacts directly with the full-node. To create `MyQuery` from their terminal, users type the following command: + +```bash +simd query staking delegations +``` + +This query command was defined by the [`staking`](../../x/staking/spec/README.md) module developer and added to the list of subcommands by the application developer when creating the CLI. + +Note that the general format is as follows: + +```bash +simd query [moduleName] [command] --flag +``` + +To provide values such as `--node` (the full-node the CLI connects to), the user can use the [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml) config file to set them or provide them as flags. + +The CLI understands a specific set of commands, defined in a hierarchical structure by the application developer: from the [root command](../core/cli.md#root-command) (`simd`), the type of command (`Myquery`), the module that contains the command (`staking`), and command itself (`delegations`). Thus, the CLI knows exactly which module handles this command and directly passes the call there. + +### gRPC + +Another interface through which users can make queries, introduced in Cosmos SDK v0.40, is [gRPC](https://grpc.io) requests to a [gRPC server](../core/grpc_rest.md#grpc-server). The endpoints are defined as [Protocol Buffers](https://developers.google.com/protocol-buffers) service methods inside `.proto` files, written in Protobuf's own language-agnostic interface definition language (IDL). The Protobuf ecosystem developed tools for code-generation from `*.proto` files into various languages. These tools allow to build gRPC clients easily. + +One such tool is [grpcurl](https://github.com/fullstorydev/grpcurl), and a gRPC request for `MyQuery` using this client looks like: + +```bash +grpcurl \ + -plaintext # We want results in plain test + -import-path ./proto \ # Import these .proto files + -proto ./proto/cosmos/staking/v1beta1/query.proto \ # Look into this .proto file for the Query protobuf service + -d '{"address":"$MY_DELEGATOR"}' \ # Query arguments + localhost:9090 \ # gRPC server endpoint + cosmos.staking.v1beta1.Query/Delegations # Fully-qualified service method name +``` + +### REST + +Another interface through which users can make queries is through HTTP Requests to a [REST server](../core/grpc_rest.md#rest-server). The REST server is fully auto-generated from Protobuf services, using [gRPC-gateway](https://github.com/grpc-ecosystem/grpc-gateway). + +An example HTTP request for `MyQuery` looks like: + +```bash +GET http://localhost:1317/cosmos/staking/v1beta1/delegators/{delegatorAddr}/delegations +``` + +## How Queries are Handled by the CLI + +The examples above show how an external user can interact with a node by querying its state. To understand more in details the exact lifecycle of a query, let's dig into how the CLI prepares the query, and how the node handles it. The interactions from the users' perspective are a bit different, but the underlying functions are almost identical because they are implementations of the same command defined by the module developer. This step of processing happens within the CLI, gRPC or REST server and heavily involves a `client.Context`. + +### Context + +The first thing that is created in the execution of a CLI command is a `client.Context`. A `client.Context` is an object that stores all the data needed to process a request on the user side. In particular, a `client.Context` stores the following: + +- **Codec**: The [encoder/decoder](../core/encoding.md) used by the application, used to marshal the parameters and query before making the Tendermint RPC request and unmarshal the returned response into a JSON object. The default codec used by the CLI is Protobuf. +- **Account Decoder**: The account decoder from the [`auth`](../..//x/auth/spec/README.md) module, which translates `[]byte`s into accounts. +- **RPC Client**: The Tendermint RPC Client, or node, to which the request will be relayed to. +- **Keyring**: A [Key Manager](../basics/accounts.md#keyring) used to sign transactions and handle other operations with keys. +- **Output Writer**: A [Writer](https://golang.org/pkg/io/#Writer) used to output the response. +- **Configurations**: The flags configured by the user for this command, including `--height`, specifying the height of the blockchain to query and `--indent`, which indicates to add an indent to the JSON response. + +The `client.Context` also contains various functions such as `Query()` which retrieves the RPC Client and makes an ABCI call to relay a query to a full-node. + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/client/context.go#L20-L50 + +The `client.Context`'s primary role is to store data used during interactions with the end-user and provide methods to interact with this data - it is used before and after the query is processed by the full-node. Specifically, in handling `MyQuery`, the `client.Context` is utilized to encode the query parameters, retrieve the full-node, and write the output. Prior to being relayed to a full-node, the query needs to be encoded into a `[]byte` form, as full-nodes are application-agnostic and do not understand specific types. The full-node (RPC Client) itself is retrieved using the `client.Context`, which knows which node the user CLI is connected to. The query is relayed to this full-node to be processed. Finally, the `client.Context` contains a `Writer` to write output when the response is returned. These steps are further described in later sections. + +### Arguments and Route Creation + +At this point in the lifecycle, the user has created a CLI command with all of the data they wish to include in their query. A `client.Context` exists to assist in the rest of the `MyQuery`'s journey. Now, the next step is to parse the command or request, extract the arguments, and encode everything. These steps all happen on the user side within the interface they are interacting with. + +#### Encoding + +In our case (querying an address's delegations), `MyQuery` contains an [address](./accounts.md#addresses) `delegatorAddress` as its only argument. However, the request can only contain `[]byte`s, as it will be relayed to a consensus engine (e.g. Tendermint Core) of a full-node that has no inherent knowledge of the application types. Thus, the `codec` of `client.Context` is used to marshal the address. + +Here is what the code looks like for the CLI command: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/x/staking/client/cli/query.go#L324-L327 + +#### gRPC Query Client Creation + +The SDK leverages code generated from Protobuf services to make queries. The `staking` module's `MyQuery` service generates a `queryClient`, which the CLI will use to make queries. Here is the relevant code: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/x/staking/client/cli/query.go#L318-L342 + +Under the hood, the `client.Context` has a `Query()` function used to retrieve the pre-configured node and relay a query to it; the function takes the query fully-qualified service method name as path (in our case: `/cosmos.staking.v1beta1.Query/Delegations`), and arguments as parameters. It first retrieves the RPC Client (called the [**node**](../core/node.md)) configured by the user to relay this query to, and creates the `ABCIQueryOptions` (parameters formatted for the ABCI call). The node is then used to make the ABCI call, `ABCIQueryWithOptions()`. + +Here is what the code looks like: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/client/query.go#L65-L91 + +## RPC + +With a call to `ABCIQueryWithOptions()`, `MyQuery` is received by a [full-node](../core/encoding.md) which will then process the request. Note that, while the RPC is made to the consensus engine (e.g. Tendermint Core) of a full-node, queries are not part of consensus and will not be broadcasted to the rest of the network, as they do not require anything the network needs to agree upon. + +Read more about ABCI Clients and Tendermint RPC in the Tendermint documentation [here](https://tendermint.com/rpc). + +## Application Query Handling + +When a query is received by the full-node after it has been relayed from the underlying consensus engine, it is now being handled within an environment that understands application-specific types and has a copy of the state. [`baseapp`](../core/baseapp.md) implements the ABCI [`Query()`](../core/baseapp.md#query) function and handles gRPC queries. The query route is parsed, and it it matches the fully-qualified service method name of an existing service method (most likely in one of the modules), then `baseapp` will relay the request to the relevant module. + +Apart from gRPC routes, `baseapp` also handles four different types of queries: `app`, `store`, `p2p`, and `custom`. The first three types (`app`, `store`, `p2p`) are purely application-level and thus directly handled by `baseapp` or the stores, but the `custom` query type requires `baseapp` to route the query to a module's [legacy queriers](../building-modules/query-services.md#legacy-queriers). To learn more about these queries, please refer to [this guide](../core/grpc_rest.md#tendermint-rpc). + +Since `MyQuery` has a Protobuf fully-qualified service method name from the `staking` module (recall `/cosmos.staking.v1beta1.Query/Delegations`), `baseapp` first parses the path, then uses its own internal `GRPCQueryRouter` to retrieve the corresponding gRPC handler, and routes the query to the module. The gRPC handler is responsible for recognizing this query, retrieving the appropriate values from the application's stores, and returning a response. Read more about query services [here](../building-modules/query-services.md). + +Once a result is received from the querier, `baseapp` begins the process of returning a response to the user. + +## Response + +Since `Query()` is an ABCI function, `baseapp` returns the response as an [`abci.ResponseQuery`](https://tendermint.com/docs/spec/abci/abci.html#messages) type. The `client.Context` `Query()` routine receives the response and. + +### CLI Response + +The application [`codec`](../core/encoding.md) is used to unmarshal the response to a JSON and the `client.Context` prints the output to the command line, applying any configurations such as the output type (text, JSON or YAML). + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/client/context.go#L248-L283 + +And that's a wrap! The result of the query is outputted to the console by the CLI. + +## Next {hide} + +Read more about [accounts](./accounts.md). {hide} diff --git a/docs/building-modules/module-interfaces.md b/docs/building-modules/module-interfaces.md index 12c692cf52e5..0f811db4bf75 100644 --- a/docs/building-modules/module-interfaces.md +++ b/docs/building-modules/module-interfaces.md @@ -126,9 +126,9 @@ service Query{ } ``` -gRPC gateway is started in-process along with the application and Tendermint. It can be enabled or disabled by setting gRPC Configuration `enable` in `app.toml`. +gRPC gateway is started in-process along with the application and Tendermint. It can be enabled or disabled by setting gRPC Configuration `enable` in [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml). -The SDK provides a command for generating [Swagger](https://swagger.io/) documentation (`protoc-gen-swagger`). Setting `swagger` in `app.toml` defines if swagger documentation should be automatically registered. +The SDK provides a command for generating [Swagger](https://swagger.io/) documentation (`protoc-gen-swagger`). Setting `swagger` in [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml) defines if swagger documentation should be automatically registered. ## Legacy REST diff --git a/docs/core/README.md b/docs/core/README.md index 23df86d26909..94a993f5a556 100644 --- a/docs/core/README.md +++ b/docs/core/README.md @@ -9,17 +9,19 @@ parent: This repository contains reference documentation on the core concepts of the Cosmos SDK. 1. [`BaseApp`](./baseapp.md) -2. [Transaction](./transactions.md) -3. [Context](./context.md) -4. [Node Client](./node.md) -5. [Store](./store.md) -6. [Encoding](./encoding.md) -7. [gRPC, REST and Tendermint Endpoints](./grpc_rest.md) -8. [Events](./events.md) -9. [Telemetry](./telemetry.md) -10. [Object-Capabilities](./ocap.md) -11. [RunTx recovery middleware](./runtx_middleware.md) -12. [Protobuf documentation](./proto-docs.md) +1. [Transaction](./transactions.md) +1. [Context](./context.md) +1. [Node Client](./node.md) +1. [Store](./store.md) +1. [Encoding](./encoding.md) +1. [gRPC, REST and Tendermint Endpoints](./grpc_rest.md) +1. [Command-Line Interface](./cli.md) +1. [Events](./events.md) +1. [Telemetry](./telemetry.md) +1. [Object-Capabilities](./ocap.md) +1. [RunTx recovery middleware](./runtx_middleware.md) +1. [Simulation](./simulation.md) +1. [Protobuf documentation](./proto-docs.md) After reading about the core concepts, check the [IBC documentation](../ibc/README.md) to learn more about the IBC core concepts and how to integrate it to you application. diff --git a/docs/core/cli.md b/docs/core/cli.md new file mode 100644 index 000000000000..8e11bf488d2b --- /dev/null +++ b/docs/core/cli.md @@ -0,0 +1,125 @@ + + +# Command-Line Interface + +This document describes how commmand-line interface (CLI) works on a high-level, for an [**application**](../basics/app-anatomy.md). A separate document for implementing a CLI for an SDK [**module**](../building-modules/intro.md) can be found [here](../building-modules/module-interfaces.md#cli). {synopsis} + +## Command-Line Interface + +### Example Command + +There is no set way to create a CLI, but SDK modules typically use the [Cobra Library](https://github.com/spf13/cobra). Building a CLI with Cobra entails defining commands, arguments, and flags. [**Commands**](#commands) understand the actions users wish to take, such as `tx` for creating a transaction and `query` for querying the application. Each command can also have nested subcommands, necessary for naming the specific transaction type. Users also supply **Arguments**, such as account numbers to send coins to, and [**Flags**](#flags) to modify various aspects of the commands, such as gas prices or which node to broadcast to. + +Here is an example of a command a user might enter to interact with the simapp CLI `simd` in order to send some tokens: + +```bash +simd tx bank send $MY_VALIDATOR_ADDRESS $RECIPIENT 1000stake --gas auto --gas-prices +``` + +The first four strings specify the command: + +- The root command for the entire application `simd`. +- The subcommand `tx`, which contains all commands that let users create transactions. +- The subcommand `bank` to indicate which module to route the command to ([`x/bank`](../../x/bank/spec/README.md) module in this case). +- The type of transaction `send`. + +The next two strings are arguments: the `from_address` the user wishes to send from, the `to_address` of the recipient, and the `amount` they want to send. Finally, the last few strings of the command are optional flags to indicate how much the user is willing to pay in fees (calculated using the amount of gas used to execute the transaction and the gas prices provided by the user). + +The CLI interacts with a [node](../core/node.md) to handle this command. The interface itself is defined in a `main.go` file. + +### Building the CLI + +The `main.go` file needs to have a `main()` function that creates a root command, to which all the application commands will be added as subcommands. The root command additionally handles: + +- **setting configurations** by reading in configuration files (e.g. the sdk config file). +- **adding any flags** to it, such as `--chain-id`. +- **instantiating the `codec`** by calling the application's `MakeCodec()` function (called `MakeTestEncodingConfig` in `simapp`). The [`codec`](../core/encoding.md) is used to encode and decode data structures for the application - stores can only persist `[]byte`s so the developer must define a serialization format for their data structures or use the default, Protobuf. +- **adding subcommand** for all the possible user interactions, including [transaction commands](#transaction-commands) and [query commands](#query-commands). + +The `main()` function finally creates an executor and [execute](https://godoc.org/github.com/spf13/cobra#Command.Execute) the root command. See an example of `main()` function from the `simapp` application: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/simd/main.go#L12-L24 + +The rest of the document will detail what needs to be implemented for each step and include smaller portions of code from the `simapp` CLI files. + +## Adding Commands to the CLI + +Every application CLI first constructs a root command, then adds functionality by aggregating subcommands (often with further nested subcommands) using `rootCmd.AddCommand()`. The bulk of an application's unique capabilities lies in its transaction and query commands, called `TxCmd` and `QueryCmd` respectively. + +### Root Command + +The root command (called `rootCmd`) is what the user first types into the command line to indicate which application they wish to interact with. The string used to invoke the command (the "Use" field) is typically the name of the application suffixed with `-d`, e.g. `simd` or `gaiad`. The root command typically includes the following commands to support basic functionality in the application. + +- **Status** command from the SDK rpc client tools, which prints information about the status of the connected [`Node`](../core/node.md). The Status of a node includes `NodeInfo`,`SyncInfo` and `ValidatorInfo`. +- **Keys** [commands](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/client/keys) from the SDK client tools, which includes a collection of subcommands for using the key functions in the SDK crypto tools, including adding a new key and saving it to the keyring, listing all public keys stored in the keyring, and deleting a key. For example, users can type `simd keys add ` to add a new key and save an encrypted copy to the keyring, using the flag `--recover` to recover a private key from a seed phrase or the flag `--multisig` to group multiple keys together to create a multisig key. For full details on the `add` key command, see the code [here](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/client/keys/add.go). For more details about usage of `--keyring-backend` for storage of key credentials look at the [keyring docs](../run-node/keyring.md). +- **Server** commands from the SDK server package. These commands are responsible for providing the mechanisms necessary to start an ABCI Tendermint application and provides the CLI framework (based on [cobra](github.com/spf13/cobra)) necessary to fully bootstrap an application. The package exposes two core functions: `StartCmd` and `ExportCmd` which creates commands to start the application and export state respectively. Click [here](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/server) to learn more. +- [**Transaction**](#transaction-commands) commands. +- [**Query**](#query-commands) commands. + +Next is an example `rootCmd` function from the `simapp` application. It instantiates the root command, adds a [_persistent_ flag](#flags) and `PreRun` function to be run before every execution, and adds all of the necessary subcommands. + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/simd/cmd/root.go#L37-L93 + +The root-level `status` and `keys` subcommands are common across most applications and do not interact with application state. The bulk of an application's functionality - what users can actually _do_ with it - is enabled by its `tx` and `query` commands. + +### Transaction Commands + +[Transactions](./transactions.md) are objects wrapping [`Msg`s](../building-modules/messages-and-queries.md#messages) that trigger state changes. To enable the creation of transactions using the CLI interface, a function `txCmd` is generally added to the `rootCmd`: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/simd/cmd/root.go#L86-L92 + +This `txCmd` function adds all the transaction available to end-users for the application. This typically includes: + +- **Sign command** from the [`auth`](../../x/auth/spec/README.md) module that signs messages in a transaction. To enable multisig, add the `auth` module's `MultiSign` command. Since every transaction requires some sort of signature in order to be valid, thithe signing command is necessary for every application. +- **Broadcast command** from the SDK client tools, to broadcast transactions. +- **All [module transaction commands](../building-modules/module-interfaces.md#transaction-commands)** the application is dependent on, retrieved by using the [basic module manager's](../building-modules/module-manager.md#basic-manager) `AddTxCommands()` function. + +Here is an example of a `txCmd` aggregating these subcommands from the `simapp` application: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/simd/cmd/root.go#L123-L149 + +### Query Commands + +[**Queries**](../building-modules/messages-and-queries.md#queries) are objects that allow users to retrieve information about the application's state. To enable the creation of transactions using the CLI interface, a function `txCmd` is generally added to the `rootCmd`: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/simd/cmd/root.go#L86-L92 + +This `queryCmd` function adds all the queries available to end-users for the application. This typically includes: + +- **QueryTx** and/or other transaction query commands] from the `auth` module which allow the user to search for a transaction by inputting its hash, a list of tags, or a block height. These queries allow users to see if transactions have been included in a block. +- **Account command** from the `auth` module, which displays the state (e.g. account balance) of an account given an address. +- **Validator command** from the SDK rpc client tools, which displays the validator set of a given height. +- **Block command** from the SDK rpc client tools, which displays the block data for a given height. +- **All [module query commands](../building-modules/module-interfaces.md#query-commands)** the application is dependent on, retrieved by using the [basic module manager's](../building-modules/module-manager.md#basic-manager) `AddQueryCommands()` function. + +Here is an example of a `queryCmd` aggregating subcommands from the `simapp` application: + ++++ https://github.com/cosmos/cosmos-sdk/blob/0.40.0/simapp/simd/cmd/root.go#L99-L121 + +## Flags + +Flags are used to modify commands; developers can include them in a `flags.go` file with their CLI. Users can explicitly include them in commands or pre-configure them by inside their [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml). Commonly pre-configured flags include the `--node` to connect to and `--chain-id` of the blockchain the user wishes to interact with. + +A _persistent_ flag (as opposed to a _local_ flag) added to a command transcends all of its children: subcommands will inherit the configured values for these flags. Additionally, all flags have default values when they are added to commands; some toggle an option off but others are empty values that the user needs to override to create valid commands. A flag can be explicitly marked as _required_ so that an error is automatically thrown if the user does not provide a value, but it is also acceptable to handle unexpected missing flags differently. + +Flags are added to commands directly (generally in the [module's CLI file](../building-modules/module-interfaces.md#flags) where module commands are defined) and no flag except for the `rootCmd` persistent flags has to be added at application level. It is common to add a _persistent_ flag for `--chain-id`, the unique identifier of the blockchain the application pertains to, to the root command. Adding this flag can be done in the `main()` function. Adding this flag makes sense as the chain ID should not be changing across commands in this application CLI. Here is an example from the `simapp` application: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/simd/cmd/root.go#L118 + +## Configurations + +It is vital that the root command of an application uses `PersistentPreRun()` cobra command property for executing the command, so all child commands have access to the server and client contexts. These contexts are set as their default values initially and maybe modified, scoped to the command, in their respective `PersistentPreRun()` functions. Note that the `client.Context` is typically pre-populated with "default" values that may be useful for all commands to inherit and override if necessary. + +Here is an example of an `PersistentPreRun()` function from `simapp``: + ++++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/simd/cmd/root.go#L54-L60 + +The `SetCmdClientContextHandler` call reads persistent flags via `ReadPersistentCommandFlags` which creates a `client.Context` and sets that on the root command's `Context`. + +The `InterceptConfigsPreRunHandler` call creates a viper literal, default `server.Context`, and a logger and sets that on the root command's `Context`. The `server.Context` will be modified and saved to disk via the internal `interceptConfigs` call, which either reads or creates a Tendermint configuration based on the home path provided. In addition, `interceptConfigs` also reads and loads the application configuration, `app.toml`, and binds that to the `server.Context` viper literal. This is vital so the application can get access to not only the CLI flags, but also to the application configuration values provided by this file. + +## Next {hide} + +Learn about [events](./events.md) {hide} diff --git a/docs/core/events.md b/docs/core/events.md index bffa432ba4bc..87bf87066133 100644 --- a/docs/core/events.md +++ b/docs/core/events.md @@ -1,5 +1,5 @@ # Events diff --git a/docs/core/grpc_rest.md b/docs/core/grpc_rest.md index e89b0653ad2a..c78da2d94e48 100644 --- a/docs/core/grpc_rest.md +++ b/docs/core/grpc_rest.md @@ -26,13 +26,13 @@ Each module exposes [`Msg` and `Query` Protobuf services](../building-modules/me https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/server/types/app.go#L39-L41 -The `grpc.Server` is a concrete gRPC server, which spawns and serves any gRPC requests. This server can be configured inside `$TMHOME/config/app.toml`: +The `grpc.Server` is a concrete gRPC server, which spawns and serves any gRPC requests. This server can be configured inside `~/.simapp/config/app.toml`: - `grpc.enable = true|false` field defines if the gRPC server should be enabled. Defaults to `true`. - `grpc.address = {string}` field defines the address (really, the port, since the host should be kept at `0.0.0.0`) the server should bind to. Defaults to `0.0.0.0:9000`. ::tip -`$TMHOME` is the directory where the node's configuration and databases are stored. By default, it's set to `~/.{app_name}`. +`~/.simapp` is the directory where the node's configuration and databases are stored. By default, it's set to `~/.{app_name}`. :: Once the gRPC server is started, you can send requests to it using a gRPC client. Some examples are given in our [Interact with the Node](../run-node/interact-node.md#using-grpc) tutorial. @@ -43,11 +43,11 @@ An overview of all available gRPC endpoints shipped with the Cosmos SDK is [Prot In Cosmos SDK v0.40, the node continues to serve a REST server. However, the existing routes present in version v0.39 and earlier are now marked as deprecated, and new routes have been added via gRPC-gateway. -All routes are configured under the following fields in `$TMHOME/config/app.toml`: +All routes are configured under the following fields in `~/.simapp/config/app.toml`: - `api.enable = true|false` field defines if the REST server should be enabled. Defaults to `true`. - `api.address = {string}` field defines the address (really, the port, since the host should be kept at `0.0.0.0`) the server should bind to. Defaults to `tcp://0.0.0.0:1317`. -- some additional API configuration options are defined in `$TMHOME/config/app.toml`, along with comments, please refer to that file directly. +- some additional API configuration options are defined in `~/.simapp/config/app.toml`, along with comments, please refer to that file directly. ### gRPC-gateway REST Routes @@ -69,13 +69,13 @@ For application developers, Legacy REST API routes needs to be wired up to the R A [Swagger](https://swagger.io/) (or OpenAPIv2) specification file is exposed under the `/swagger` route on the API server. Swagger is an open specification describing the API endpoints a server serves, including description, input arguments, return types and much more about each endpoint. -Enabling the `/swagger` endpoint is configurable inside `$TMHOME/config/app.toml` via the `api.swagger` field, which is set to true by default. +Enabling the `/swagger` endpoint is configurable inside `~/.simapp/config/app.toml` via the `api.swagger` field, which is set to true by default. For application developers, you may want to generate your own Swagger definitions based on your custom modules. The SDK's [Swagger generation script](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc4/scripts/protoc-swagger-gen.sh) is a good place to start. ## Tendermint RPC -Independently from the Cosmos SDK, Tendermint also exposes a RPC server. This RPC server can be configured by tuning parameters under the `rpc` table in the `$TMHOME/config/config.toml`, the default listening address is `tcp://0.0.0.0:26657`. An OpenAPI specification of all Tendermint RPC endpoints is available [here](https://docs.tendermint.com/master/rpc/). +Independently from the Cosmos SDK, Tendermint also exposes a RPC server. This RPC server can be configured by tuning parameters under the `rpc` table in the `~/.simapp/config/config.toml`, the default listening address is `tcp://0.0.0.0:26657`. An OpenAPI specification of all Tendermint RPC endpoints is available [here](https://docs.tendermint.com/master/rpc/). Some Tendermint RPC endpoints are directly related to the Cosmos SDK: @@ -98,4 +98,4 @@ Some Tendermint RPC endpoints are directly related to the Cosmos SDK: ## Next {hide} -Learn about [events](./events.md) {hide} +Learn about [the CLI](./cli.md) {hide} diff --git a/docs/core/ocap.md b/docs/core/ocap.md index 523595c32479..3f0ec2e1d7ec 100644 --- a/docs/core/ocap.md +++ b/docs/core/ocap.md @@ -1,5 +1,5 @@ # Object-Capability Model diff --git a/docs/core/runtx_middleware.md b/docs/core/runtx_middleware.md index 8d205ca284fe..9c978fbbe00b 100644 --- a/docs/core/runtx_middleware.md +++ b/docs/core/runtx_middleware.md @@ -1,5 +1,5 @@ # RunTx recovery middleware @@ -22,12 +22,12 @@ type RecoveryHandler func(recoveryObj interface{}) error **Contract:** -* RecoveryHandler returns `nil` if `recoveryObj` wasn't handled and should be passed to the next recovery middleware; -* RecoveryHandler returns a non-nil `error` if `recoveryObj` was handled; +- RecoveryHandler returns `nil` if `recoveryObj` wasn't handled and should be passed to the next recovery middleware; +- RecoveryHandler returns a non-nil `error` if `recoveryObj` was handled; ## Custom RecoveryHandler register -``BaseApp.AddRunTxRecoveryHandler(handlers ...RecoveryHandler)`` +`BaseApp.AddRunTxRecoveryHandler(handlers ...RecoveryHandler)` BaseApp method adds recovery middleware to the default recovery chain. diff --git a/docs/using-the-sdk/simulation.md b/docs/core/simulation.md similarity index 54% rename from docs/using-the-sdk/simulation.md rename to docs/core/simulation.md index e8fcb8088f07..9b838b40b526 100644 --- a/docs/using-the-sdk/simulation.md +++ b/docs/core/simulation.md @@ -1,10 +1,14 @@ + + # Cosmos Blockchain Simulator The Cosmos SDK offers a full fledged simulation framework to fuzz test every message defined by a module. -On the SDK, this functionality is provided by the[`SimApp`](https://github.com/cosmos/cosmos-sdk/blob/master/simapp/app.go), which is a -`Baseapp` application that is used for running the [`simulation`](https://github.com/cosmos/cosmos-sdk/tree/master/x/simulation) module. +On the SDK, this functionality is provided by the[`SimApp`](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/app.go), which is a +`Baseapp` application that is used for running the [`simulation`](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/x/simulation) module. This module defines all the simulation logic as well as the operations for randomized parameters like accounts, balances etc. @@ -26,30 +30,30 @@ provided operations (randomized or not). The simulation app has different commands, each of which tests a different failure type: -* `AppImportExport`: The simulator exports the initial app state and then it -creates a new app with the exported `genesis.json` as an input, checking for -inconsistencies between the stores. -* `AppSimulationAfterImport`: Queues two simulations together. The first one provides the app state (_i.e_ genesis) to the second. Useful to test software upgrades or hard-forks from a live chain. -* `AppStateDeterminism`: Checks that all the nodes return the same values, in the same order. -* `BenchmarkInvariants`: Analysis of the performance of running all modules' invariants (_i.e_ sequentially runs a [benchmark](https://golang.org/pkg/testing/#hdr-Benchmarks) test). An invariant checks for -differences between the values that are on the store and the passive tracker. Eg: total coins held by accounts vs total supply tracker. -* `FullAppSimulation`: General simulation mode. Runs the chain and the specified operations for a given number of blocks. Tests that there're no `panics` on the simulation. It does also run invariant checks on every `Period` but they are not benchmarked. +- `AppImportExport`: The simulator exports the initial app state and then it + creates a new app with the exported `genesis.json` as an input, checking for + inconsistencies between the stores. +- `AppSimulationAfterImport`: Queues two simulations together. The first one provides the app state (_i.e_ genesis) to the second. Useful to test software upgrades or hard-forks from a live chain. +- `AppStateDeterminism`: Checks that all the nodes return the same values, in the same order. +- `BenchmarkInvariants`: Analysis of the performance of running all modules' invariants (_i.e_ sequentially runs a [benchmark](https://golang.org/pkg/testing/#hdr-Benchmarks) test). An invariant checks for + differences between the values that are on the store and the passive tracker. Eg: total coins held by accounts vs total supply tracker. +- `FullAppSimulation`: General simulation mode. Runs the chain and the specified operations for a given number of blocks. Tests that there're no `panics` on the simulation. It does also run invariant checks on every `Period` but they are not benchmarked. Each simulation must receive a set of inputs (_i.e_ flags) such as the number of blocks that the simulation is run, seed, block size, etc. -Check the full list of flags [here](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/simapp/sim_test.go#L34-L50). +Check the full list of flags [here](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/simapp/config.go#L32-L55). ## Simulator Modes In addition to the various inputs and commands, the simulator runs in three modes: 1. Completely random where the initial state, module parameters and simulation -parameters are **pseudo-randomly generated**. + parameters are **pseudo-randomly generated**. 2. From a `genesis.json` file where the initial state and the module parameters are defined. -This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested. + This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested. 3. From a `params.json` file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. -This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated. -The list of available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/x/simulation/params.go#L170-L178). + This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated. + The list of available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/x/simulation/params.go#L44-L52). ::: tip These modes are not mutually exclusive. So you can for example run a randomly @@ -59,7 +63,7 @@ generated genesis state (`1`) with manually generated simulation params (`3`). ## Usage This is a general example of how simulations are run. For more specific examples -check the SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/Makefile#L88-L123). +check the SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/Makefile#L251-L287). ```bash $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \ @@ -72,26 +76,26 @@ check the SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c Here are some suggestions when encountering a simulation failure: -* Export the app state at the height were the failure was found. You can do this -by passing the `-ExportStatePath` flag to the simulator. -* Use `-Verbose` logs. They could give you a better hint on all the operations -involved. -* Reduce the simulation `-Period`. This will run the invariants checks more -frequently. -* Print all the failed invariants at once with `-PrintAllInvariants`. -* Try using another `-Seed`. If it can reproduce the same error and if it fails -sooner you will spend less time running the simulations. -* Reduce the `-NumBlocks` . How's the app state at the height previous to the -failure? -* Run invariants on every operation with `-SimulateEveryOperation`. _Note_: this -will slow down your simulation **a lot**. -* Try adding logs to operations that are not logged. You will have to define a -[Logger](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/x/staking/keeper/keeper.go#L65:17) on your `Keeper`. +- Export the app state at the height were the failure was found. You can do this + by passing the `-ExportStatePath` flag to the simulator. +- Use `-Verbose` logs. They could give you a better hint on all the operations + involved. +- Reduce the simulation `-Period`. This will run the invariants checks more + frequently. +- Print all the failed invariants at once with `-PrintAllInvariants`. +- Try using another `-Seed`. If it can reproduce the same error and if it fails + sooner you will spend less time running the simulations. +- Reduce the `-NumBlocks` . How's the app state at the height previous to the + failure? +- Run invariants on every operation with `-SimulateEveryOperation`. _Note_: this + will slow down your simulation **a lot**. +- Try adding logs to operations that are not logged. You will have to define a + [Logger](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0/x/staking/keeper/keeper.go#L66-L69) on your `Keeper`. ## Use simulation in your SDK-based application Learn how you can integrate the simulation into your SDK-based application: -* Application Simulation Manager -* [Building modules: Simulator](../building-modules/simulator.md) -* Simulator tests +- Application Simulation Manager +- [Building modules: Simulator](../building-modules/simulator.md) +- Simulator tests diff --git a/docs/core/telemetry.md b/docs/core/telemetry.md index b536fb480dff..9e434eef2bec 100644 --- a/docs/core/telemetry.md +++ b/docs/core/telemetry.md @@ -1,5 +1,5 @@ # Telemetry diff --git a/docs/interfaces/README.md b/docs/interfaces/README.md deleted file mode 100644 index f7fc2cd6e29f..000000000000 --- a/docs/interfaces/README.md +++ /dev/null @@ -1,14 +0,0 @@ - - -# Interfaces (Deprecated) - -This repository contains documentation on interfaces for Cosmos SDK applications. - -1. [Introduction to Interaces](./interfaces-intro.md) -2. [Lifecycle of a Query](./query-lifecycle.md) -3. [Command-Line Interface](./cli.md) -4. [Rest Interface](./rest.md) diff --git a/docs/interfaces/cli.md b/docs/interfaces/cli.md deleted file mode 100644 index d447abe82f17..000000000000 --- a/docs/interfaces/cli.md +++ /dev/null @@ -1,136 +0,0 @@ - - -# Command-Line Interface - -This document describes how to create a commmand-line interface (CLI) for an [**application**](../basics/app-anatomy.md). A separate document for implementing a CLI for an SDK [**module**](../building-modules/intro.md) can be found [here](#../building-modules/module-interfaces.md#cli). {synopsis} - -## Pre-requisite Readings - -* [Lifecycle of a Query](./query-lifecycle.md) {prereq} - -## Command-Line Interface - -One of the main entrypoints of an application is the command-line interface. This entrypoint is created via a `main.go` file which compiles to a binary, conventionally placed in the application's `./cmd/cli` folder. The CLI for an application is typically be referred to as the name of the application suffixed with `-cli`, e.g. `appcli`. Here is where the interfaces docs lie in the directory from the [nameservice tutorial](https://cosmos.network/docs/tutorial). - -### Example Command - -There is no set way to create a CLI, but SDK modules typically use the [Cobra Library](https://github.com/spf13/cobra). Building a CLI with Cobra entails defining commands, arguments, and flags. [**Commands**](#commands) understand the actions users wish to take, such as `tx` for creating a transaction and `query` for querying the application. Each command can also have nested subcommands, necessary for naming the specific transaction type. Users also supply **Arguments**, such as account numbers to send coins to, and [**Flags**](#flags) to modify various aspects of the commands, such as gas prices or which node to broadcast to. - -Here is an example of a command a user might enter to interact with the nameservice CLI `nscli` in order to buy a name: - -```bash -nscli tx nameservice buy-name --gas auto --gas-prices -``` - -The first four strings specify the command: - -- The root command for the entire application `nscli`. -- The subcommand `tx`, which contains all commands that let users create transactions. -- The subcommand `nameservice` to indicate which module to route the command to (`nameservice` module in this case). -- The type of transaction `buy-name`. - -The next two strings are arguments: the `name` the user wishes to buy and the `amount` they want to pay for it. Finally, the last few strings of the command are flags to indicate how much the user is willing to pay in fees (calculated using the amount of gas used to execute the transaction and the gas prices provided by the user). - -The CLI interacts with a [node](../core/node.md) (running `nsd`) to handle this command. The interface itself is defined in a `main.go` file. - -### Building the CLI - -The `main.go` file needs to have a `main()` function that does the following to run the command-line interface: - -* **Instantiate the `codec`** by calling the application's `MakeCodec()` function. The [`codec`](../core/encoding.md) is used to code and encode data structures for the application - stores can only persist `[]byte`s so the developer must define a serialization format for their data structures or use the default, [Amino](../core/encoding.md#amino). -* **Configurations** are set by reading in configuration files (e.g. the sdk config file). -* **Create the root command** to which all the application commands will be added as subcommands and add any required flags to it, such as `--chain-id`. -* **Add subcommands** for all the possible user interactions, including [transaction commands](#transaction-commands) and [query commands](#query-commands). -* **Create an Executor** and [execute](https://godoc.org/github.com/spf13/cobra#Command.Execute) the root command. - -See an example of `main()` function from the `nameservice` application: - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L23-L66 - -The rest of the document will detail what needs to be implemented for each step and include smaller portions of code from the nameservice CLI `main.go` file. - -## Adding Commands to the CLI - -Every application CLI first constructs a root command, then adds functionality by aggregating subcommands (often with further nested subcommands) using `rootCmd.AddCommand()`. The bulk of an application's unique capabilities lies in its transaction and query commands, called `TxCmd` and `QueryCmd` respectively. - -### Root Command - -The root command (called `rootCmd`) is what the user first types into the command line to indicate which application they wish to interact with. The string used to invoke the command (the "Use" field) is typically the name of the application suffixed with `-cli`, e.g. `appcli`. The root command typically includes the following commands to support basic functionality in the application. - -* **Status** command from the SDK rpc client tools, which prints information about the status of the connected [`Node`](../core/node.md). The Status of a node includes `NodeInfo`,`SyncInfo` and `ValidatorInfo`. -* **Config** [command](https://github.com/cosmos/cosmos-sdk/blob/master/client/config.go) from the SDK client tools, which allows the user to edit a `config.toml` file that sets values for [flags](#flags) such as `--chain-id` and which `--node` they wish to connect to. -The `config` command can be invoked by typing `appcli config` with optional arguments ` [value]` and a `--get` flag to query configurations or `--home` flag to create a new configuration. -* **Keys** [commands](https://github.com/cosmos/cosmos-sdk/blob/master/client/keys) from the SDK client tools, which includes a collection of subcommands for using the key functions in the SDK crypto tools, including adding a new key and saving it to disk, listing all public keys stored in the key manager, and deleting a key. For example, users can type `appcli keys add ` to add a new key and save an encrypted copy to disk, using the flag `--recover` to recover a private key from a seed phrase or the flag `--multisig` to group multiple keys together to create a multisig key. For full details on the `add` key command, see the code [here](https://github.com/cosmos/cosmos-sdk/blob/master/client/keys/add.go). For more details about usage of `--keyring-backend` for storage of key credentials look at the [keyring docs](/keyring.md). -* [**Transaction**](#transaction-commands) commands. -* [**Query**](#query-commands) commands. - -Next is an example `main()` function from the `nameservice` application. It instantiates the root command, adds a [*persistent* flag](#flags) and `PreRun` function to be run before every execution, and adds all of the necessary subcommands. - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L23-L66 - -The root-level `status`, `config`, and `keys` subcommands are common across most applications and do not interact with application state. The bulk of an application's functionality - what users can actually *do* with it - is enabled by its transaction commands. - -### Transaction Commands - -[Transactions](#./transactions.md) are objects wrapping [messages](../building-modules/messages-and-queries.md#messages) that trigger state changes. To enable the creation of transactions using the CLI interface, a function `txCmd` is generally added to the `rootCmd`: - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L51 - -This `txCmd` function adds all the transaction available to end-users for the application. This typically includes: - -* **Sign command** from the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/master/x/auth/spec) module that signs messages in a transaction. To enable multisig, add the `auth` module's `MultiSign` command. Since every transaction requires some sort of signature in order to be valid, thithe signing command is necessary for every application. -* **Broadcast command** from the SDK client tools, to broadcast transactions. -* **Send command** from the [`bank`](https://github.com/cosmos/cosmos-sdk/tree/master/x/bank/spec) module, which is a transaction that allows accounts to send coins to one another, including gas and fees for transactions. -* **All [module transaction commands](../building-modules/module-interfaces.md#transaction-commands)** the application is dependent on, retrieved by using the [basic module manager's](../building-modules/module-manager.md#basic-manager) `AddTxCommands()` function. - -Here is an example of a `txCmd` aggregating these subcommands from the `nameservice` application: - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L97-L118 - - -### Query Commands - -[**Queries**](../building-modules/messages-and-queries.md#queries) are objects that allow users to retrieve information about the application's state. To enable the creation of transactions using the CLI interface, a function `txCmd` is generally added to the `rootCmd`: - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L50 - -This `queryCmd` function adds all the queries available to end-users for the application. This typically includes: - -* **QueryTx** and/or other transaction query commands] from the `auth` module which allow the user to search for a transaction by inputting its hash, a list of tags, or a block height. These queries allow users to see if transactions have been included in a block. -* **Account command** from the `auth` module, which displays the state (e.g. account balance) of an account given an address. -* **Validator command** from the SDK rpc client tools, which displays the validator set of a given height. -* **Block command** from the SDK rpc client tools, which displays the block data for a given height. -* **All [module query commands](../building-modules/module-interfaces.md#query-commands)** the application is dependent on, retrieved by using the [basic module manager's](../building-modules/module-manager.md#basic-manager) `AddQueryCommands()` function. - -Here is an example of a `queryCmd` aggregating subcommands from the `nameservice` application: - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L74-L95 - -## Flags - -Flags are used to modify commands; developers can include them in a `flags.go` file with their CLI. Users can explicitly include them in commands or pre-configure them by entering a command in the format `appcli config ` into their command line. Commonly pre-configured flags include the `--node` to connect to and `--chain-id` of the blockchain the user wishes to interact with. - -A *persistent* flag (as opposed to a _local_ flag) added to a command transcends all of its children: subcommands will inherit the configured values for these flags. Additionally, all flags have default values when they are added to commands; some toggle an option off but others are empty values that the user needs to override to create valid commands. A flag can be explicitly marked as _required_ so that an error is automatically thrown if the user does not provide a value, but it is also acceptable to handle unexpected missing flags differently. - -Flags are added to commands directly (generally in the [module's CLI file](../building-modules/module-interfaces.md#flags) where module commands are defined) and no flag except for the `rootCmd` persistent flags has to be added at application level. It is common to add a _persistent_ flag for `--chain-id`, the unique identifier of the blockchain the application pertains to, to the root command. Adding this flag can be done in the `main()` function. Adding this flag makes sense as the chain ID should not be changing across commands in this application CLI. Here is an example from the `nameservice` application: - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L41 - - -## Configurations - -The last function to define in `main.go` is `initConfig`, which does exactly what it sounds like - initialize configurations. To call this function, set it as a `PersistentPreRunE` function for the root command, so that it always executes before the main execution of the root command and any of its subcommands. `initConfig()` does the following: - -1. Read in the `config.toml` file. This same file is edited through `config` commands. -2. Use the [Viper](https://github.com/spf13/viper) to read in configurations from the file and set them. -3. Set any persistent flags defined by the user: `--chain-id`, `--encoding`, `--output`, etc. - -Here is an example of an `initConfig()` function from the [nameservice tutorial CLI](https://cosmos.network/docs/tutorial/entrypoint.html#nscli): - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L120-L141 - -And an example of how to add `initConfig` as a `PersistentPreRunE` to the root command: - -+++ https://github.com/cosmos/sdk-tutorials/blob/86a27321cf89cc637581762e953d0c07f8c78ece/nameservice/cmd/nscli/main.go#L42-L44 diff --git a/docs/interfaces/interfaces-intro.md b/docs/interfaces/interfaces-intro.md deleted file mode 100644 index fa2c8bf8d496..000000000000 --- a/docs/interfaces/interfaces-intro.md +++ /dev/null @@ -1,44 +0,0 @@ - - -# Interfaces - -Typically, SDK applications include interfaces to let end-users interact with the application. This document introduces the different types of interfaces for SDK applications. {synopsis} - -## Pre-requisite Readings - -* [Anatomy of an SDK Application](../basics/app-anatomy.md) {prereq} -* [Lifecycle of a Transaction](../basics/tx-lifecycle.md) {prereq} - -## Types of Application Interfaces - -SDK applications generally have a Command-Line Interface (CLI) and REST Interface to support interactions with a [full-node](../core/node.md). The SDK is opinionated about how to create these two interfaces; all modules specify [Cobra commands](https://github.com/spf13/cobra) and register routes using [Gorilla Mux routers](https://github.com/gorilla/mux). The CLI and REST Interface are conventionally defined in the application `./app/cmd/cli` folder. - -## Module vs Application Interfaces - -The process of creating an application interface is distinct from creating a [module interface](../building-modules/module-interfaces.md), though the two are closely intertwined. As expected, module interfaces handle the bulk of the underlying logic, defining ways for end-users to create [messages](../building-modules/messages-and-queries.md#messages) and [queries](../building-modules/messages-and-queries.md#queries) to the subset of application state within their scope. On the other hand, application interfaces aggregate module-level interfaces in order to route `messages` and `queries` to the appropriate modules. Application interfaces also handle root-level responsibilities such as signing and broadcasting [transactions](../core/transactions.md) that wrap messages. - -### Module Developer Responsibilities - -With regards to interfaces, module developers need to include the following definitions: - -* **CLI commands:** Specifically, [Transaction commands](../building-modules/module-interfaces.md#transaction-commands) and [Query commands](../building-modules/module-interfaces.md#query-commands). These are commands that users will invoke when interacting with the application to create transactions and queries. For example, if an application enables sending coins through the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/master/x/auth/spec) module, users will create `tx auth send` transactions. -* **Request Handlers:** Also categorized into Transaction and Query requests. Transactions will require HTTP [Request Types](../building-modules/module-interfaces.md#request-types) in addition to [Request Handlers](../building-modules/module-interfaces.md#request-handlers) in order to encapsulate all of the user's options (e.g. gas prices). -* **REST Routes:** Given a router, the module interface registers paths with the aforementioned [Request Handlers](../building-modules/module-interfaces.md#request-handlers) for each type of request. - -Module interfaces are designed to be generic. Both commands and request types include required user input (through flags or request body) which are different for each application. This section of documents will only detail application interfaces; to read about how to build module interfaces, click [here](../building-modules/module-interfaces.md). - -### Application Developer Responsibilities - -With regards to interfaces, application developers need to include: - -* **CLI Root Command:** The [root command](./cli.md#root-command) adds subcommands to include all of the functionality for the application, mainly module [transaction](./cli.md#transaction-commands) and [query](./cli.md#query-commands) commands from the application's module(s). -* **App Configurations:** All application-specific values are the responsibility of the application developer, including the [`codec`](../core/encoding.md) used to marshal requests before relaying them to a node. -* **User Configurations:** Some values are specific to the user, such as the user's address and which node they are connected to. The CLI has a [configurations](./cli.md#configurations) function to set these values. -* **RegisterRoutes Function:** [Routes](./rest.md#registerroutes) must be registered and passed to an instantiated [REST server](./rest.md#rest-server) so that it knows how to route requests for this particular application. - - -## Next {hide} - -Read about the [Lifecycle of a Query](./query-lifecycle.md) {hide} diff --git a/docs/interfaces/lite/getting_started.md b/docs/interfaces/lite/getting_started.md deleted file mode 100644 index 23c92821b4e3..000000000000 --- a/docs/interfaces/lite/getting_started.md +++ /dev/null @@ -1,20 +0,0 @@ -# Getting Started - -To start a REST server, we need to specify the following parameters: - -| Parameter | Type | Default | Required | Description | -| ----------- | --------- | ----------------------- | -------- | ---------------------------------------------------- | -| chain-id | string | null | true | chain id of the full node to connect | -| node | URL | "tcp://localhost:46657" | true | address of the full node to connect | -| laddr | URL | "tcp://localhost:1317" | true | address to run the rest server on | -| trust-store | DIRECTORY | "$HOME/.lcd" | false | directory for save checkpoints and validator sets | - -For example: - -```bash -gaiacli rest-server --chain-id=test \ - --laddr=tcp://localhost:1317 \ - --node tcp://localhost:26657 \ -``` - -For more information about the Gaia-Lite RPC, see the [swagger documentation](https://cosmos.network/rpc/) diff --git a/docs/interfaces/lite/pics/C2H.png b/docs/interfaces/lite/pics/C2H.png deleted file mode 100644 index 49f7e07f320e..000000000000 Binary files a/docs/interfaces/lite/pics/C2H.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/H2C.png b/docs/interfaces/lite/pics/H2C.png deleted file mode 100644 index 027eafcde1f4..000000000000 Binary files a/docs/interfaces/lite/pics/H2C.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/MA.png b/docs/interfaces/lite/pics/MA.png deleted file mode 100644 index ae3823962855..000000000000 Binary files a/docs/interfaces/lite/pics/MA.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/absence1.png b/docs/interfaces/lite/pics/absence1.png deleted file mode 100755 index 70b4c9e8d227..000000000000 Binary files a/docs/interfaces/lite/pics/absence1.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/absence2.png b/docs/interfaces/lite/pics/absence2.png deleted file mode 100755 index 6ce5bcde31de..000000000000 Binary files a/docs/interfaces/lite/pics/absence2.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/absence3.png b/docs/interfaces/lite/pics/absence3.png deleted file mode 100755 index d3afdc2c5f7f..000000000000 Binary files a/docs/interfaces/lite/pics/absence3.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/architecture.png b/docs/interfaces/lite/pics/architecture.png deleted file mode 100644 index 741a90c26775..000000000000 Binary files a/docs/interfaces/lite/pics/architecture.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/changeProcess.png b/docs/interfaces/lite/pics/changeProcess.png deleted file mode 100755 index 1771c239b92f..000000000000 Binary files a/docs/interfaces/lite/pics/changeProcess.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/commitValidation.png b/docs/interfaces/lite/pics/commitValidation.png deleted file mode 100755 index 89985bcc12ea..000000000000 Binary files a/docs/interfaces/lite/pics/commitValidation.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/create-account.png b/docs/interfaces/lite/pics/create-account.png deleted file mode 100644 index 35a249b97d9e..000000000000 Binary files a/docs/interfaces/lite/pics/create-account.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/deposit.png b/docs/interfaces/lite/pics/deposit.png deleted file mode 100644 index 1fb3acdc5aa2..000000000000 Binary files a/docs/interfaces/lite/pics/deposit.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/existProof.png b/docs/interfaces/lite/pics/existProof.png deleted file mode 100755 index ee5de0851bf1..000000000000 Binary files a/docs/interfaces/lite/pics/existProof.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/high-level.png b/docs/interfaces/lite/pics/high-level.png deleted file mode 100644 index 73a14e0d73db..000000000000 Binary files a/docs/interfaces/lite/pics/high-level.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/light-client-architecture.png b/docs/interfaces/lite/pics/light-client-architecture.png deleted file mode 100644 index df44aeef5055..000000000000 Binary files a/docs/interfaces/lite/pics/light-client-architecture.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/loadbalanceDiagram.png b/docs/interfaces/lite/pics/loadbalanceDiagram.png deleted file mode 100644 index 56956ee9d8ec..000000000000 Binary files a/docs/interfaces/lite/pics/loadbalanceDiagram.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/simpleMerkleTree.png b/docs/interfaces/lite/pics/simpleMerkleTree.png deleted file mode 100755 index 5c4e2b76e4ac..000000000000 Binary files a/docs/interfaces/lite/pics/simpleMerkleTree.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/substoreProof.png b/docs/interfaces/lite/pics/substoreProof.png deleted file mode 100755 index 90dadaef37af..000000000000 Binary files a/docs/interfaces/lite/pics/substoreProof.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/transfer-tokens.png b/docs/interfaces/lite/pics/transfer-tokens.png deleted file mode 100644 index c6635a30c8b5..000000000000 Binary files a/docs/interfaces/lite/pics/transfer-tokens.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/transfer.png b/docs/interfaces/lite/pics/transfer.png deleted file mode 100644 index 7642e996cf32..000000000000 Binary files a/docs/interfaces/lite/pics/transfer.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/trustPropagate.png b/docs/interfaces/lite/pics/trustPropagate.png deleted file mode 100755 index a743cfc72147..000000000000 Binary files a/docs/interfaces/lite/pics/trustPropagate.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/updateValidatorToHeight.png b/docs/interfaces/lite/pics/updateValidatorToHeight.png deleted file mode 100755 index b2c9349b6b45..000000000000 Binary files a/docs/interfaces/lite/pics/updateValidatorToHeight.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/validatorSetChange.png b/docs/interfaces/lite/pics/validatorSetChange.png deleted file mode 100755 index 16dbf39be7ea..000000000000 Binary files a/docs/interfaces/lite/pics/validatorSetChange.png and /dev/null differ diff --git a/docs/interfaces/lite/pics/withdraw.png b/docs/interfaces/lite/pics/withdraw.png deleted file mode 100644 index 2249b5e34839..000000000000 Binary files a/docs/interfaces/lite/pics/withdraw.png and /dev/null differ diff --git a/docs/interfaces/lite/readme.md b/docs/interfaces/lite/readme.md deleted file mode 100644 index e31ccd0752e5..000000000000 --- a/docs/interfaces/lite/readme.md +++ /dev/null @@ -1,91 +0,0 @@ ---- -parent: - order: false ---- - -# Light Client Overview - -**See the Cosmos SDK Light Client RPC documentation [here](https://cosmos.network/rpc/)** - -## Introduction - -A light client allows clients, such as mobile phones, to receive proofs of the state of the -blockchain from any full node. Light clients do not have to trust any full node, since they are able -to verify any proof they receive. - -A light client can provide the same security as a full node with minimal requirements for -bandwidth, computing and storage resource. It can also provide modular functionality -according to users' configuration. These features allow developers to build secure, efficient, -and usable mobile apps, websites, and other applications without deploying or -maintaining any full blockchain nodes. - -### What is a Light Client? - -The Cosmos SDK Light Client (Gaia-lite) is split into two separate components. The first component is generic for -any Tendermint-based application. It handles the security and connectivity aspects of following the header -chain and verify proofs from full nodes against a locally trusted validator set. Furthermore, it exposes the same -API as any Tendermint Core node. The second component is specific for the Cosmos Hub (`gaiad`). It works as a query -endpoint and exposes the application specific functionality, which can be arbitrary. All queries against the -application state must go through the query endpoint. The advantage of the query endpoint is that it can verify -the proofs that the application returns. - -### High-Level Architecture - -An application developer that wants to build a third party client application for the Cosmos Hub (or any -other zone) should build it against its canonical API. That API is a combination of multiple parts. -All zones have to expose ICS0 (TendermintAPI). Beyond that any zone is free to choose any -combination of module APIs, depending on which modules the state machine uses. The Cosmos Hub will -initially support [ICS0](https://cosmos.network/rpc/#/ICS0) (TendermintAPI), [ICS1](https://cosmos.network/rpc/#/ICS1) (KeyAPI), [ICS20](https://cosmos.network/rpc/#/ICS20) (TokenAPI), [ICS21](https://cosmos.network/rpc/#/ICS21) (StakingAPI), -[ICS22](https://cosmos.network/rpc/#/ICS22) (GovernanceAPI) and [ICS23](https://cosmos.network/rpc/#/ICS23) (SlashingAPI). - -![high-level](./pics/high-level.png) - -All applications are expected to run only against Gaia-lite. Gaia-lite is the only piece of software -that offers stability guarantees around the zone API. - -### Comparison - -A full node of ABCI is different from a light client in the following ways: - -|| Full Node | Gaia-lite | Description| -|-| ------------- | ----- | -------------- | -| Execute and verify transactions|Yes|No|A full node will execute and verify all transactions while Gaia-lite won't.| -| Verify and save blocks|Yes|No|A full node will verify and save all blocks while Gaia-lite won't.| -| Consensus participation|Yes|No|Only when a full node is a validator will it participate in consensus. Lite nodes never participate in consensus.| -| Bandwidth cost|High|Low|A full node will receive all blocks. If bandwidth is limited, it will fall behind the main network. What's more, if it happens to be a validator, it will slow down the consensus process. Light clients require little bandwidth, only when serving local requests.| -| Computing resources|High|Low|A full node will execute all transactions and verify all blocks, which requires considerable computing resources.| -| Storage resources|High|Low|A full node will save all blocks and ABCI states. Gaia-lite just saves validator sets and some checkpoints.| -| Power consumption|High|Low|Full nodes must be deployed on machines which have high performance and will be running all the time. Gaia-lite can be deployed on the same machines as users' applications, or on independent machines but with lower performance. Light clients can be shut down anytime when necessary. Gaia-lite consumes very little power, so even mobile devices can meet the power requirements.| -| Provide APIs|All cosmos APIs|Modular APIs|A full node supports all Cosmos APIs. Gaia-lite provides modular APIs according to users' configuration.| -| Secuity level| High|High|A full node will verify all transactions and blocks by itself. A light client can't do this, but it can query data from other full nodes and verify the data independently. Therefore, both full nodes and light clients don't need to trust any third nodes and can achieve high security.| - -According to the above table, Gaia-lite can meet many users' functionality and security requirements, but require little bandwidth, computing, storage, and power. - -## Achieving Security - -### Trusted Validator Set - -The base design philosophy of Gaia-lite follows two rules: - -1. **Doesn't trust any blockchain nodes, including validator nodes and other full nodes** -2. **Only trusts the whole validator set** - -The original trusted validator set should be prepositioned into its trust store. Usually this -validator set comes from a genesis file. During runtime, if Gaia-lite detects a different validator set, -it will verify it and save the new validated validator set to the trust store. - -![validator-set-change](./pics/validatorSetChange.png) - -### Trust Propagation - -From the above section, we come to know how to get a trusted validator set and how lcd keeps track of -validator set evolution. The validator set is the foundation of trust, and the trust can propagate to -other blockchain data, such as blocks and transactions. The propagation architecture is shown as - -follows: - -![change-process](./pics/trustPropagate.png) - -In general, with a trusted validator set, a light client can verify each block commit which contains all pre-commit -data and block header data. Then the block hash, data hash and appHash are trusted. Based on this -and merkle proof, all transactions data and ABCI states can be verified too. diff --git a/docs/interfaces/lite/specification.md b/docs/interfaces/lite/specification.md deleted file mode 100644 index 4ce849acaa1d..000000000000 --- a/docs/interfaces/lite/specification.md +++ /dev/null @@ -1,209 +0,0 @@ -# Specifications - -This specification describes how to implement the LCD. LCD supports modular APIs. Currently, only -ICS0 (TendermintAPI), ICS1 (Key API) and ICS20 (Token API) are supported. Later, if necessary, more -APIs can be included. - -## Build and Verify Proof of ABCI States - -As we all know, storage of cosmos-sdk based application contains multi-substores. Each substore is -implemented by a IAVL store. These substores are organized by simple Merkle tree. To build the tree, -we need to extract name, height and store root hash from these substores to build a set of simple -Merkle leaf nodes, then calculate hash from leaf nodes to root. The root hash of the simple Merkle -tree is the AppHash which will be included in block header. - -![Simple Merkle Tree](./pics/simpleMerkleTree.png) - -As we have discussed in LCD trust-propagation, -the AppHash can be verified by checking voting power against a trusted validator set. Here we just -need to build proof from ABCI state to AppHash. The proof contains two parts: - -* IAVL proof -* Substore to AppHash proof - -### IAVL Proof - -The proof has two types: existence proof and absence proof. If the query key exists in the IAVL -store, then it returns key-value and its existence proof. On the other hand, if the key doesn't -exist, then it only returns absence proof which can demonstrate the key definitely doesn't exist. - -### IAVL Existence Proof - -```go -type CommitID struct { - Version int64 - Hash []byte -} - -type storeCore struct { - CommitID CommitID -} - -type MultiStoreCommitID struct { - Name string - Core storeCore -} - -type proofInnerNode struct { - Height int8 - Size int64 - Version int64 - Left []byte - Right []byte -} - -type KeyExistsProof struct { - MultiStoreCommitInfo []MultiStoreCommitID //All substore commitIDs - StoreName string //Current substore name - Height int64 //The commit height of current substore - RootHash cmn.HexBytes //The root hash of this IAVL tree - Version int64 //The version of the key-value in this IAVL tree - InnerNodes []proofInnerNode //The path from to root node to key-value leaf node -} -``` - -The data structure of exist proof is shown as above. The process to build and verify existence proof -is shown as follows: - -![Exist Proof](./pics/existProof.png) - -Steps to build proof: - -* Access the IAVL tree from the root node. -* Record the visited nodes in InnerNodes, -* Once the target leaf node is found, assign leaf node version to proof version -* Assign the current IAVL tree height to proof height -* Assign the current IAVL tree rootHash to proof rootHash -* Assign the current substore name to proof StoreName -* Read multistore commitInfo from db by height and assign it to proof StoreCommitInfo - -Steps to verify proof: - -* Build leaf node with key, value and proof version. -* Calculate leaf node hash -* Assign the hash to the first innerNode's rightHash, then calculate first innerNode hash -* Propagate the hash calculation process. If prior innerNode is the left child of next innerNode, then assign the prior innerNode hash to the left hash of next innerNode. Otherwise, assign the prior innerNode hash to the right hash of next innerNode. -* The hash of last innerNode should be equal to the rootHash of this proof. Otherwise, the proof is invalid. - -### IAVL Absence Proof - -As we all know, all IAVL leaf nodes are sorted by the key of each leaf nodes. So we can calculate -the position of the target key in the whole key set of this IAVL tree. As shown below, we can find -out the left key and the right key. If we can demonstrate that both left key and right key -definitely exist, and they are adjacent nodes. Thus the target key definitely doesn't exist. - -![Absence Proof1](./pics/absence1.png) - -If the target key is larger than the right most leaf node or less than the left most key, then the -target key definitely doesn't exist. - -![Absence Proof2](./pics/absence2.png)![Absence Proof3](./pics/absence3.png) - -```go -type proofLeafNode struct { - KeyBytes cmn.HexBytes - ValueBytes cmn.HexBytes - Version int64 -} - -type pathWithNode struct { - InnerNodes []proofInnerNode - Node proofLeafNode -} - -type KeyAbsentProof struct { - MultiStoreCommitInfo []MultiStoreCommitID - StoreName string - Height int64 - RootHash cmn.HexBytes - Left *pathWithNode // Proof the left key exist - Right *pathWithNode //Proof the right key exist -} -``` - -The above is the data structure of absence proof. Steps to build proof: - -* Access the IAVL tree from the root node. -* Get the deserved index(Marked as INDEX) of the key in whole key set. -* If the returned index equals to 0, the right index should be 0 and left node doesn't exist -* If the returned index equals to the size of the whole key set, the left node index should be INDEX-1 and the right node doesn't exist. -* Otherwise, the right node index should be INDEX and the left node index should be INDEX-1 -* Assign the current IAVL tree height to proof height -* Assign the current IAVL tree rootHash to proof rootHash -* Assign the current substore name to proof StoreName -* Read multistore commitInfo from db by height and assign it to proof StoreCommitInfo - -Steps to verify proof: - -* If only right node exist, verify its exist proof and verify if it is the left most node -* If only left node exist, verify its exist proof and verify if it is the right most node. -* If both right node and left node exist, verify if they are adjacent. - -### Substores to AppHash Proof - -After verify the IAVL proof, then we can start to verify substore proof against AppHash. Firstly, -iterate MultiStoreCommitInfo and find the substore commitID by proof StoreName. Verify if yhe Hash -in commitID equals to proof RootHash. If not, the proof is invalid. Then sort the substore -commitInfo array by the hash of substore name. Finally, build the simple Merkle tree with all -substore commitInfo array and verify if the Merkle root hash equal to appHash. - -![substore proof](./pics/substoreProof.png) - -```go -func SimpleHashFromTwoHashes(left []byte, right []byte) []byte { - var hasher = ripemd160.New() - - err := encodeByteSlice(hasher, left) - if err != nil { - panic(err) - } - - err = encodeByteSlice(hasher, right) - if err != nil { - panic(err) - } - - return hasher.Sum(nil) -} - -func SimpleHashFromHashes(hashes [][]byte) []byte { - // Recursive impl. - switch len(hashes) { - case 0: - return nil - case 1: - return hashes[0] - default: - left := SimpleHashFromHashes(hashes[:(len(hashes)+1)/2]) - right := SimpleHashFromHashes(hashes[(len(hashes)+1)/2:]) - return SimpleHashFromTwoHashes(left, right) - } -} -``` - -## Verify block header against validator set - -Above sections refer appHash frequently. But where does the trusted appHash come from? Actually, -the appHash exist in block header, next we need to verify blocks header at specific height against -LCD trusted validator set. The validation flow is shown as follows: - -![commit verification](./pics/commitValidation.png) - -When the trusted validator set doesn't match the block header, we need to try to update our -validator set to the height of this block. LCD has a rule that each validator set change should not -affect more than 1/3 voting power. Compare with the trusted validator set, if the voting power of -target validator set changes more than 1/3. We have to verify if there are hidden validator set -changes before the target validator set. Only when all validator set changes obey this rule, can our -validator set update be accomplished. - -For instance: - -![Update validator set to height](./pics/updateValidatorToHeight.png) - -* Update to 10000, tooMuchChangeErr -* Update to 5050, tooMuchChangeErr -* Update to 2575, Success -* Update to 5050, Success -* Update to 10000,tooMuchChangeErr -* Update to 7525, Success -* Update to 10000, Success diff --git a/docs/interfaces/query-lifecycle.md b/docs/interfaces/query-lifecycle.md deleted file mode 100644 index ae4a2e800b50..000000000000 --- a/docs/interfaces/query-lifecycle.md +++ /dev/null @@ -1,162 +0,0 @@ - - -# Query Lifecycle - -This document describes the lifecycle of a query in a SDK application, from the user interface to application stores and back. The query will be referred to as `Query`. {synopsis} - -## Pre-requisite Readings - -* [Introduction to Interfaces](./interfaces-intro.md) {prereq} - -## Query Creation - -A [**query**](../building-modules/messages-and-queries.md#queries) is a request for information made by end-users of applications through an interface and processed by a full-node. Users can query information about the network, the application itself, and application state directly from the application's stores or modules. Note that queries are different from [transactions](../core/transactions.md) (view the lifecycle [here](../basics/tx-lifecycle.md)), particularly in that they do not require consensus to be processed (as they do not trigger state-transitions); they can be fully handled by one full-node. - -For the purpose of explaining the query lifecycle, let's say `Query` is requesting a list of delegations made by a certain delegator address in the application called `app`. As to be expected, the [`staking`](https://github.com/cosmos/cosmos-sdk/tree/master/x/staking/spec) module handles this query. But first, there are a few ways `Query` can be created by users. - -### CLI - -The main interface for an application is the command-line interface. Users connect to a full-node and run the CLI directly from their machines - the CLI interacts directly with the full-node. To create `Query` from their terminal, users type the following command: - -```bash -appcli query staking delegations -``` - -This query command was defined by the [`staking`](https://github.com/cosmos/cosmos-sdk/tree/master/x/staking/spec) module developer and added to the list of subcommands by the application developer when creating the CLI. The code for this particular command is the following: - -+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/x/staking/client/cli/query.go#L250-L293 - -Note that the general format is as follows: - -```bash -appcli query [moduleName] [command] --flag -``` - -To provide values such as `--node` (the full-node the CLI connects to), the user can use the `config` command to set themn or provide them as flags. - -The CLI understands a specific set of commands, defined in a hierarchical structure by the application developer: from the [root command](./cli.md#root-command) (`appcli`), the type of command (`query`), the module that contains the command (`staking`), and command itself (`delegations`). Thus, the CLI knows exactly which module handles this command and directly passes the call there. - -### REST - -Another interface through which users can make queries is through HTTP Requests to a [REST server](./rest.md#rest-server). The REST server contains, among other things, a [`Context`](#context) and [mux](./rest.md#gorilla-mux) router. The request looks like this: - -```bash -GET http://localhost:{PORT}/staking/delegators/{delegatorAddr}/delegations -``` - -To provide values such as `--node` (the full-node the CLI connects to) that are required by [`baseReq`](../building-modules/module-interfaces.md#basereq), the user must configure their local REST server with the values or provide them in the request body. - -The router automatically routes the `Query` HTTP request to the staking module `delegatorDelegationsHandlerFn()` function. - -+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/x/staking/client/rest/query.go#L103-L106 - -Since this function is defined within the module and thus has no inherent knowledge of the application `Query` belongs to, it takes in the application `codec` and `Context` as parameters. - -To summarize, when users interact with the interfaces, they create a CLI command or HTTP request. `Query` now exists in one of these two forms, but needs to be transformed into an object understood by a full-node. - -## Query Preparation - -The interactions from the users' perspective are a bit different, but the underlying functions are almost identical because they are implementations of the same command defined by the module developer. This step of processing happens within the CLI or REST server and heavily involves a `Context`. - -### Context - -The first thing that is created in the execution of a CLI command is a `Context`, while the REST Server directly provides a `Context` for the REST Request handler. A [Context](../core/context.md) is an immutable object that stores all the data needed to process a request on the user side. In particular, a `Context` stores the following: - -* **Codec**: The [encoder/decoder](../core/encoding.md) used by the application, used to marshal the parameters and query before making the Tendermint RPC request and unmarshal the returned response into a JSON object. -* **Account Decoder**: The account decoder from the [`auth`](https://github.com/cosmos/cosmos-sdk/tree/master/x/auth/spec) module, which translates `[]byte`s into accounts. -* **RPC Client**: The Tendermint RPC Client, or node, to which the request will be relayed to. -* **Keybase**: A [Key Manager](../basics/accounts.md#keybase) used to sign transactions and handle other operations with keys. -* **Output Writer**: A [Writer](https://golang.org/pkg/io/#Writer) used to output the response. -* **Configurations**: The flags configured by the user for this command, including `--height`, specifying the height of the blockchain to query and `--indent`, which indicates to add an indent to the JSON response. - -The `Context` also contains various functions such as `Query()` which retrieves the RPC Client and makes an ABCI call to relay a query to a full-node. - -+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/client/context/context.go#L23-L47 - -The `Context`'s primary role is to store data used during interactions with the end-user and provide methods to interact with this data - it is used before and after the query is processed by the full-node. Specifically, in handling `Query`, the `Context` is utilized to encode the query parameters, retrieve the full-node, and write the output. Prior to being relayed to a full-node, the query needs to be encoded into a `[]byte` form, as full-nodes are application-agnostic and do not understand specific types. The full-node (RPC Client) itself is retrieved using the `Context`, which knows which node the user CLI is connected to. The query is relayed to this full-node to be processed. Finally, the `Context` contains a `Writer` to write output when the response is returned. These steps are further described in later sections. - -### Arguments and Route Creation - -At this point in the lifecycle, the user has created a CLI command or HTTP Request with all of the data they wish to include in their `Query`. A `Context` exists to assist in the rest of the `Query`'s journey. Now, the next step is to parse the command or request, extract the arguments, create a `queryRoute`, and encode everything. These steps all happen on the user side within the interface they are interacting with. - -#### Encoding - -In this case, `Query` contains an [address](../basics/accounts.md#addresses) `delegatorAddress` as its only argument. However, the request can only contain `[]byte`s, as it will be relayed to a consensus engine (e.g. Tendermint Core) of a full-node that has no inherent knowledge of the application types. Thus, the `codec` of `Context` is used to marshal the address. - -Here is what the code looks like for the CLI command: - -```go -delAddr, err := sdk.AccAddressFromBech32(args[0]) -bz, err := cdc.MarshalJSON(types.NewQueryDelegatorParams(delAddr)) -``` - -Here is what the code looks like for the HTTP Request: - -```go -vars := mux.Vars(r) -bech32delegator := vars["delegatorAddr"] -delegatorAddr, err := sdk.AccAddressFromBech32(bech32delegator) -cliCtx, ok := rest.ParseQueryHeightOrReturnBadRequest(w, cliCtx, r) -if !ok { - return -} -params := types.NewQueryDelegatorParams(delegatorAddr) -``` - -#### Query Route Creation - -Important to note is that there will never be a "query" object created for `Query`; the SDK actually takes a simpler approach. Instead of an object, all the full-node needs to process a query is its `route` which specifies exactly which module to route the query to and the name of this query type. The `route` will be passed to the application `baseapp`, then module, then [querier](../building-modules/query-services.md#legacy-queriers), and each will understand the `route` and pass it to the appropriate next step. [`baseapp`](../core/baseapp.md#query-routing) will understand this query to be a `custom` query in the module `staking`, and the `staking` module querier supports the type `QueryDelegatorDelegations`. Thus, the route will be `"custom/staking/delegatorDelegations"`. - -Here is what the code looks like: - -```go -route := fmt.Sprintf("custom/%s/%s", queryRoute, types.QueryDelegatorDelegations) -``` - -Now, `Query` exists as a set of encoded arguments and a route to a specific module and its query type. It is ready to be relayed to a full-node. - -#### ABCI Query Call - -The `Context` has a `Query()` function used to retrieve the pre-configured node and relay a query to it; the function takes the query `route` and arguments as parameters. It first retrieves the RPC Client (called the [**node**](../core/node.md)) configured by the user to relay this query to, and creates the `ABCIQueryOptions` (parameters formatted for the ABCI call). The node is then used to make the ABCI call, `ABCIQueryWithOptions()`. - -Here is what the code looks like: - -+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/client/context/query.go#L75-L112 - -## RPC - -With a call to `ABCIQueryWithOptions()`, `Query` is received by a [full-node](../core/encoding.md) which will then process the request. Note that, while the RPC is made to the consensus engine (e.g. Tendermint Core) of a full-node, queries are not part of consensus and will not be broadcasted to the rest of the network, as they do not require anything the network needs to agree upon. - -Read more about ABCI Clients and Tendermint RPC in the Tendermint documentation [here](https://tendermint.com/rpc). - -## Application Query Handling - -When a query is received by the full-node after it has been relayed from the underlying consensus engine, it is now being handled within an environment that understands application-specific types and has a copy of the state. [`baseapp`](../core/baseapp.md) implements the ABCI [`Query()`](../core/baseapp.md#query) function and handles four different types of queries: `app`, `store`, `p2p`, and `custom`. The `queryRoute` is parsed such that the first string must be one of the four options, then the rest of the path is parsed within the subroutines handling each type of query. The first three types (`app`, `store`, `p2p`) are purely application-level and thus directly handled by `baseapp` or the stores, but the `custom` query type requires `baseapp` to route the query to a module's [query service](../building-modules/query-services.md). - -Since `Query` is a custom query type from the `staking` module, `baseapp` first parses the path, then uses the `QueryRouter` to retrieve the corresponding querier, and routes the query to the module. The querier is responsible for recognizing this query, retrieving the appropriate values from the application's stores, and returning a response. Read more about query services [here](../building-modules/query-services.md). - -Once a result is received from the querier, `baseapp` begins the process of returning a response to the user. - -## Response - -Since `Query()` is an ABCI function, `baseapp` returns the response as an [`abci.ResponseQuery`](https://tendermint.com/docs/spec/abci/abci.html#messages) type. The `Context` `Query()` routine receives the response and. - -+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/client/context/query.go#L127-L165 - -### CLI Response - -The application [`codec`](../core/encoding.md) is used to unmarshal the response to a JSON and the `Context` prints the output to the command line, applying any configurations such as `--indent`. - -+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/x/staking/client/cli/query.go#L252-L293 - -### REST Response - -The [REST server](./rest.md#rest-server) uses the `Context` to format the response properly, then uses the HTTP package to write the appropriate response or error. - -+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/x/staking/client/rest/utils.go#L115-L148 - -## Next {hide} - -Read about how to build a [Command-Line Interface](./cli.md), or a [REST Interface](./rest.md) {hide} diff --git a/docs/interfaces/rest.md b/docs/interfaces/rest.md deleted file mode 100644 index 00e94b1d9a9d..000000000000 --- a/docs/interfaces/rest.md +++ /dev/null @@ -1,70 +0,0 @@ - - -# REST Interface - -This document describes how to create a REST interface for an SDK **application**. A separate document for creating a [**module**](../building-modules/intro.md) REST interface can be found [here](#../module-interfaces.md#legacy-rest). {synopsis} - -## Pre-requisite Readings - -- [Query Lifecycle](./query-lifecycle.md) {prereq} -- [Application CLI](./cli.md) {prereq} - -## Application REST Interface - -Building the REST Interface for an application is done by [aggregating REST Routes](#registering-routes) defined in the application's modules. This interface is served by a REST Server [REST server](#rest-server), which route requests and output responses in the application itself. The SDK comes with its own REST Server by default. To enable it, the `rest.ServeCommand` command needs to be added as a subcommand of the `rootCmd` in the `main()` function of the [CLI interface](./cli.md): - -```go -rootCmd.AddCommand(rest.ServeCommand(cdc, registerRoutes)) -``` - -Users will then be able to use the application CLI to start a new REST server, a local server through which they can securely interact with the application without downloading the entire state. The command entered by users would look something like this: - -```bash -appcli rest-server --chain-id -``` - -## REST Server - -A REST Server is used to receive and route HTTP Requests, obtain the results from the application, and return a response to the user. The REST Server defined by the SDK `rest` package contains the following: - -- **Router:** A router for HTTP requests. A new router can be instantiated for an application and used to match routes based on path, request method, headers, etc. The SDK uses the [Gorilla Mux Router](https://github.com/gorilla/mux). -- **Context:** A [`Context`](./query-lifecycle.md#context) created for a user interaction. -- **Keybase:** A [Keybase](../basics/accounts.md#keybase) is a key manager. -- **Logger:** A logger from Tendermint `Log`, a log package structured around key-value pairs that allows logging level to be set differently for different keys. The logger takes `Debug()`, `Info()`, and `Error()`s. -- **Listener:** A [listener](https://golang.org/pkg/net/#Listener) from the net package. - -Of the five, the only attribute that application developers need interact with is the `router`: they need to add routes to it so that the REST server can properly handle queries. See the next section for more information on registering routes. - -In order to enable the REST Server in an SDK application, the `rest.ServeCommand` needs to be added to the application's command-line interface. See the [above section](#application-rest-interface) for more information. - -## Registering Routes - -To include routes for each module in an application, the CLI must have some kind of function to register routes in its REST Server. This function is called `RegisterRoutes()`, and is utilized by the `ServeCommand` and must include routes for each of the application's modules. Since each module used by an SDK application implements a [`RegisterRESTRoutes`](../building-modules/module-interfaces.md#legacy-rest) function, application developers simply use the [Module Manager](../building-modules/module-manager.md) to call this function for each module (this is done in the [application's constructor](../basics/app-anatomy.md#constructor-function)). - -At the bare minimum, a `RegisterRoutes()` function should use the SDK client package `RegisterRoutes()` function to be able to route RPC calls, and instruct the application Module Manager to call `RegisterRESTRoutes()` for all of its modules. This is done in the `main.go` file of the CLI (typically located in `./cmd/appcli/main.go`). - -```go -func registerRoutes(rs *rest.RestServer) { - rpc.RegisterRoutes(rs.ClientCtx, rs.Mux) - app.ModuleBasics.RegisterRESTRoutes(rs.ClientCtx, rs.Mux) -} -``` - -This function is specific to the application and passed in to the `ServeCommand`, which should be added to the `rootCmd` as such: - -```go -rootCmd.AddCommand(rest.ServeCommand(cdc, registerRoutes)) -``` - -## Cross-Origin Resource Sharing (CORS) - -[CORS policies](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) are not enabled by default to help with security. If you would like to use the rest-server in a public environment we recommend you provide a reverse proxy, this can be done with [nginx](https://www.nginx.com/). For testing and development purposes there is an `unsafe_cors` flag that can be passed to the cmd to enable accepting cors from everyone. - -```sh -gaiacli rest-server --chain-id=test \ - --laddr=tcp://localhost:1317 \ - --node tcp://localhost:26657 \ - --unsafe-cors -``` diff --git a/docs/run-node/interact-node.md b/docs/run-node/interact-node.md index c7b30c57b073..a07981341fe7 100644 --- a/docs/run-node/interact-node.md +++ b/docs/run-node/interact-node.md @@ -62,7 +62,7 @@ Since the code generation library largely depends on your own tech stack, we wil [grpcurl])https://github.com/fullstorydev/grpcurl is like `curl` but for gRPC. It is also available as a Go library, but we will use it only as a CLI command for debugging and testing purposes. Follow the instructions in the previous link to install it. -Assuming you have a local node running (either a localnet, or connected a live network), you should be able to run the following command to list the Protobuf services available (you can replace `localhost:9000` by the gRPC server endpoint of another node, which is configured under the `grpc.address` field inside `app.toml`): +Assuming you have a local node running (either a localnet, or connected a live network), you should be able to run the following command to list the Protobuf services available (you can replace `localhost:9000` by the gRPC server endpoint of another node, which is configured under the `grpc.address` field inside [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml)): ```bash grpcurl -plaintext localhost:9090 list @@ -217,7 +217,7 @@ curl \ Make sure to replace `localhost:1317` with the REST endpoint of your node, configured under the `api.address` field. -The list of all available REST endpoints is available as a Swagger specification file, it can be viewed at `localhost:1317/swagger`. Make sure that the `api.swagger` field is set to true in your `app.toml` file. +The list of all available REST endpoints is available as a Swagger specification file, it can be viewed at `localhost:1317/swagger`. Make sure that the `api.swagger` field is set to true in your [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml) file. ### Query for historical state using REST @@ -233,6 +233,10 @@ curl \ Assuming the state at that block has not yet been pruned by the node, this query should return a non-empty response. +### Cross-Origin Resource Sharing (CORS) + +[CORS policies](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) are not enabled by default to help with security. If you would like to use the rest-server in a public environment we recommend you provide a reverse proxy, this can be done with [nginx](https://www.nginx.com/). For testing and development purposes there is an `enabled-unsafe-cors` field inside [`app.toml`](../run-node/run-node.md#configuring-the-node-using-apptoml). + ## Next {hide} Sending transactions using gRPC and REST requires some additional steps: generating the transaction, signing it, and finally broadcasting it. Read about [generating and signing transactions](./txs.md). {hide} diff --git a/docs/run-node/run-node.md b/docs/run-node/run-node.md index 393172b9aa0c..aee7d84e1301 100644 --- a/docs/run-node/run-node.md +++ b/docs/run-node/run-node.md @@ -85,6 +85,12 @@ The previous command allow you to run a single node. This is enough for the next The naive way would be to run the same commands again in separate terminal windows. This is possible, however in the SDK, we leverage the power of [Docker Compose](https://docs.docker.com/compose/) to run a localnet. If you need inspiration on how to set up your own localnet with Docker Compose, you can have a look at the SDK's [`docker-compose.yml`](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc3/docker-compose.yml). +## Configuring the Node Using `app.toml` + +The Cosmos SDK automatically generates an `app.toml` file inside `~/.simapp/config`. This file is used to configure your app, such as state pruning strategies, telemetry, gRPC and REST servers configuration, state sync... The file itself is heavily commented, please refer to it directly to tweak your node. + +Make sure to restart your node after modifying `app.toml`. + ## Next {hide} Read about the [Interacting with your Node](./interact-node.md) {hide}