Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: amend docs for 52 changes #21992

Merged
merged 4 commits into from
Oct 1, 2024
Merged

docs: amend docs for 52 changes #21992

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7598925
doc changes
tac0turtle Oct 1, 2024
1a0f825
add mention of recommended state
tac0turtle Oct 1, 2024
18175e6
add in section to keeper on state management
tac0turtle Oct 1, 2024
598fc22
Update docs/learn/advanced/05-encoding.md
tac0turtle Oct 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions docs/build/building-modules/06-beginblock-endblock.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ When needed, `BeginBlocker` and `EndBlocker` are implemented as part of the [`Ha

The actual implementation of `BeginBlocker` and `EndBlocker` in `abci.go` are very similar to that of a [`Msg` service](./03-msg-services.md):

* They generally use the [`keeper`](./06-keeper.md) and [`ctx`](../../learn/advanced/02-context.md) to retrieve information about the latest state.
* They generally use the [`keeper`](./06-keeper.md) and [`ctx`](https://pkg.go.dev/context) to retrieve information about the latest state.
* If needed, they use the `keeper` and `ctx` to trigger state-transitions.
* If needed, they can emit [`events`](../../learn/advanced/08-events.md) via the `environments`'s `EventManager`.

Expand All @@ -35,13 +35,20 @@ It is possible for developers to define the order of execution between the `Begi
See an example implementation of `BeginBlocker` from the `distribution` module:

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/distribution/abci.go#L14-L38
https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/distribution/keeper/abci.go#L13-L40
```

and an example implementation of `EndBlocker` from the `staking` module:
and an example of `EndBlocker` from the `gov` module:

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/abci.go#L22-L27
https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/gov/keeper/abci.go#L22
```

and an example implementation of `EndBlocker` with validator updates from the `staking` module:

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/staking/keeper/abci.go#L12-L17
```


<!-- TODO: leaving this here to update docs with core api changes -->
11 changes: 9 additions & 2 deletions docs/build/building-modules/06-keeper.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,14 @@ type Keeper struct {
For example, here is the type definition of the `keeper` from the `staking` module:

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/staking/keeper/keeper.go#L23-L31
https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/staking/keeper/keeper.go#L54-L115
```

Let us go through the different parameters:

* An expected `keeper` is a `keeper` external to a module that is required by the internal `keeper` of said module. External `keeper`s are listed in the internal `keeper`'s type definition as interfaces. These interfaces are themselves defined in an `expected_keepers.go` file in the root of the module's folder. In this context, interfaces are used to reduce the number of dependencies, as well as to facilitate the maintenance of the module itself.
* `KVStoreService`s grant access to the store(s) of the [multistore](../../learn/advanced/04-store.md) managed by the module. They should always remain unexposed to external modules.
* `cdc` is the [codec](../../learn/advanced/05-encoding.md) used to marshall and unmarshall structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces.
* `cdc` is the [codec](../../learn/advanced/05-encoding.md) used to marshal and unmarshal structs to/from `[]byte`. The `cdc` can be any of `codec.BinaryCodec`, `codec.JSONCodec` or `codec.Codec` based on your requirements. It can be either a proto or amino codec as long as they implement these interfaces.
* The authority listed is a module account or user account that has the right to change module level parameters. Previously this was handled by the param module, which has been deprecated.

Of course, it is possible to define different types of internal `keeper`s for the same module (e.g. a read-only `keeper`). Each type of `keeper` comes with its own constructor function, which is called from the [application's constructor function](../../learn/beginner/00-app-anatomy.md). This is where `keeper`s are instantiated, and where developers make sure to pass correct instances of modules' `keeper`s to other modules that require them.
Expand All @@ -60,3 +60,10 @@ Of course, it is possible to define different types of internal `keeper`s for th
<!-- markdown-link-check-disable -->
State management is recommended to be done via [Collections](../packages/collections)
<!-- The above link is created via the script to generate docs -->

## State Management

In the Cosmos SDK, it is crucial to be methodical and selective when managing state within a module, as improper state management can lead to inefficiency, security risks, and scalability issues. Not all data belongs in the on-chain state; it's important to store only essential blockchain data that needs to be verified by consensus. Storing unnecessary information, especially client-side data, can bloat the state and slow down performance. Instead, developers should focus on using an off-chain database to handle supplementary data, extending the API as needed. This approach minimizes on-chain complexity, optimizes resource usage, and keeps the blockchain state lean and efficient, ensuring scalability and smooth operations.


The Cosmos SDK leverages Protocol Buffers (protobuf) for efficient state management, providing a well-structured, binary encoding format that ensures compatibility and performance across different modules. The SDK’s recommended approach for managing state is through the [collections package](../pacakges/02-collections.md), which simplifies state handling by offering predefined data structures like maps and indexed sets, reducing the complexity of managing raw state data. While users can opt for custom encoding schemes if they need more flexibility or have specialized requirements, they should be aware that such custom implementations may not integrate seamlessly with indexers that decode state data on the fly. This could lead to challenges in data retrieval, querying, and interoperability, making protobuf a safer and more future-proof choice for most use cases.
Comment on lines +66 to +69
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

LGTM: Comprehensive State Management guidance with minor typo

This new section provides excellent guidance on state management in Cosmos SDK. It covers crucial aspects such as:

  • The importance of selective on-chain data storage
  • The use of off-chain databases for supplementary data
  • Leveraging Protocol Buffers for efficient state management
  • Recommending the collections package for simplified state handling

The content is informative and will be valuable for developers working with Cosmos SDK modules.

There's a typo in the link to the collections package documentation. "pacakges" should be "packages".

Please apply this fix:

-The SDK's recommended approach for managing state is through the [collections package](../pacakges/02-collections.md),
+The SDK's recommended approach for managing state is through the [collections package](../packages/02-collections.md),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
In the Cosmos SDK, it is crucial to be methodical and selective when managing state within a module, as improper state management can lead to inefficiency, security risks, and scalability issues. Not all data belongs in the on-chain state; it's important to store only essential blockchain data that needs to be verified by consensus. Storing unnecessary information, especially client-side data, can bloat the state and slow down performance. Instead, developers should focus on using an off-chain database to handle supplementary data, extending the API as needed. This approach minimizes on-chain complexity, optimizes resource usage, and keeps the blockchain state lean and efficient, ensuring scalability and smooth operations.
The Cosmos SDK leverages Protocol Buffers (protobuf) for efficient state management, providing a well-structured, binary encoding format that ensures compatibility and performance across different modules. The SDK’s recommended approach for managing state is through the [collections package](../pacakges/02-collections.md), which simplifies state handling by offering predefined data structures like maps and indexed sets, reducing the complexity of managing raw state data. While users can opt for custom encoding schemes if they need more flexibility or have specialized requirements, they should be aware that such custom implementations may not integrate seamlessly with indexers that decode state data on the fly. This could lead to challenges in data retrieval, querying, and interoperability, making protobuf a safer and more future-proof choice for most use cases.
In the Cosmos SDK, it is crucial to be methodical and selective when managing state within a module, as improper state management can lead to inefficiency, security risks, and scalability issues. Not all data belongs in the on-chain state; it's important to store only essential blockchain data that needs to be verified by consensus. Storing unnecessary information, especially client-side data, can bloat the state and slow down performance. Instead, developers should focus on using an off-chain database to handle supplementary data, extending the API as needed. This approach minimizes on-chain complexity, optimizes resource usage, and keeps the blockchain state lean and efficient, ensuring scalability and smooth operations.
The Cosmos SDK leverages Protocol Buffers (protobuf) for efficient state management, providing a well-structured, binary encoding format that ensures compatibility and performance across different modules. The SDK's recommended approach for managing state is through the [collections package](../packages/02-collections.md), which simplifies state handling by offering predefined data structures like maps and indexed sets, reducing the complexity of managing raw state data. While users can opt for custom encoding schemes if they need more flexibility or have specialized requirements, they should be aware that such custom implementations may not integrate seamlessly with indexers that decode state data on the fly. This could lead to challenges in data retrieval, querying, and interoperability, making protobuf a safer and more future-proof choice for most use cases.

4 changes: 4 additions & 0 deletions docs/build/building-modules/12-errors.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ common or general errors which can be further wrapped to provide additional spec

There are two ways to return errors. You can register custom errors with a codespace that is meant to provide more information to clients and normal go errors. The Cosmos SDK uses a mixture of both.

:::Note
Errors v2 has been created as a zero dependency errors package. GRPC errors and tracing support is removed natively from the errors package. Users are required to wrap stack traces and add tracing information to their errors.
:::

:::Warning
If errors are registered they are part of consensus and cannot be changed in a minor release
:::
Expand Down
44 changes: 4 additions & 40 deletions docs/learn/advanced/05-encoding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ via Protobuf. This means that modules may use Protobuf encoding, but the types m
implement `ProtoMarshaler`. If modules wish to avoid implementing this interface
for their types, this is autogenerated via [buf](https://buf.build/)

If modules use [Collections](../../build/packages/02-collections.md) or [ORM](../../build/packages/03-orm.md), encoding and decoding are handled, marshal and unmarshal should not be handled manually unless for specific cases identified by the developer.
Modules are recommended to use [collections](../../build/pacakges/02-collections.md) for handling encoding and decoding of state. Usage of collections handles marshal and unmarshal for you. By default protobuf is used but other encodings can be used if preferred.
tac0turtle marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Correct typo and improve sentence structure

The new content provides valuable information about using collections for state encoding and decoding. However, there are a few improvements to be made:

  1. There's a typo in the word "pacakges" in the link. It should be "packages".
  2. The sentence structure can be improved for better clarity.

Here's a suggested revision:

-Modules are recommended to use [collections](../../build/pacakges/02-collections.md) for handling encoding and decoding of state. Usage of collections handles marshal and unmarshal for you. By default protobuf is used but other encodings can be used if preferred. 
+Modules are recommended to use [collections](../../build/packages/02-collections.md) for handling encoding and decoding of state. Collections handle marshaling and unmarshaling for you. By default, Protobuf is used, but other encodings can be used if preferred.

This revision corrects the typo, improves sentence structure, and adds proper capitalization for "Protobuf".

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Modules are recommended to use [collections](../../build/pacakges/02-collections.md) for handling encoding and decoding of state. Usage of collections handles marshal and unmarshal for you. By default protobuf is used but other encodings can be used if preferred.
Modules are recommended to use [collections](../../build/packages/02-collections.md) for handling encoding and decoding of state. Collections handle marshaling and unmarshaling for you. By default, Protobuf is used, but other encodings can be used if preferred.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~50-~50: Did you mean: “By default,”?
Context: ... handles marshal and unmarshal for you. By default protobuf is used but other encodings ca...

(BY_DEFAULT_COMMA)

[uncategorized] ~50-~50: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...hal for you. By default protobuf is used but other encodings can be used if preferre...

(COMMA_COMPOUND_SENTENCE_2)

### Gogoproto

Expand Down Expand Up @@ -78,15 +78,15 @@ the consensus engine accepts only transactions in the form of raw bytes.
* The `TxDecoder` object performs the decoding.

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/types/tx_msg.go#L91-L95
https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/types/tx_msg.go#L91-L95
```

A standard implementation of both these objects can be found in the [`auth/tx` module](https://docs.cosmos.network/main/build/modules/auth#transactions):

https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/decoder.go
https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/auth/tx/decoder.go

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/x/auth/tx/encoder.go
https://github.com/cosmos/cosmos-sdk/blob/v0.52.0-beta.1/x/auth/tx/encoder.go
```

See [ADR-020](../../architecture/adr-020-protobuf-transaction-encoding.md) for details of how a transaction is encoded.
Expand Down Expand Up @@ -245,39 +245,3 @@ Protobuf types can be defined to encode:

We encourage developers to follow industry guidelines: [Protocol Buffers style guide](https://developers.google.com/protocol-buffers/docs/style)
and [Buf](https://buf.build/docs/style-guide), see more details in [ADR 023](../../architecture/adr-023-protobuf-naming.md)

### How to update modules to protobuf encoding

If modules do not contain any interfaces (e.g. `Account` or `Content`), then they
may simply migrate any existing types that
are encoded and persisted via their concrete Amino codec to Protobuf (see 1. for further guidelines) and accept a `Marshaler` as the codec which is implemented via the `ProtoCodec`
without any further customization.

However, if a module type composes an interface, it must wrap it in the `sdk.Any` (from `/types` package) type. To do that, a module-level .proto file must use [`google.protobuf.Any`](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto) for respective message type interface types.

For example, in the `x/evidence` module defines an `Evidence` interface, which is used by the `MsgSubmitEvidence`. The structure definition must use `sdk.Any` to wrap the evidence file. In the proto file we define it as follows:

```protobuf
// proto/cosmos/evidence/v1beta1/tx.proto

message MsgSubmitEvidence {
string submitter = 1;
google.protobuf.Any evidence = 2 [(cosmos_proto.accepts_interface) = "cosmos.evidence.v1beta1.Evidence"];
}
```

The Cosmos SDK `codec.Codec` interface provides support methods `MarshalInterface` and `UnmarshalInterface` to easy encoding of state to `Any`.

Module should register interfaces using `InterfaceRegistry` which provides a mechanism for registering interfaces: `RegisterInterface(protoName string, iface interface{}, impls ...proto.Message)` and implementations: `RegisterImplementations(iface interface{}, impls ...proto.Message)` that can be safely unpacked from Any, similarly to type registration with Amino:

```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.50.0-alpha.0/codec/types/interface_registry.go#L28-L75
```

In addition, an `UnpackInterfaces` phase should be introduced to deserialization to unpack interfaces before they're needed. Protobuf types that contain a protobuf `Any` either directly or via one of their members should implement the `UnpackInterfacesMessage` interface:

```go
type UnpackInterfacesMessage interface {
UnpackInterfaces(InterfaceUnpacker) error
}
```
Loading