Skip to content

Commit

Permalink
resolve
Browse files Browse the repository at this point in the history
  • Loading branch information
mmsqe committed Dec 12, 2024
1 parent 09e980a commit 1313a05
Show file tree
Hide file tree
Showing 41 changed files with 1,106 additions and 129 deletions.
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,34 @@ Ref: https://keepachangelog.com/en/1.0.0/

# Changelog

## [Unreleased-Upstream]

### Features

* (baseapp) [#205](https://github.com/crypto-org-chain/cosmos-sdk/pull/205) Add `TxExecutor` baseapp option, add `TxIndex`/`TxCount`/`MsgIndex`/`BlockGasUsed` fields to `Context, to support tx parallel execution.
* (baseapp) [#206](https://github.com/crypto-org-chain/cosmos-sdk/pull/206) Support mount object store in baseapp, add `ObjectStore` api in context, [#585](https://github.com/crypto-org-chain/cosmos-sdk/pull/585) Skip snapshot for object store.
* (bank) [#237](https://github.com/crypto-org-chain/cosmos-sdk/pull/237) Support virtual accounts in sending coins.
* [#243](https://github.com/crypto-org-chain/cosmos-sdk/pull/243) Support `RunAtomic` API in `Context` to use new CoW branched cache store.
* [#248](https://github.com/crypto-org-chain/cosmos-sdk/pull/248) Init btree store lazily to save allocations when no content insert into it.
* [#252](https://github.com/crypto-org-chain/cosmos-sdk/pull/252) Add `BlockGasWanted` to `Context` to support feemarket module.
* [#269](https://github.com/crypto-org-chain/cosmos-sdk/pull/269) Add `StreamingManager` to baseapp to extend the abci listeners.
* (baseapp) [#565](https://github.com/crypto-org-chain/cosmos-sdk/pull/565) Support incarnation cache when executed in block-stm.

### Improvements

* [#261](https://github.com/crypto-org-chain/cosmos-sdk/pull/261) `ctx.BlockHeader` don't do protobuf deep copy, shallow copy seems enough, reduce scope of mutex in `PriorityNonceMempool.Remove`.
* [#507](https://github.com/crypto-org-chain/cosmos-sdk/pull/507) mempool respect gas wanted returned by ante handler
* [#744](https://github.com/crypto-org-chain/cosmos-sdk/pull/744) Pass raw transactions to tx executor so it can do pre-estimations.
* [#884](https://github.com/crypto-org-chain/cosmos-sdk/pull/884) Avoid decoding tx for in PrepareProposal if it's NopMempool.

## [Unreleased]

## [v0.50.11](https://github.com/cosmos/cosmos-sdk/releases/tag/v0.50.11) - 2024-12-04

### Features

* (crypto/keyring) [#21653](https://github.com/cosmos/cosmos-sdk/pull/21653) New Linux-only backend that adds Linux kernel's `keyctl` support.
* (crypto/keyring) [#20212](https://github.com/cosmos/cosmos-sdk/pull/20212) Expose the db keyring used in the keystore.

### Improvements

Expand All @@ -52,11 +73,14 @@ Ref: https://keepachangelog.com/en/1.0.0/

### Bug Fixes

* (sims) [21906](https://github.com/cosmos/cosmos-sdk/pull/21906) Skip sims test when running dry on validators
* (cli) [#21919](https://github.com/cosmos/cosmos-sdk/pull/21919) Query address-by-acc-num by account_id instead of id.
* (server) [#22564](https://github.com/cosmos/cosmos-sdk/pull/22564) Fix fallback genesis path in server
* (x/group) [#22425](https://github.com/cosmos/cosmos-sdk/pull/22425) Proper address rendering in error
* (sims) [#21906](https://github.com/cosmos/cosmos-sdk/pull/21906) Skip sims test when running dry on validators
* (cli) [#21919](https://github.com/cosmos/cosmos-sdk/pull/21919) Query address-by-acc-num by account_id instead of id.
* (x/group) [#22229](https://github.com/cosmos/cosmos-sdk/pull/22229) Accept `1` and `try` in CLI for group proposal exec.
* (cli) [#22656](https://github.com/cosmos/cosmos-sdk/pull/22656) Prune cmd should disable async pruning.

## [v0.50.10](https://github.com/cosmos/cosmos-sdk/releases/tag/v0.50.10) - 2024-09-20

Expand Down
2 changes: 1 addition & 1 deletion baseapp/abci_utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ func (h *DefaultProposalHandler) PrepareProposalHandler() sdk.PrepareProposalHan
selectedTxsNums int
invalidTxs []sdk.Tx // invalid txs to be removed out of the loop to avoid dead lock
)
h.mempool.SelectBy(ctx, req.Txs, func(memTx mempool.Tx) bool {
mempool.SelectBy(ctx, h.mempool, req.Txs, func(memTx mempool.Tx) bool {
signerData, err := h.signerExtAdapter.GetSigners(memTx.Tx)
if err != nil {
// propagate the error to the caller
Expand Down
23 changes: 23 additions & 0 deletions collections/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,29 @@ Ref: https://keepachangelog.com/en/1.0.0/

## [Unreleased]

## [v0.4.0](https://github.com/cosmos/cosmos-sdk/releases/tag/collections%2Fv0.4.0)

### Features

* [#17024](https://github.com/cosmos/cosmos-sdk/pull/17024) - Introduces `Triple`, a composite key with three keys.

### API Breaking

* [#17290](https://github.com/cosmos/cosmos-sdk/pull/17290) - Collections iteration methods (Iterate, Walk) will not error when the collection is empty.

### Improvements

* [#17021](https://github.com/cosmos/cosmos-sdk/pull/17021) Make collections implement the `appmodule.HasGenesis` interface.

## [v0.3.0](https://github.com/cosmos/cosmos-sdk/releases/tag/collections%2Fv0.3.0)

### Features

* [#16074](https://github.com/cosmos/cosmos-sdk/pull/16607) - Introduces `Clear` method for `Map` and `KeySet`
* [#16773](https://github.com/cosmos/cosmos-sdk/pull/16773)
* Adds `AltValueCodec` which provides a way to decode a value in two ways.
* Adds the possibility to specify an alternative way to decode the values of `KeySet`, `indexes.Multi`, `indexes.ReversePair`.

## [v0.2.0](https://github.com/cosmos/cosmos-sdk/releases/tag/collections%2Fv0.2.0)

### Features
Expand Down
79 changes: 79 additions & 0 deletions collections/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1117,3 +1117,82 @@ func (k Keeper) GetAccount(ctx sdk.context, addr sdk.AccAddress) (sdk.AccountI,
return k.Accounts.Get(ctx, addr)
}
```

## Triple key

The `collections.Triple` is a special type of key composed of three keys, it's identical to `collections.Pair`.

Let's see an example.

```go
package example

import (
"context"

"cosmossdk.io/collections"
storetypes "cosmossdk.io/store/types"
"github.com/cosmos/cosmos-sdk/codec"
)

type AccAddress = string
type ValAddress = string

type Keeper struct {
// let's simulate we have redelegations which are stored as a triple key composed of
// the delegator, the source validator and the destination validator.
Redelegations collections.KeySet[collections.Triple[AccAddress, ValAddress, ValAddress]]
}

func NewKeeper(storeKey *storetypes.KVStoreKey) Keeper {
sb := collections.NewSchemaBuilder(sdk.OpenKVStore(storeKey))
return Keeper{
Redelegations: collections.NewKeySet(sb, collections.NewPrefix(0), "redelegations", collections.TripleKeyCodec(collections.StringKey, collections.StringKey, collections.StringKey)
}
}

// RedelegationsByDelegator iterates over all the redelegations of a given delegator and calls onResult providing
// each redelegation from source validator towards the destination validator.
func (k Keeper) RedelegationsByDelegator(ctx context.Context, delegator AccAddress, onResult func(src, dst ValAddress) (stop bool, err error)) error {
rng := collections.NewPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator)
return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) {
return onResult(key.K2(), key.K3())
})
}

// RedelegationsByDelegatorAndValidator iterates over all the redelegations of a given delegator and its source validator and calls onResult for each
// destination validator.
func (k Keeper) RedelegationsByDelegatorAndValidator(ctx context.Context, delegator AccAddress, validator ValAddress, onResult func(dst ValAddress) (stop bool, err error)) error {
rng := collections.NewSuperPrefixedTripleRange[AccAddress, ValAddress, ValAddress](delegator, validator)
return k.Redelegations.Walk(ctx, rng, func(key collections.Triple[AccAddress, ValAddress, ValAddress]) (stop bool, err error) {
return onResult(key.K3())
})
}
```

## Advanced Usages

### Alternative Value Codec

The `codec.AltValueCodec` allows a collection to decode values using a different codec than the one used to encode them.
Basically it enables to decode two different byte representations of the same concrete value.
It can be used to lazily migrate values from one bytes representation to another, as long as the new representation is
not able to decode the old one.

A concrete example can be found in `x/bank` where the balance was initially stored as `Coin` and then migrated to `Int`.

```go

var BankBalanceValueCodec = codec.NewAltValueCodec(sdk.IntValue, func(b []byte) (sdk.Int, error) {
coin := sdk.Coin{}
err := coin.Unmarshal(b)
if err != nil {
return sdk.Int{}, err
}
return coin.Amount, nil
})
```

The above example shows how to create an `AltValueCodec` that can decode both `sdk.Int` and `sdk.Coin` values. The provided
decoder function will be used as a fallback in case the default decoder fails. When the value will be encoded back into state
it will use the default encoder. This allows to lazily migrate values to a new bytes representation.
47 changes: 47 additions & 0 deletions collections/codec/alternative_value.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
package codec

// NewAltValueCodec returns a new AltValueCodec. canonicalValueCodec is the codec that you want the value
// to be encoded and decoded as, alternativeDecoder is a function that will attempt to decode the value
// in case the canonicalValueCodec fails to decode it.
func NewAltValueCodec[V any](canonicalValueCodec ValueCodec[V], alternativeDecoder func([]byte) (V, error)) ValueCodec[V] {
return AltValueCodec[V]{
canonicalValueCodec: canonicalValueCodec,
alternativeDecoder: alternativeDecoder,
}
}

// AltValueCodec is a codec that can decode a value from state in an alternative format.
// This is useful for migrating data from one format to another. For example, in x/bank
// balances were initially encoded as sdk.Coin, now they are encoded as math.Int.
// The AltValueCodec will be trying to decode the value as math.Int, and if that fails,
// it will attempt to decode it as sdk.Coin.
// NOTE: if the canonical format can also decode the alternative format, then this codec
// will produce undefined and undesirable behavior.
type AltValueCodec[V any] struct {
canonicalValueCodec ValueCodec[V]
alternativeDecoder func([]byte) (V, error)
}

// Decode will attempt to decode the value from state using the canonical value codec.
// If it fails to decode, it will attempt to decode the value using the alternative decoder.
func (a AltValueCodec[V]) Decode(b []byte) (V, error) {
v, err := a.canonicalValueCodec.Decode(b)
if err != nil {
return a.alternativeDecoder(b)
}
return v, nil
}

// Below there is the implementation of ValueCodec relying on the canonical value codec.

func (a AltValueCodec[V]) Encode(value V) ([]byte, error) { return a.canonicalValueCodec.Encode(value) }

func (a AltValueCodec[V]) EncodeJSON(value V) ([]byte, error) {
return a.canonicalValueCodec.EncodeJSON(value)
}

func (a AltValueCodec[V]) DecodeJSON(b []byte) (V, error) { return a.canonicalValueCodec.DecodeJSON(b) }

func (a AltValueCodec[V]) Stringify(value V) string { return a.canonicalValueCodec.Stringify(value) }

func (a AltValueCodec[V]) ValueType() string { return a.canonicalValueCodec.ValueType() }
53 changes: 53 additions & 0 deletions collections/codec/alternative_value_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
package codec_test

import (
"encoding/json"
"testing"

"github.com/stretchr/testify/require"

"cosmossdk.io/collections/codec"
"cosmossdk.io/collections/colltest"
)

type altValue struct {
Value uint64 `json:"value"`
}

func TestAltValueCodec(t *testing.T) {
// we assume we want to migrate the value from json(altValue) to just be
// the raw value uint64.
canonical := codec.KeyToValueCodec(codec.NewUint64Key[uint64]())
alternative := func(v []byte) (uint64, error) {
var alt altValue
err := json.Unmarshal(v, &alt)
if err != nil {
return 0, err
}
return alt.Value, nil
}

cdc := codec.NewAltValueCodec(canonical, alternative)

t.Run("decodes alternative value", func(t *testing.T) {
expected := uint64(100)
alternativeEncodedBytes, err := json.Marshal(altValue{Value: expected})
require.NoError(t, err)
got, err := cdc.Decode(alternativeEncodedBytes)
require.NoError(t, err)
require.Equal(t, expected, got)
})

t.Run("decodes canonical value", func(t *testing.T) {
expected := uint64(100)
canonicalEncodedBytes, err := cdc.Encode(expected)
require.NoError(t, err)
got, err := cdc.Decode(canonicalEncodedBytes)
require.NoError(t, err)
require.Equal(t, expected, got)
})

t.Run("conformance", func(t *testing.T) {
colltest.TestValueCodec(t, cdc, uint64(100))
})
}
2 changes: 1 addition & 1 deletion collections/codec/bool.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ func (b boolKey[T]) Decode(buffer []byte) (int, T, error) {
}
}

func (b boolKey[T]) Size(key T) int { return 1 }
func (b boolKey[T]) Size(_ T) int { return 1 }

func (b boolKey[T]) EncodeJSON(value T) ([]byte, error) {
return json.Marshal(value)
Expand Down
5 changes: 5 additions & 0 deletions collections/colltest/codec.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ func TestKeyCodec[T any](t *testing.T, keyCodec codec.KeyCodec[T], key T) {
decoded, err := keyCodec.DecodeJSON(keyJSON)
require.NoError(t, err)
require.Equal(t, key, decoded, "json encoding and decoding did not produce the same results")

// check type
require.NotEmpty(t, keyCodec.KeyType())
// check string
_ = keyCodec.Stringify(key)
}

// TestValueCodec asserts the correct behavior of a ValueCodec over the type T.
Expand Down
8 changes: 4 additions & 4 deletions collections/genesis.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func (m Map[K, V]) importGenesis(ctx context.Context, reader io.Reader) error {
}

func (m Map[K, V]) exportGenesis(ctx context.Context, writer io.Writer) error {
_, err := writer.Write([]byte("["))
_, err := io.WriteString(writer, "[")
if err != nil {
return err
}
Expand All @@ -48,7 +48,7 @@ func (m Map[K, V]) exportGenesis(ctx context.Context, writer io.Writer) error {
// add a comma before encoding the object
// for all objects besides the first one.
if !first {
_, err = writer.Write([]byte(","))
_, err = io.WriteString(writer, ",")
if err != nil {
return err
}
Expand Down Expand Up @@ -91,7 +91,7 @@ func (m Map[K, V]) exportGenesis(ctx context.Context, writer io.Writer) error {
}
}

_, err = writer.Write([]byte("]"))
_, err = io.WriteString(writer, "]")
return err
}

Expand Down Expand Up @@ -148,6 +148,6 @@ func (m Map[K, V]) doDecodeJSON(reader io.Reader, onEntry func(key K, value V) e
}

func (m Map[K, V]) defaultGenesis(writer io.Writer) error {
_, err := writer.Write([]byte(`[]`))
_, err := io.WriteString(writer, `[]`)
return err
}
3 changes: 3 additions & 0 deletions collections/genesis_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ type testFixture struct {
}

func initFixture(t *testing.T) *testFixture {
t.Helper()
sk, ctx := deps()
schemaBuilder := NewSchemaBuilder(sk)
m := NewMap(schemaBuilder, NewPrefix(1), "map", StringKey, Uint64Value)
Expand All @@ -110,6 +111,7 @@ func initFixture(t *testing.T) *testFixture {
}

func createTestGenesisSource(t *testing.T) appmodule.GenesisSource {
t.Helper()
expectedOrder := []string{"item", "key_set", "map", "sequence"}
currentIndex := 0
return func(field string) (io.ReadCloser, error) {
Expand Down Expand Up @@ -149,6 +151,7 @@ func (b *bufCloser) Close() error {
}

func newBufCloser(t *testing.T, str string) *bufCloser {
t.Helper()
b := &bufCloser{
Buffer: bytes.NewBufferString(str),
closed: false,
Expand Down
Loading

0 comments on commit 1313a05

Please sign in to comment.