Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add valset injection through r/sys/validators #2229

Merged
merged 97 commits into from
Jul 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
c4ef46b
Add the initial /sys/vals PoS implementation
zivkovicmilos May 16, 2024
8a6ea5c
Fix imports
zivkovicmilos May 17, 2024
e53e6bf
Add gno.mods, add initial PoA
zivkovicmilos May 17, 2024
9a8bc1d
Simplify API
zivkovicmilos May 17, 2024
b4a2452
Add initial PoS add validator tests
zivkovicmilos May 21, 2024
1c6a775
Add additional PoS tests
zivkovicmilos May 21, 2024
54884fd
Add initial PoS remove validator tests
zivkovicmilos May 21, 2024
afc2fcc
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos May 24, 2024
d2e1e24
Add unit tests for PoS validator removal
zivkovicmilos May 24, 2024
08f1351
Add unit tests for PoA validator operations
zivkovicmilos May 24, 2024
ee678fe
Cleanup
zivkovicmilos May 24, 2024
5a9385d
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos May 24, 2024
fe8ac45
Cleanup
zivkovicmilos May 24, 2024
4c21012
Fix stdshim std.Emit
zivkovicmilos May 24, 2024
9814fc1
Tidy unused methods
zivkovicmilos May 27, 2024
935f7b0
Add initial POC code for valset injections in Endblocker
zivkovicmilos May 28, 2024
958e20c
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos May 30, 2024
6a76e48
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jun 3, 2024
997f6af
Apply suggestions for set scraping
zivkovicmilos Jun 3, 2024
2e5aca1
Merge master; adapt r/sys/vals
zivkovicmilos Jun 17, 2024
6e61bf8
Add PoC
zivkovicmilos Jun 17, 2024
28abe9b
Drop JSON output for valsets
zivkovicmilos Jun 17, 2024
26325c2
Cleanup
zivkovicmilos Jun 17, 2024
eb34c7a
Tidy gno mod
zivkovicmilos Jun 17, 2024
a3049c3
Move back to r/sys/vals
zivkovicmilos Jun 17, 2024
cfd900d
Tidy gno mod
zivkovicmilos Jun 17, 2024
f1151be
Tidy
zivkovicmilos Jun 18, 2024
32fc706
Tidy
zivkovicmilos Jun 18, 2024
2be7527
Move testing helpers to uassert
zivkovicmilos Jun 18, 2024
ce9329a
Add the 'testing' package to the whitelist
zivkovicmilos Jun 19, 2024
b388ef4
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jun 20, 2024
2c85e06
Revert "Move testing helpers to uassert"
zivkovicmilos Jun 20, 2024
40cf5bb
Tidy go mods
zivkovicmilos Jun 20, 2024
5562b5b
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jun 21, 2024
a28623f
Drop PoA / PoS in this scope
zivkovicmilos Jun 26, 2024
2d365dd
Temporary rename
zivkovicmilos Jun 26, 2024
d1e3c96
Temporary rename
zivkovicmilos Jun 26, 2024
32ec0b6
Switch to testutils
zivkovicmilos Jun 26, 2024
d213e89
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jun 26, 2024
d81d300
Add voting power to AddValidator API
zivkovicmilos Jun 26, 2024
68534e7
Add GetValidator to Protocol API
zivkovicmilos Jun 26, 2024
51414b0
Switch to avl.Tree for map in PoC
zivkovicmilos Jun 26, 2024
53de738
Rename Protocol -> ValsetProtocol
zivkovicmilos Jun 26, 2024
8b6130a
Temporary rename
zivkovicmilos Jun 26, 2024
0860453
Temporary rename
zivkovicmilos Jun 26, 2024
68a2723
Temporary rename
zivkovicmilos Jun 26, 2024
ed2f7a6
Update examples/gno.land/r/sys/vals/vals.gno
zivkovicmilos Jun 27, 2024
3a7fd9c
Simplify verify methods
zivkovicmilos Jun 27, 2024
751b5d3
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jun 28, 2024
6a5978f
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jun 28, 2024
862b323
Merge branch 'dev/zivkovicmilos/sys-vals' into dev/zivkovicmilos/val-…
zivkovicmilos Jun 28, 2024
96a6d7a
Change event parsing for vals
zivkovicmilos Jun 28, 2024
4de53ff
Revert linter fix
zivkovicmilos Jun 28, 2024
b5e10f1
Use AVL to manage valsets
zivkovicmilos Jul 1, 2024
122e178
Simplify ValsetProtocol API
zivkovicmilos Jul 1, 2024
80fe0b4
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jul 1, 2024
ee0fff1
Merge branch 'dev/zivkovicmilos/sys-vals' into dev/zivkovicmilos/val-…
zivkovicmilos Jul 1, 2024
5371852
Apply renames
zivkovicmilos Jul 1, 2024
e45814b
Tidy mods
zivkovicmilos Jul 1, 2024
96e1364
Rename vals -> validators
zivkovicmilos Jul 1, 2024
5e9c3e0
Update examples/gno.land/r/gnoland/home/home.gno
moul Jul 1, 2024
0f2fb39
Update examples/gno.land/r/gnoland/home/home_filetest.gno
moul Jul 1, 2024
da51c23
Update examples/gno.land/r/sys/validators/poc.gno
zivkovicmilos Jul 2, 2024
b25171d
Move getChanges to gnosdk.gno
zivkovicmilos Jul 2, 2024
fa341a6
Simplify valset to global package vars
zivkovicmilos Jul 2, 2024
a0fc2c1
Standardize return API for ValsetProtocol
zivkovicmilos Jul 2, 2024
55cdfb2
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jul 2, 2024
cc6bc62
Merge master
zivkovicmilos Jul 2, 2024
48d6471
Merge branch 'master' into dev/zivkovicmilos/sys-vals
zivkovicmilos Jul 2, 2024
665ae34
Merge branch 'dev/zivkovicmilos/sys-vals' into dev/zivkovicmilos/val-…
zivkovicmilos Jul 2, 2024
845a8d7
Merge branch 'master' into dev/zivkovicmilos/val-injection
moul Jul 2, 2024
8d00e5f
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 2, 2024
ab78e08
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 2, 2024
d2c4b6c
Add unit tests for EndBlocker
zivkovicmilos Jul 2, 2024
0b3cf19
Add base event switch for in-memory node
zivkovicmilos Jul 3, 2024
40a8e03
Change up the GetChanges API
zivkovicmilos Jul 3, 2024
5f5e172
Use binary search for the changes
zivkovicmilos Jul 3, 2024
85e196d
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 4, 2024
b3231c9
Share event switch
zivkovicmilos Jul 4, 2024
83412aa
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 4, 2024
35df9eb
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 5, 2024
cafac9a
Revert to insanity
zivkovicmilos Jul 5, 2024
2592b43
Rename to vmk
zivkovicmilos Jul 5, 2024
d1100a7
Update gno.land/pkg/gnoland/vals.go
zivkovicmilos Jul 5, 2024
120fee9
Update gno.land/pkg/gnoland/vals.go
zivkovicmilos Jul 5, 2024
60cfdbf
Remove eventType
zivkovicmilos Jul 5, 2024
53cd5eb
Merge branch 'dev/zivkovicmilos/val-injection' of https://github.com/…
zivkovicmilos Jul 5, 2024
e628720
Switch to QueryEval instead of MsgCall
zivkovicmilos Jul 5, 2024
6c1fadd
Update gno.land/pkg/gnoland/vals.go
moul Jul 5, 2024
4d8dd4d
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 6, 2024
a16ca4b
Switch to avl.Tree for change management
zivkovicmilos Jul 6, 2024
a027d94
Use starting point for avl traversal
zivkovicmilos Jul 6, 2024
c946718
Simplify saveChange
zivkovicmilos Jul 6, 2024
7be72d2
Remove leftover bounds check
zivkovicmilos Jul 6, 2024
d19543d
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 6, 2024
e62705a
Merge branch 'master' into dev/zivkovicmilos/val-injection
zivkovicmilos Jul 6, 2024
abc2fc1
Display max N changes in the render
zivkovicmilos Jul 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion contribs/gnodev/pkg/dev/node.go
Original file line number Diff line number Diff line change
Expand Up @@ -527,7 +527,7 @@ func buildNode(logger *slog.Logger, emitter emitter.Emitter, cfg *gnoland.InMemo
func newNodeConfig(tmc *tmcfg.Config, chainid string, appstate gnoland.GnoGenesisState) *gnoland.InMemoryNodeConfig {
// Create Mocked Identity
pv := gnoland.NewMockedPrivValidator()
genesis := gnoland.NewDefaultGenesisConfig(pv.GetPubKey(), chainid)
genesis := gnoland.NewDefaultGenesisConfig(chainid)
genesis.AppState = appstate

// Add self as validator
Expand Down
8 changes: 4 additions & 4 deletions examples/gno.land/r/gov/dao/prop1_filetest.gno
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ func main() {
// Status: succeeded
// Author: g1wymu47drhr0kuq2098m792lytgtj2nyx77yrsm
// --
// Valset changes to apply:
// - g12345678 (10)
// - g000000000 (10)
// - g000000000 (0)
// Valset changes:
// - #123: g12345678 (10)
// - #123: g000000000 (10)
// - #123: g000000000 (0)
5 changes: 5 additions & 0 deletions examples/gno.land/r/sys/validators/gno.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
module gno.land/r/sys/validators

require (
gno.land/p/demo/avl v0.0.0-latest
gno.land/p/demo/seqid v0.0.0-latest
gno.land/p/demo/testutils v0.0.0-latest
gno.land/p/demo/uassert v0.0.0-latest
gno.land/p/demo/ufmt v0.0.0-latest
gno.land/p/gov/proposal v0.0.0-latest
gno.land/p/nt/poa v0.0.0-latest
gno.land/p/sys/validators v0.0.0-latest
Expand Down
26 changes: 17 additions & 9 deletions examples/gno.land/r/sys/validators/gnosdk.gno
Original file line number Diff line number Diff line change
@@ -1,16 +1,24 @@
package validators

import "gno.land/p/sys/validators"
import (
"gno.land/p/sys/validators"
)

// getChanges returns the validator changes stored on the realm.
// This function is unexported and intended to be called by gno.land through the GnoSDK
func getChanges() []validators.Validator {
// Construct the changes
valsetChanges := make([]validators.Validator, len(changes))
copy(valsetChanges, changes)
// GetChanges returns the validator changes stored on the realm, since the given block number.
// This function is intended to be called by gno.land through the GnoSDK
func GetChanges(from int64) []validators.Validator {
valsetChanges := make([]validators.Validator, 0)

// Reset the changes set
changes = changes[:0]
// Gather the changes from the specified block
changes.Iterate(getBlockID(from), "", func(_ string, value interface{}) bool {
chs := value.([]change)

for _, ch := range chs {
valsetChanges = append(valsetChanges, ch.validator)
}

return false
})

return valsetChanges
}
4 changes: 2 additions & 2 deletions examples/gno.land/r/sys/validators/init.gno
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
package validators

import (
"gno.land/p/demo/avl"
"gno.land/p/nt/poa"
"gno.land/p/sys/validators"
)

func init() {
// The default valset protocol is PoA
vp = poa.NewPoA()

// No changes to apply initially
changes = make([]validators.Validator, 0)
changes = avl.NewTree()
}
83 changes: 70 additions & 13 deletions examples/gno.land/r/sys/validators/validators.gno
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,24 @@ package validators

import (
"std"
"strconv"

"gno.land/p/demo/avl"
"gno.land/p/demo/seqid"
"gno.land/p/demo/ufmt"
"gno.land/p/sys/validators"
)

var (
vp validators.ValsetProtocol // p is the underlying validator set protocol
changes []validators.Validator // changes are the set changes that happened between scrapes
changes *avl.Tree // changes holds any valset changes; seqid(block number) -> []change
)

// change represents a single valset change, tied to a specific block number
type change struct {
blockNum int64 // the block number associated with the valset change
validator validators.Validator // the validator update
}

// addValidator adds a new validator to the validator set.
// If the validator is already present, the method errors out
func addValidator(validator validators.Validator) {
Expand All @@ -21,7 +29,12 @@ func addValidator(validator validators.Validator) {
}

// Validator added, note the change
changes = append(changes, val)
ch := change{
blockNum: std.GetHeight(),
validator: val,
}

saveChange(ch)

// Emit the validator set change
std.Emit(validators.ValidatorAddedEvent)
Expand All @@ -36,25 +49,69 @@ func removeValidator(address std.Address) {
}

// Validator removed, note the change
changes = append(changes, validators.Validator{
Address: val.Address,
PubKey: val.PubKey,
VotingPower: 0, // nullified the voting power indicates removal
})
ch := change{
blockNum: std.GetHeight(),
validator: validators.Validator{
Address: val.Address,
PubKey: val.PubKey,
VotingPower: 0, // nullified the voting power indicates removal
},
}

saveChange(ch)

// Emit the validator set change
std.Emit(validators.ValidatorRemovedEvent)
}

// saveChange saves the valset change
func saveChange(ch change) {
id := getBlockID(ch.blockNum)
zivkovicmilos marked this conversation as resolved.
Show resolved Hide resolved

setRaw, exists := changes.Get(id)
if !exists {
changes.Set(id, []change{ch})

return
}

// Save the change
set := setRaw.([]change)
set = append(set, ch)

changes.Set(id, set)
moul marked this conversation as resolved.
Show resolved Hide resolved
}

// getBlockID converts the block number to a sequential ID
func getBlockID(blockNum int64) string {
return seqid.ID(uint64(blockNum)).String()
}

func Render(_ string) string {
if len(changes) == 0 {
var (
size = changes.Size()
maxDisplay = 10
)

if size == 0 {
return "No valset changes to apply."
}

output := "Valset changes to apply:\n"
for _, change := range changes {
output += "- " + string(change.Address) + " (" + strconv.FormatUint(change.VotingPower, 10) + ")\n"
}
output := "Valset changes:\n"
changes.ReverseIterateByOffset(size-maxDisplay, maxDisplay, func(_ string, value interface{}) bool {
chs := value.([]change)

for _, ch := range chs {
output += ufmt.Sprintf(
"- #%d: %s (%d)\n",
ch.blockNum,
ch.validator.Address.String(),
ch.validator.VotingPower,
)
}

return false
})

return output
}
102 changes: 102 additions & 0 deletions examples/gno.land/r/sys/validators/validators_test.gno
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package validators

import (
"testing"

"std"

"gno.land/p/demo/avl"
"gno.land/p/demo/testutils"
"gno.land/p/demo/uassert"
"gno.land/p/demo/ufmt"
"gno.land/p/sys/validators"
)

// generateTestValidators generates a dummy validator set
func generateTestValidators(count int) []validators.Validator {
vals := make([]validators.Validator, 0, count)

for i := 0; i < count; i++ {
val := validators.Validator{
Address: testutils.TestAddress(ufmt.Sprintf("%d", i)),
PubKey: "public-key",
VotingPower: 10,
}

vals = append(vals, val)
}

return vals
}

func TestValidators_AddRemove(t *testing.T) {
// Clear any changes
changes = avl.NewTree()

var (
vals = generateTestValidators(100)
initialHeight = int64(123)
)

// Add in the validators
for _, val := range vals {
addValidator(val)

// Make sure the validator is added
uassert.True(t, vp.IsValidator(val.Address))

std.TestSkipHeights(1)
}

for i := initialHeight; i < initialHeight+int64(len(vals)); i++ {
// Make sure the changes are saved
chs := GetChanges(i)

// We use the funky index calculation to make sure
// changes are properly handled for each block span
uassert.Equal(t, initialHeight+int64(len(vals))-i, int64(len(chs)))

for index, val := range vals[i-initialHeight:] {
// Make sure the changes are equal to the additions
ch := chs[index]

uassert.Equal(t, val.Address, ch.Address)
uassert.Equal(t, val.PubKey, ch.PubKey)
uassert.Equal(t, val.VotingPower, ch.VotingPower)
}
}

// Save the beginning height for the removal
initialRemoveHeight := std.GetHeight()

// Clear any changes
changes = avl.NewTree()

// Remove the validators
for _, val := range vals {
removeValidator(val.Address)

// Make sure the validator is removed
uassert.False(t, vp.IsValidator(val.Address))

std.TestSkipHeights(1)
}

for i := initialRemoveHeight; i < initialRemoveHeight+int64(len(vals)); i++ {
// Make sure the changes are saved
chs := GetChanges(i)

// We use the funky index calculation to make sure
// changes are properly handled for each block span
uassert.Equal(t, initialRemoveHeight+int64(len(vals))-i, int64(len(chs)))

for index, val := range vals[i-initialRemoveHeight:] {
// Make sure the changes are equal to the additions
ch := chs[index]

uassert.Equal(t, val.Address, ch.Address)
uassert.Equal(t, val.PubKey, ch.PubKey)
uassert.Equal(t, uint64(0), ch.VotingPower)
}
}
}
8 changes: 6 additions & 2 deletions gno.land/cmd/gnoland/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
bft "github.com/gnolang/gno/tm2/pkg/bft/types"
"github.com/gnolang/gno/tm2/pkg/commands"
"github.com/gnolang/gno/tm2/pkg/crypto"
"github.com/gnolang/gno/tm2/pkg/events"
osm "github.com/gnolang/gno/tm2/pkg/os"
"github.com/gnolang/gno/tm2/pkg/telemetry"
"go.uber.org/zap"
Expand Down Expand Up @@ -239,14 +240,17 @@ func execStart(ctx context.Context, c *startCfg, io commands.IO) error {
io.Println(startGraphic)
}

// Create a top-level shared event switch
evsw := events.NewEventSwitch()

// Create application and node
cfg.LocalApp, err = gnoland.NewApp(nodeDir, c.skipFailingGenesisTxs, logger, c.genesisMaxVMCycles)
cfg.LocalApp, err = gnoland.NewApp(nodeDir, c.skipFailingGenesisTxs, evsw, logger)
if err != nil {
return fmt.Errorf("unable to create the Gnoland app, %w", err)
}

// Create a default node, with the given setup
gnoNode, err := node.DefaultNewNode(cfg, genesisPath, logger)
gnoNode, err := node.DefaultNewNode(cfg, genesisPath, evsw, logger)
if err != nil {
return fmt.Errorf("unable to create the Gnoland node, %w", err)
}
Expand Down
Loading
Loading