Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(lib/babe): fix setting first slot of network, fix loading BABE epoch params #1640

Merged
merged 98 commits into from
Jun 24, 2021
Merged
Show file tree
Hide file tree
Changes from 94 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
e92668d
add 3 auth genesis
noot Jun 9, 2021
8a67205
Merge branch 'noot/devnet-genesis' into noot/fix-babe-epoch
noot Jun 10, 2021
f893b82
improve epoch transition logic
noot Jun 10, 2021
37e457a
implement code substitutes
edwardmack Jun 10, 2021
a919188
fix babe tests
noot Jun 10, 2021
c23fee5
add wait until epoch starts
noot Jun 11, 2021
d23f2be
lint
noot Jun 11, 2021
52d0f73
merge w development
noot Jun 11, 2021
5cccb37
Merge branch 'development' into ed/code_substitutes
edwardmack Jun 11, 2021
b6c88b1
cleanup
noot Jun 11, 2021
578ff9c
restore genesis
noot Jun 11, 2021
6177b10
Merge branch 'development' of github.com:ChainSafe/gossamer into noot…
noot Jun 11, 2021
a49b3b5
add log
noot Jun 11, 2021
f3182de
set first slot only if block 1 not final; make unchangeable after blo…
noot Jun 11, 2021
74d73f6
fix setting of babe epoch data on restart after first epoch
noot Jun 11, 2021
17181f4
move digest handler to its own package; handle digests after built block
noot Jun 11, 2021
a5cfb7c
fmt
noot Jun 11, 2021
cdee399
fix babe tests
noot Jun 11, 2021
46dbf3b
cleanup core interfaces, fix digest package tests
noot Jun 11, 2021
74f6ade
add small wait before initiating babe
noot Jun 11, 2021
b42e644
lint, test fixes
noot Jun 14, 2021
a97a55b
cleanup, test epoch transition w 3 nodes
noot Jun 14, 2021
af18efb
fix
noot Jun 14, 2021
13cf25c
address comments
noot Jun 14, 2021
0b7e8f5
Merge branch 'noot/fix-babe-epoch' of github.com:ChainSafe/gossamer i…
noot Jun 14, 2021
e1ea906
increase test epoch length
noot Jun 14, 2021
2544e1e
merge w other branch
noot Jun 14, 2021
2a57a77
cleanup
noot Jun 14, 2021
2fb14ed
lint
noot Jun 14, 2021
fe09256
remove unused code
noot Jun 14, 2021
67227e9
start to fix tests
noot Jun 15, 2021
da1d50e
Merge branch 'development' into ed/code_substitutes
edwardmack Jun 15, 2021
18460c8
Merge branch 'development' of github.com:ChainSafe/gossamer into noot…
noot Jun 15, 2021
af33101
add go.mod
noot Jun 15, 2021
d2f38b2
update b.Paused to close channel
noot Jun 15, 2021
74f68f7
merge w development
noot Jun 15, 2021
eb63f16
fix
noot Jun 15, 2021
d580158
address PR comments
edwardmack Jun 15, 2021
27dc09f
update babe.Pause
noot Jun 15, 2021
7a5ce9e
remove unneeded files
noot Jun 15, 2021
749da75
add pause/resume test
noot Jun 15, 2021
1058a9a
lint
noot Jun 15, 2021
df318d6
small cleanup
noot Jun 15, 2021
5716370
Merge branch 'ed/code_substitutes' into noot/block-handle-refactor
noot Jun 15, 2021
f474f14
begin core handle block import refactor
noot Jun 15, 2021
b7af203
move digest handler to its own package
noot Jun 15, 2021
ce94028
move AddBlock and StoreTrie logic to core.handleBlock
noot Jun 15, 2021
9e4364b
update babe to use core.HandleBlockImport
noot Jun 15, 2021
a2f5452
merge w development
noot Jun 15, 2021
5d40e3d
fix test
noot Jun 15, 2021
f36e546
merge w development
noot Jun 15, 2021
9984b89
add handler to babe cfg
noot Jun 15, 2021
7ef0e79
cleanup
noot Jun 15, 2021
ed5ad02
set core digestHandler
noot Jun 16, 2021
74a3d19
fix dot, core, sync tests
noot Jun 16, 2021
5c4dff5
lint
noot Jun 16, 2021
29a4f10
merge w development
noot Jun 16, 2021
034a685
cleanup
noot Jun 16, 2021
a0ac8e7
cleanup
noot Jun 16, 2021
d71ae70
fix unit tests
noot Jun 16, 2021
0f52ff1
fix
noot Jun 16, 2021
9bc76ee
add lock for closing/writing to core blockAddCh
noot Jun 16, 2021
4010684
Merge branch 'development' of github.com:ChainSafe/gossamer into noot…
noot Jun 16, 2021
7598f83
store trie before adding block
noot Jun 16, 2021
b6bd916
fix
noot Jun 16, 2021
033d499
merge w refactor branch
noot Jun 16, 2021
786dc1c
fix unit tests
noot Jun 16, 2021
c7e724f
Merge branch 'noot/block-handle-refactor' of github.com:ChainSafe/gos…
noot Jun 16, 2021
4b67fe7
fix sync tests
noot Jun 16, 2021
f50ebaf
fix dot test
noot Jun 16, 2021
73c4168
fix babe to use correct handler func
noot Jun 16, 2021
35f38bc
lint
noot Jun 16, 2021
62b7277
fix babe to use right handler func
noot Jun 16, 2021
d1b23b1
fix
noot Jun 17, 2021
4d4942a
fix unit test
noot Jun 17, 2021
ff3d140
merge w development
noot Jun 17, 2021
68c130e
add go.sum
noot Jun 17, 2021
589380e
Merge branch 'development' of github.com:ChainSafe/gossamer into noot…
noot Jun 17, 2021
95ed8fb
merge w development
noot Jun 17, 2021
da1615c
Merge branch 'noot/block-handle-refactor' of github.com:ChainSafe/gos…
noot Jun 17, 2021
d1ad9cf
add dot/state unit tests, fix dot/digest tests
noot Jun 17, 2021
692d6b5
merge w development
noot Jun 18, 2021
37e3942
fix rpc tests
noot Jun 18, 2021
3433473
restore genesis, fix grandpa tests
noot Jun 18, 2021
bdd4cdc
Merge branch 'development' of github.com:ChainSafe/gossamer into noot…
noot Jun 18, 2021
02d013d
fix grandpa tests
noot Jun 18, 2021
9feab39
fix rpc tests
noot Jun 18, 2021
5ffbf8f
add unit tests for babe setupParameters
noot Jun 18, 2021
059eff3
lint
noot Jun 18, 2021
c2feee1
lint
noot Jun 18, 2021
c6c4769
fix rpc integration tests
noot Jun 18, 2021
ffa805e
cleanup
noot Jun 21, 2021
8eb50e9
address comments
noot Jun 22, 2021
63e009b
lint
noot Jun 22, 2021
7dc7f22
Merge branch 'development' of github.com:ChainSafe/gossamer into noot…
noot Jun 23, 2021
d4aabcf
address comments
noot Jun 23, 2021
4f81dbe
add go.mod
noot Jun 23, 2021
df6ae53
Merge branch 'development' into noot/fix-first-slot
noot Jun 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 0 additions & 27 deletions dot/core/mocks/verifier.go

This file was deleted.

4 changes: 3 additions & 1 deletion dot/digest/digest_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@ func addTestBlocksToStateWithParent(t *testing.T, previousHash common.Hash, dept
Header: &types.Header{
ParentHash: previousHash,
Number: big.NewInt(int64(i)).Add(previousNum, big.NewInt(int64(i))),
Digest: types.Digest{},
Digest: types.Digest{
types.NewBabeSecondaryPlainPreDigest(0, uint64(i)).ToPreRuntimeDigest(),
},
},
Body: &types.Body{},
}
Expand Down
2 changes: 1 addition & 1 deletion dot/network/sync.go
Original file line number Diff line number Diff line change
Expand Up @@ -578,7 +578,7 @@ func (q *syncQueue) processBlockRequests() {
}

func (q *syncQueue) trySync(req *syncRequest) {
if q.ctx.Err() != nil {
if q.ctx.Err() != nil || len(q.s.host.peers()) == 0 {
return
}

Expand Down
2 changes: 1 addition & 1 deletion dot/node_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ func TestStartNode(t *testing.T) {
go func() {
// TODO: need to wait until all services are started so that wg.Add is called, otherwise
// will call wg.Done before the counter is at 1
time.Sleep(time.Second * 15)
time.Sleep(time.Second * 20)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to listen on some sort of channel that can be triggered when the waitGroup is done? That way we don't have to wait on arbitrary delays, which could vary from system to system.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to have a started channel in *Node that's closed when all the services are started

node.Stop()
}()

Expand Down
1 change: 0 additions & 1 deletion dot/rpc/modules/author_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,6 @@ func TestAuthorModule_SubmitExtrinsic_invalid_input(t *testing.T) {
ext := Extrinsic{fmt.Sprintf("%x", "1")}

res := new(ExtrinsicHashResponse)

err := auth.SubmitExtrinsic(nil, &ext, res)
require.EqualError(t, err, "could not byteify non 0x prefixed string")
}
Expand Down
37 changes: 31 additions & 6 deletions dot/rpc/modules/chain_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,17 @@ func TestChainGetHeader_Genesis(t *testing.T) {
header, err := state.Block.BestBlockHeader()
require.NoError(t, err)

d, err := types.NewBabeSecondaryPlainPreDigest(0, 1).ToPreRuntimeDigest().Encode()
require.NoError(t, err)

expected := &ChainBlockHeaderResponse{
ParentHash: header.ParentHash.String(),
Number: common.BytesToHex(header.Number.Bytes()),
StateRoot: header.StateRoot.String(),
ExtrinsicsRoot: header.ExtrinsicsRoot.String(),
Digest: ChainBlockHeaderDigest{},
Digest: ChainBlockHeaderDigest{
Logs: []string{common.BytesToHex(d)},
},
}

hash := state.Block.BestBlockHash()
Expand All @@ -64,12 +69,17 @@ func TestChainGetHeader_Latest(t *testing.T) {
header, err := state.Block.BestBlockHeader()
require.NoError(t, err)

d, err := types.NewBabeSecondaryPlainPreDigest(0, 1).ToPreRuntimeDigest().Encode()
require.NoError(t, err)

expected := &ChainBlockHeaderResponse{
ParentHash: header.ParentHash.String(),
Number: common.BytesToHex(header.Number.Bytes()),
StateRoot: header.StateRoot.String(),
ExtrinsicsRoot: header.ExtrinsicsRoot.String(),
Digest: ChainBlockHeaderDigest{},
Digest: ChainBlockHeaderDigest{
Logs: []string{common.BytesToHex(d)},
},
}

res := &ChainBlockHeaderResponse{}
Expand Down Expand Up @@ -101,12 +111,17 @@ func TestChainGetBlock_Genesis(t *testing.T) {
header, err := state.Block.BestBlockHeader()
require.NoError(t, err)

d, err := types.NewBabeSecondaryPlainPreDigest(0, 1).ToPreRuntimeDigest().Encode()
require.NoError(t, err)

expectedHeader := &ChainBlockHeaderResponse{
ParentHash: header.ParentHash.String(),
Number: common.BytesToHex(header.Number.Bytes()),
StateRoot: header.StateRoot.String(),
ExtrinsicsRoot: header.ExtrinsicsRoot.String(),
Digest: ChainBlockHeaderDigest{},
Digest: ChainBlockHeaderDigest{
Logs: []string{common.BytesToHex(d)},
},
}

hash := state.Block.BestBlockHash()
Expand Down Expand Up @@ -134,12 +149,17 @@ func TestChainGetBlock_Latest(t *testing.T) {
header, err := state.Block.BestBlockHeader()
require.NoError(t, err)

d, err := types.NewBabeSecondaryPlainPreDigest(0, 1).ToPreRuntimeDigest().Encode()
require.NoError(t, err)

expectedHeader := &ChainBlockHeaderResponse{
ParentHash: header.ParentHash.String(),
Number: common.BytesToHex(header.Number.Bytes()),
StateRoot: header.StateRoot.String(),
ExtrinsicsRoot: header.ExtrinsicsRoot.String(),
Digest: ChainBlockHeaderDigest{},
Digest: ChainBlockHeaderDigest{
Logs: []string{common.BytesToHex(d)},
},
}

expected := &ChainBlockResponse{
Expand Down Expand Up @@ -267,6 +287,9 @@ func TestChainGetFinalizedHeadByRound(t *testing.T) {

header := &types.Header{
Number: big.NewInt(1),
Digest: types.Digest{
types.NewBabeSecondaryPlainPreDigest(0, 1).ToPreRuntimeDigest(),
},
}
err = state.Block.SetHeader(header)
require.NoError(t, err)
Expand Down Expand Up @@ -358,8 +381,10 @@ func loadTestBlocks(gh common.Hash, bs *state.BlockState) error {

// Create header & blockData for block 1
header1 := &types.Header{
Number: big.NewInt(1),
Digest: types.Digest{},
Number: big.NewInt(1),
Digest: types.Digest{
types.NewBabeSecondaryPlainPreDigest(0, 1).ToPreRuntimeDigest(),
},
ParentHash: blockHash0,
StateRoot: trie.EmptyHash,
}
Expand Down
30 changes: 30 additions & 0 deletions dot/state/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,36 @@ func (s *BaseState) loadFirstSlot() (uint64, error) {
return binary.LittleEndian.Uint64(data), nil
}

func (s *BaseState) storeEpochLength(l uint64) error {
buf := make([]byte, 8)
binary.LittleEndian.PutUint64(buf, l)
return s.db.Put(epochLengthKey, buf)
}

func (s *BaseState) loadEpochLength() (uint64, error) {
data, err := s.db.Get(epochLengthKey)
if err != nil {
return 0, err
}

return binary.LittleEndian.Uint64(data), nil
}

func (s *BaseState) storeSlotDuration(duration uint64) error {
buf := make([]byte, 8)
binary.LittleEndian.PutUint64(buf, duration)
return s.db.Put(slotDurationKey, buf)
}

func (s *BaseState) loadSlotDuration() (uint64, error) {
data, err := s.db.Get(slotDurationKey)
if err != nil {
return 0, err
}

return binary.LittleEndian.Uint64(data), nil
}

// storePruningData stores the pruner configuration.
func (s *BaseState) storePruningData(mode pruner.Config) error {
encMode, err := json.Marshal(mode)
Expand Down
26 changes: 26 additions & 0 deletions dot/state/base_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,29 @@ func TestStoreAndLoadBestBlockHash(t *testing.T) {
require.NoError(t, err)
require.Equal(t, hash, res)
}

func TestLoadStoreEpochLength(t *testing.T) {
db := NewInMemoryDB(t)
base := NewBaseState(db)

length := uint64(2222)
err := base.storeEpochLength(length)
require.NoError(t, err)

ret, err := base.loadEpochLength()
require.NoError(t, err)
require.Equal(t, length, ret)
}

func TestLoadAndStoreSlotDuration(t *testing.T) {
db := NewInMemoryDB(t)
base := NewBaseState(db)

d := uint64(3000)
err := base.storeSlotDuration(d)
require.NoError(t, err)

ret, err := base.loadSlotDuration()
require.NoError(t, err)
require.Equal(t, d, ret)
}
42 changes: 41 additions & 1 deletion dot/state/block.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ type BlockState struct {
baseState *BaseState
db chaindb.Database
sync.RWMutex
genesisHash common.Hash
genesisHash common.Hash
lastFinalised common.Hash

// block notifiers
imported map[byte]chan<- *types.Block
Expand Down Expand Up @@ -74,6 +75,11 @@ func NewBlockState(db chaindb.Database, bt *blocktree.BlockTree) (*BlockState, e
}

bs.genesisHash = genesisBlock.Header.Hash()
bs.lastFinalised, err = bs.GetFinalizedHash(0, 0)
if err != nil {
return nil, fmt.Errorf("failed to get last finalised hash: %w", err)
}

return bs, nil
}

Expand Down Expand Up @@ -372,6 +378,16 @@ func (bs *BlockState) HasFinalizedBlock(round, setID uint64) (bool, error) {
return bs.db.Has(finalizedHashKey(round, setID))
}

// NumberIsFinalised checks if a block number is finalised or not
func (bs *BlockState) NumberIsFinalised(num *big.Int) (bool, error) {
header, err := bs.GetFinalizedHeader(0, 0)
if err != nil {
return false, err
}

return num.Cmp(header.Number) <= 0, nil
}

// GetFinalizedHeader returns the latest finalised block header
func (bs *BlockState) GetFinalizedHeader(round, setID uint64) (*types.Header, error) {
h, err := bs.GetFinalizedHash(round, setID)
Expand Down Expand Up @@ -407,6 +423,15 @@ func (bs *BlockState) SetFinalizedHash(hash common.Hash, round, setID uint64) er
return fmt.Errorf("cannot finalise unknown block %s", hash)
}

// if nothing was previously finalised, set the first slot of the network to the
// slot number of block 1, which is now being set as final
if bs.lastFinalised.Equal(bs.genesisHash) && !hash.Equal(bs.genesisHash) {
err := bs.setFirstSlotOnFinalisation()
if err != nil {
return err
}
}

go bs.notifyFinalized(hash, round, setID)
if round > 0 {
err := bs.SetRound(round)
Expand All @@ -431,9 +456,24 @@ func (bs *BlockState) SetFinalizedHash(hash common.Hash, round, setID uint64) er
bs.pruneKeyCh <- header
}

bs.lastFinalised = hash
noot marked this conversation as resolved.
Show resolved Hide resolved
return bs.db.Put(finalizedHashKey(round, setID), hash[:])
}

func (bs *BlockState) setFirstSlotOnFinalisation() error {
header, err := bs.GetHeaderByNumber(big.NewInt(1))
if err != nil {
return err
}

slot, err := types.GetSlotFromHeader(header)
if err != nil {
return err
}

return bs.baseState.storeFirstSlot(slot)
}

// SetRound sets the latest finalised GRANDPA round in the db
// TODO: this needs to use both setID and round
func (bs *BlockState) SetRound(round uint64) error {
Expand Down
Loading