Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snow VM #1831

Merged
merged 69 commits into from
Jan 15, 2025
Merged

Snow VM #1831

Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
9e347ab
Create snow vm wrapper to provide snowman.Block types
aaronbuchwald Dec 8, 2024
02c86a2
update vm package to match new snow.Chain interface
aaronbuchwald Dec 9, 2024
a5f86a6
add back state sync ready check
aaronbuchwald Dec 10, 2024
11e56af
Separate snow vm from chain
aaronbuchwald Dec 10, 2024
da945fa
Add snow tests
aaronbuchwald Dec 11, 2024
053cf8a
Add snow fuzz test
aaronbuchwald Dec 11, 2024
f496712
clean up
aaronbuchwald Dec 11, 2024
de739b3
Add state sync vm tests
aaronbuchwald Dec 12, 2024
b47f5cb
Add FinishStateSync and FinishOnAcceptedAncestor test
aaronbuchwald Dec 13, 2024
e4fa1ec
Handle init ready/not ready
aaronbuchwald Dec 13, 2024
e09bc90
fix state sync tests
aaronbuchwald Dec 13, 2024
9e4daa0
move
aaronbuchwald Dec 13, 2024
a5dcdc2
Re-arrange
aaronbuchwald Dec 13, 2024
39fe626
Update vm metrics
aaronbuchwald Dec 13, 2024
8fb7697
Separate chain index
aaronbuchwald Dec 14, 2024
2c24704
rename snow options to application
aaronbuchwald Dec 14, 2024
9ea286c
working with MakeChainIndexF
aaronbuchwald Dec 17, 2024
156ded0
Add hypervm unit tests
aaronbuchwald Dec 18, 2024
46b1bfa
Add external subscriber unit test
aaronbuchwald Dec 18, 2024
a86f91c
Add direct state API unit test
aaronbuchwald Dec 18, 2024
6bd3a69
Add ws integration test
aaronbuchwald Dec 19, 2024
3bdd769
fix ws client integration test to handle buggy ws server
aaronbuchwald Dec 19, 2024
7f7afd2
Remove outstanding test comments
aaronbuchwald Dec 19, 2024
78079a2
WIP: add state sync integration test
aaronbuchwald Dec 23, 2024
f46637d
Integration tests working
aaronbuchwald Dec 26, 2024
e8845d5
remove dead code
aaronbuchwald Dec 26, 2024
e8be361
Remove deadcode and add shutdown to vm tests
aaronbuchwald Dec 26, 2024
bb5fc69
trim config
aaronbuchwald Dec 27, 2024
2cd7b9f
Add back stateful block Stringer
aaronbuchwald Dec 29, 2024
02af352
Fix lint
aaronbuchwald Dec 29, 2024
0d5e56e
fix: keep txs after gossiping instead of dropping after gossip to pro…
aaronbuchwald Dec 29, 2024
e1aba3a
cleanup lint
aaronbuchwald Dec 29, 2024
391bd4f
fix vm test
aaronbuchwald Dec 29, 2024
31b2552
fix nolint
aaronbuchwald Dec 29, 2024
1762edd
replace nolint:G404 with nolint:gosec
aaronbuchwald Dec 29, 2024
d24e07d
fix client access race condition during force gossip test
aaronbuchwald Dec 29, 2024
f8c4b97
fix race during state sync test
aaronbuchwald Dec 29, 2024
42cd39a
re-arrange comment to avoid false positive dupword linter
aaronbuchwald Dec 29, 2024
b7f34ef
add version to morpheus constructor
aaronbuchwald Dec 29, 2024
02dfb3e
cleanup snow package
aaronbuchwald Dec 29, 2024
320c411
remove lifecycle package
aaronbuchwald Dec 29, 2024
972fb0b
Remove unused test err
aaronbuchwald Dec 29, 2024
b9c0f71
fix mark ready in MakeChainIndex and test configs
aaronbuchwald Dec 30, 2024
92b73a8
Cleanup
aaronbuchwald Dec 30, 2024
fe04049
Fix handling of stateReady in MakeChainIndex
aaronbuchwald Dec 30, 2024
ed6de54
name closers
aaronbuchwald Dec 30, 2024
8f83c7b
fix e2e tests
aaronbuchwald Jan 2, 2025
6d4fe49
Address comments
aaronbuchwald Jan 3, 2025
f767948
narrow hcontext to config only
aaronbuchwald Jan 3, 2025
c8880a6
Add needed config helpers to keep service configs the same
aaronbuchwald Jan 4, 2025
06f6a46
fix lint
aaronbuchwald Jan 4, 2025
b90413a
go mod tidy
aaronbuchwald Jan 4, 2025
86a8b47
Address comments
aaronbuchwald Jan 4, 2025
0ed5c18
fix missing metric init
aaronbuchwald Jan 4, 2025
7df4fb5
fix built txs metric and morpheusvm main
aaronbuchwald Jan 4, 2025
38f01d4
Add comment to chainLock
aaronbuchwald Jan 6, 2025
787b271
Merge branch 'main' into snow-vm
aaronbuchwald Jan 10, 2025
b62b541
Fix merge
aaronbuchwald Jan 10, 2025
169c637
fix lint
aaronbuchwald Jan 10, 2025
836087c
remove integration tests from CI
aaronbuchwald Jan 14, 2025
1b01e8e
Snow vm integration (#1867)
aaronbuchwald Jan 15, 2025
328ecf9
cleanup
aaronbuchwald Jan 15, 2025
e9cdda7
Remove recursive verification and improve snow tests
aaronbuchwald Jan 15, 2025
546859d
Update sort func
aaronbuchwald Jan 15, 2025
efcefc7
replace readyL with atomic bool
aaronbuchwald Jan 15, 2025
77e618c
Merge branch 'main' into snow-vm
aaronbuchwald Jan 15, 2025
098714f
Merge branch 'main' into snow-vm
aaronbuchwald Jan 15, 2025
514bdf4
Remove dead code getExclusiveBlockRange
aaronbuchwald Jan 15, 2025
1262be9
bump initial balance
aaronbuchwald Jan 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions api/dependencies.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,11 @@ type VM interface {
ctx context.Context,
txs []*chain.Transaction,
) (errs []error)
LastAcceptedBlockResult() *chain.ExecutedBlock
LastAcceptedBlock(ctx context.Context) *chain.StatelessBlock
UnitPrices(context.Context) (fees.Dimensions, error)
CurrentValidators(
context.Context,
) (map[ids.NodeID]*validators.GetValidatorOutput, map[string]struct{})
GetVerifyAuth() bool
ReadState(ctx context.Context, keys [][]byte) ([][]byte, []error)
ImmutableState(ctx context.Context) (state.Immutable, error)
BalanceHandler() chain.BalanceHandler
Expand Down
10 changes: 5 additions & 5 deletions api/jsonrpc/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,11 +116,11 @@ type LastAcceptedReply struct {
Timestamp int64 `json:"timestamp"`
}

func (j *JSONRPCServer) LastAccepted(_ *http.Request, _ *struct{}, reply *LastAcceptedReply) error {
blk := j.vm.LastAcceptedBlockResult()
reply.Height = blk.Block.Hght
reply.BlockID = blk.Block.ID()
reply.Timestamp = blk.Block.Tmstmp
func (j *JSONRPCServer) LastAccepted(req *http.Request, _ *struct{}, reply *LastAcceptedReply) error {
blk := j.vm.LastAcceptedBlock(req.Context())
reply.Height = blk.Hght
reply.BlockID = blk.ID()
reply.Timestamp = blk.Tmstmp
return nil
}

Expand Down
7 changes: 5 additions & 2 deletions api/state/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,11 @@ func (s *JSONRPCStateServer) ReadState(req *http.Request, args *ReadStateRequest

var errs []error
res.Values, errs = s.stateReader.ReadState(ctx, args.Keys)
for _, err := range errs {
res.Errors = append(res.Errors, err.Error())
res.Errors = make([]string, len(errs))
for i, err := range errs {
if err != nil {
res.Errors[i] = err.Error()
}
Comment on lines +58 to +62
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes a bug discovered in a newly added test. If the error was nil, then the previous code triggered a panic.

}
return nil
}
6 changes: 2 additions & 4 deletions auth/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ import (
"errors"
"fmt"
"os"

"github.com/ava-labs/avalanchego/utils/wrappers"
)

var (
Expand All @@ -17,8 +15,8 @@ var (
)

// WithDefaultPrivateKeyFactories registers the default PrivateKeyFactories
func WithDefaultPrivateKeyFactories(authProvider *AuthProvider, errs *wrappers.Errs) {
errs.Add(
func WithDefaultPrivateKeyFactories(authProvider *AuthProvider) error {
return errors.Join(
authProvider.Register(ED25519Key, NewED25519PrivateKeyFactory()),
authProvider.Register(Secp256r1Key, NewSECP256R1PrivateKeyFactory()),
authProvider.Register(BLSKey, NewBLSPrivateKeyFactory()),
Expand Down
38 changes: 0 additions & 38 deletions chain/accepter.go

This file was deleted.

57 changes: 0 additions & 57 deletions chain/assembler.go

This file was deleted.

4 changes: 2 additions & 2 deletions chain/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ func (b *Base) Execute(r Rules, timestamp int64) error {
// TODO: make this modulus configurable
return fmt.Errorf("%w: timestamp=%d", ErrMisalignedTime, b.Timestamp)
case b.Timestamp < timestamp: // tx: 100 block: 110
return ErrTimestampTooLate
return fmt.Errorf("%w: tx timestamp (%d) < block timestamp (%d)", ErrTimestampTooLate, b.Timestamp, timestamp)
case b.Timestamp > timestamp+r.GetValidityWindow(): // tx: 100 block 10
return ErrTimestampTooEarly
return fmt.Errorf("%w: tx timestamp (%d) > block timestamp (%d) + validity window (%d)", ErrTimestampTooEarly, b.Timestamp, timestamp, r.GetValidityWindow())
Comment on lines +38 to +40
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added details while debugging newly added tests. These errors are much more helpful with the added information.

case b.ChainID != r.GetChainID():
return ErrInvalidChainID
default:
Expand Down
47 changes: 25 additions & 22 deletions chain/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ import (
"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/avalanchego/trace"
"github.com/ava-labs/avalanchego/utils/logging"
"github.com/ava-labs/avalanchego/x/merkledb"
"go.opentelemetry.io/otel/attribute"
"go.uber.org/zap"

Expand Down Expand Up @@ -45,8 +44,6 @@ func HandlePreExecute(log logging.Logger, err error) bool {
return true
case errors.Is(err, ErrTimestampTooLate):
return false
case errors.Is(err, ErrInvalidBalance):
return false
case errors.Is(err, ErrAuthNotActivated):
return false
case errors.Is(err, ErrAuthFailed):
Expand Down Expand Up @@ -96,16 +93,19 @@ func NewBuilder(
}
}

func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent *ExecutionBlock) (*ExecutionBlock, *ExecutedBlock, merkledb.View, error) {
func (c *Builder) BuildBlock(ctx context.Context, parentOutputBlock *OutputBlock) (*ExecutionBlock, *OutputBlock, error) {
ctx, span := c.tracer.Start(ctx, "Chain.BuildBlock")
defer span.End()

parent := parentOutputBlock.ExecutionBlock
parentView := parentOutputBlock.View

// Select next timestamp
nextTime := time.Now().UnixMilli()
r := c.ruleFactory.GetRules(nextTime)
if nextTime < parent.Tmstmp+r.GetMinBlockGap() {
c.log.Debug("block building failed", zap.Error(ErrTimestampTooEarly))
return nil, nil, nil, ErrTimestampTooEarly
return nil, nil, fmt.Errorf("%w: proposed build block time (%d) < parentTimestamp (%d) + minBlockGap (%d)", ErrTimestampTooEarly, nextTime, parent.Tmstmp, r.GetMinBlockGap())
}
var (
parentID = parent.ID()
Expand All @@ -118,12 +118,12 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent
feeKey := FeeKey(c.metadataManager.FeePrefix())
feeRaw, err := parentView.GetValue(ctx, feeKey)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
parentFeeManager := fees.NewManager(feeRaw)
feeManager, err := parentFeeManager.ComputeNext(nextTime, r)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
maxUnits := r.GetMaxBlockUnits()
targetUnits := r.GetWindowTargetUnits()
Expand Down Expand Up @@ -366,7 +366,7 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent
c.log.Debug("transactions restored to mempool", zap.Int("count", restored))
}()
c.log.Warn("build failed", zap.Error(execErr))
return nil, nil, nil, execErr
return nil, nil, execErr
}
break
}
Expand All @@ -392,7 +392,7 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent
// Perform basic validity checks to make sure the block is well-formatted
if len(blockTransactions) == 0 {
if nextTime < parent.Tmstmp+r.GetMinEmptyBlockGap() {
return nil, nil, nil, fmt.Errorf("%w: allowed in %d ms", ErrNoTxs, parent.Tmstmp+r.GetMinEmptyBlockGap()-nextTime) //nolint:spancheck
return nil, nil, fmt.Errorf("%w: allowed in %d ms", ErrNoTxs, parent.Tmstmp+r.GetMinEmptyBlockGap()-nextTime) //nolint:spancheck
}
c.metrics.emptyBlockBuilt.Inc()
}
Expand All @@ -414,27 +414,27 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent
feeKeyStr: parentFeeManager.Bytes(),
})
if err := tsv.Insert(ctx, heightKey, binary.BigEndian.AppendUint64(nil, height)); err != nil {
return nil, nil, nil, fmt.Errorf("%w: unable to insert height", err)
return nil, nil, fmt.Errorf("%w: unable to insert height", err)
}
if err := tsv.Insert(ctx, timestampKey, binary.BigEndian.AppendUint64(nil, uint64(timestamp))); err != nil {
return nil, nil, nil, fmt.Errorf("%w: unable to insert timestamp", err)
return nil, nil, fmt.Errorf("%w: unable to insert timestamp", err)
}
if err := tsv.Insert(ctx, feeKey, feeManager.Bytes()); err != nil {
return nil, nil, nil, fmt.Errorf("%w: unable to insert fees", err)
return nil, nil, fmt.Errorf("%w: unable to insert fees", err)
}
tsv.Commit()

// Fetch [parentView] root as late as possible to allow
// for async processing to complete
parentStateRoot, err := parentView.GetMerkleRoot(ctx)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}

// Get view from [tstate] after writing all changed keys
view, err := ts.ExportMerkleDBView(ctx, c.tracer, parentView)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}

// Initialize finalized metadata fields
Expand All @@ -446,7 +446,7 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent
parentStateRoot,
)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}

// Kickoff root generation
Expand Down Expand Up @@ -478,12 +478,15 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent
)
execBlock, err := NewExecutionBlock(blk)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
return execBlock, &ExecutedBlock{
Block: blk,
Results: results,
UnitPrices: feeManager.UnitPrices(),
UnitsConsumed: feeManager.UnitsConsumed(),
}, view, nil
return execBlock, &OutputBlock{
ExecutionBlock: execBlock,
View: view,
ExecutionResults: ExecutionResults{
Results: results,
UnitPrices: feeManager.UnitPrices(),
UnitsConsumed: feeManager.UnitsConsumed(),
},
}, nil
}
21 changes: 5 additions & 16 deletions chain/chain.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ type ValidityWindow = *validitywindow.TimeValidityWindow[*Transaction]
type Chain struct {
builder *Builder
processor *Processor
accepter *Accepter
preExecutor *PreExecutor
blockParser *BlockParser
}
Expand Down Expand Up @@ -69,11 +68,6 @@ func NewChain(
metrics,
config,
),
accepter: NewAccepter(
tracer,
validityWindow,
metrics,
),
preExecutor: NewPreExecutor(
ruleFactory,
validityWindow,
Expand All @@ -84,15 +78,15 @@ func NewChain(
}, nil
}

func (c *Chain) BuildBlock(ctx context.Context, parentView state.View, parent *ExecutionBlock) (*ExecutionBlock, *ExecutedBlock, merkledb.View, error) {
return c.builder.BuildBlock(ctx, parentView, parent)
func (c *Chain) BuildBlock(ctx context.Context, parentOutputBlock *OutputBlock) (*ExecutionBlock, *OutputBlock, error) {
return c.builder.BuildBlock(ctx, parentOutputBlock)
}

func (c *Chain) Execute(
ctx context.Context,
parentView state.View,
parentView merkledb.View,
b *ExecutionBlock,
) (*ExecutedBlock, merkledb.View, error) {
) (*OutputBlock, error) {
return c.processor.Execute(ctx, parentView, b)
}

Expand All @@ -103,18 +97,13 @@ func (c *Chain) AsyncVerify(
return c.processor.AsyncVerify(ctx, b)
}

func (c *Chain) AcceptBlock(ctx context.Context, blk *ExecutionBlock) error {
return c.accepter.AcceptBlock(ctx, blk)
}

func (c *Chain) PreExecute(
ctx context.Context,
parentBlk *ExecutionBlock,
view state.View,
tx *Transaction,
verifyAuth bool,
) error {
return c.preExecutor.PreExecute(ctx, parentBlk, view, tx, verifyAuth)
return c.preExecutor.PreExecute(ctx, parentBlk, view, tx)
}

func (c *Chain) ParseBlock(ctx context.Context, bytes []byte) (*ExecutionBlock, error) {
Expand Down
Loading
Loading