Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IIP-13] implement liquid staking indexer #3853

Closed
wants to merge 54 commits into from

Conversation

envestcc
Copy link
Member

@envestcc envestcc commented Apr 27, 2023

Description

This pr implements a new indexer that enables the querying of liquid staking state natively. This is a necessary step before combining native staking with liquid staking. The new indexer includes:

  • receive contract events to sync liquid staking state
  • export interface to query liquid staking state

Clickup-task

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • make test
  • [] fullsync
  • [] Other test (please specify)

Test Configuration:

  • Firmware version:
  • Hardware:
  • Toolchain:
  • SDK:

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

blockindex/liquidstaking_indexer.go Outdated Show resolved Hide resolved
blockindex/liquidstaking_indexer.go Outdated Show resolved Hide resolved
blockindex/liquidstaking_indexer.go Outdated Show resolved Hide resolved
blockindex/liquidstaking_indexer.go Outdated Show resolved Hide resolved
blockdao.BlockIndexer

CandidateVotes(candidate address.Address) *big.Int
Buckets() ([]*Bucket, error)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what buckets are this interface returned? unknown from the naming

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ContractStakingBucket maybe better

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, I mean the func name is blurry

if err := s.kvstore.Stop(ctx); err != nil {
return err
}
return nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the cache be reset here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strictly speaking, it is recommended to reset the cache upon Stop to ensure proper functioning when it is re-Start, even though it is not actually utilized in that manner.

// new dirty cache for this block
// it's not necessary to use thread safe cache here, because only one thread will call this function
// and no update to cache will happen before dirty merge to clean
dirty := newLiquidStakingDirty(s.cache.cache)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bad naming for s.cache.cache

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s.cache.unsafe() may be better

if receipt.Status != uint64(iotextypes.ReceiptStatus_Success) {
continue
}
act, ok := actionMap[receipt.ActionHash]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dustinxie are actions and receipts in order in the block. Do we need a hashmap to get the receipt of the tx

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These codes are no longer needed since the removal of the unused 'act' argument.

func (s *liquidStakingIndexer) Buckets() ([]*Bucket, error) {
vbs := []*Bucket{}
for id, bi := range s.cache.getAllBucketInfo() {
bt := s.cache.mustGetBucketType(bi.TypeIndex)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The race condition isn't solved by a threadsafe map. The issue still exists if Putblock() is executed when another thread is running in the loop.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls check whether a mutex is necessary for liquidStakingIndexer class

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, multiple cache reads before and after the commit still require locking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The improvement looks good for me. But one mutex for this module is enough

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the mutex in cache

return s.cache.getActiveBucketType(), nil
}

func (s *liquidStakingIndexer) handleEvent(ctx context.Context, dirty *liquidStakingDirty, blk *block.Block, act *action.SealedEnvelope, log *action.Log) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the purpose of act as one of the arguments?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

act has been an unused argument and should be removed

case "Transfer":
return dirty.handleTransferEvent(event)
default:
return nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should inconsistency be ignored here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question, events that do not require handling should be clearly labeled and return nil, while unexpected events should return error.

Comment on lines 661 to 666
if err := s.kvstore.WriteBatch(batch); err != nil {
return err
}
if err := s.cache.merge(delta); err != nil {
return err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should acid be satified here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, i think two changes need to do:

  1. move cache.merge before kvstore.WriteBatch
  2. reloadCache when error occured

Comment on lines +44 to +46
Index uint64
Candidate address.Address
Owner address.Address
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: different values with those in bucketinfo?

Copy link
Member Author

@envestcc envestcc May 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ContractStakingBucket is corresponding to staking.VoteBucket;
and type ContractStakingBucket = staking.VoteBucket will be added in pr 3860


type (
// ContractStakingIndexer is the interface of contract staking indexer
ContractStakingIndexer interface {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add liquidstaking_indexer_test.go for ContractStakingIndexer?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

already added tests in e2etest/liquid_staking_test.go

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also added unittests in liquidstaking_indexer_test.go

@sonarcloud
Copy link

sonarcloud bot commented May 18, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 12 Code Smells

No Coverage information No Coverage information
5.8% 5.8% Duplication

@envestcc
Copy link
Member Author

This pr is too large to review, so it has been splited into three smaller pr: #3861 , #3862 , #3863

@envestcc envestcc closed this May 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants