Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPC for staking txns and txns history queries #2554

Merged
merged 9 commits into from
Apr 2, 2020

Conversation

denniswon
Copy link
Contributor

@denniswon denniswon commented Mar 20, 2020

Issue

NOTE: This PR is rebased on my other PR #2553.

#2227
#2346

Test

~/go/src/github.com/harmony-one/harmony (stakingtxn_history) $ go test ./core/rawdb
ok github.com/harmony-one/harmony/core/rawdb

Test/Run Logs

Run local network with explorer node, make a query to its rpc query endpoint, and verify that getStakingTransactionsHistory() RPC returns the correct history of staking txns of an account.

Operational Checklist

Requires rolling update. Other than this, not much risk involved/foreseen.

@denniswon denniswon requested review from rlan35, fxfactorial, Daniel-VDM and a team March 20, 2020 14:16
@denniswon denniswon self-assigned this Mar 20, 2020
@fxfactorial fxfactorial removed their request for review March 22, 2020 16:50
@denniswon denniswon force-pushed the stakingtxn_history branch 5 times, most recently from 189041e to cf4d4a0 Compare March 24, 2020 23:54
@denniswon
Copy link
Contributor Author

denniswon commented Mar 24, 2020

Note. With force push above, #2553 has been merged to this pr. @gupadhyaya I made the changes based on your review on that pr here on commit
58e8692.

@denniswon denniswon changed the title RPC added and node explorer rawdb update for staking txn history support RPC for staking txns and txns history queries Mar 24, 2020
@denniswon denniswon requested a review from gupadhyaya March 25, 2020 00:00
@denniswon denniswon force-pushed the stakingtxn_history branch from cf4d4a0 to 30ecb35 Compare March 25, 2020 06:35
@denniswon denniswon force-pushed the stakingtxn_history branch from 30ecb35 to ad6f3b7 Compare March 26, 2020 01:00
internal/hmyapi/apiv2/transactionpool.go Outdated Show resolved Hide resolved
internal/hmyapi/apiv2/transactionpool.go Outdated Show resolved Hide resolved
@stephen-tse
Copy link
Collaborator

@rlan35 is this pr blocking stake heist? if so, let's expedite the review and merge? cc @LeoHChen

internal/hmyapi/apiv2/transactionpool.go Outdated Show resolved Hide resolved
internal/hmyapi/apiv2/transactionpool.go Outdated Show resolved Hide resolved
internal/hmyapi/apiv2/transactionpool.go Outdated Show resolved Hide resolved
@denniswon denniswon force-pushed the stakingtxn_history branch from aaf0b02 to 0d89daa Compare March 30, 2020 22:23
@denniswon
Copy link
Contributor Author

@gupadhyaya updated per review.

@denniswon
Copy link
Contributor Author

denniswon commented Mar 31, 2020

{"level":"debug","port":"9000","ip":"18.144.51.20","blockHeight":2827,"caller":"/home/ubuntu/harmony-one/harmony/node/node_explorer.go:         109",
"time":"2020-03-31T01:10:36.033816346Z","message":"[Explorer] Adding new block for explorer node"}                                   30259 
{"level":"error","port":"9000","ip":"18.144.51.20","caller":"/home/ubuntu/harmony-one/harmony/core/blockchain.go:1530",
"time":"2020-03-         31T01:10:36.034274485Z","message":"\n########## BAD BLOCK #########\nChain config: {ChainID: 3 EIP155: 0 CrossTx: 0 Staking: 2                  CrossLink: 2 ReceiptLog: 0}\n\nNumber: 2827\nEpoch: 74\nNumTxn: 0\nNumStkTxn: 0\nHash:                                                          0x638ffb055fb76dbfb00359eaa1149c857245d18b2907b23238d62a1ccf43ecf4\n\n\nError: unknown ancestor\n##############################\n"}       30260 
{"level":"error","port":"9000","ip":"18.144.51.20","error":"unknown ancestor","caller":"/home/ubuntu/harmony-one/harmony/node/                  node_explorer.go:130","time":"2020-03-31T01:10:36.034290031Z","message":"[Explorer] Error when adding new block for explorer node"}       30261 
{"level":"info","port":"9000","ip":"18.144.51.20","blockNum":2827,"caller":"/home/ubuntu/harmony-one/harmony/node/node_explorer.go:140", 
"time":"2020-03-31T01:10:36.034322476Z","message":"[Explorer] Committing block into explorer DB"}
30262 {"level":"info","port":"9000","ip":"18.144.51.20","peer":{"Addrs":["/ip4/127.0.0.1/tcp/9000","/ip4/172.31.21.25/tcp/9000","/ip4/54.193.         17.25/tcp/9000"],"ID":"QmPF9b4xV6uUQcf3RQJanv2Z32BuWPWFNXRpv6t2Lq8fmZ"},"caller":"/home/ubuntu/harmony-one/harmony/api/service/networkinfo/service.go:237","time":"2020-03-31T01:10:36.056615722Z","message":"connected to peer node"}
30263 {"level":"info","port":"9000","ip":"18.144.51.20","peer":{"IP":"54.193.17.25","Port":"9000","ConsensusPubKey":null,"Addrs":["/ip4/127.0.        0.1/tcp/9000","/ip4/172.31.21.25/tcp/9000","/ip4/54.193.17.25/tcp/9000"],
"PeerID":"QmPF9b4xV6uUQcf3RQJanv2Z32BuWPWFNXRpv6t2Lq8fmZ"},"caller":"/home/ubuntu/harmony-one/harmony/api/service/networkinfo/service.go:255","time":"2020-03-31T01:10:36.056673469Z","message":"Notify peerChan"}

But I do see the new rpcs getting called correctly

{"level":"info","port":"9000","ip":"18.144.51.20","caller":"/home/ubuntu/harmony-one/harmony/node/node_explorer.go:185","time":"2020-03-        31T01:11:05.817062947Z","message":"[explorer] getstakingtransactionshistory"}                                                             30446 {"level":"error","port":"9000","ip":"18.144.51.20","error":"leveldb: not found","caller":"/home/ubuntu/harmony-one/harmony/node/                node_explorer.go:191","time":"2020-03-31T01:11:05.817096039Z","message":"[Explorer] Cannot get storage db instance"}                      30447 
....
 30452 {"level":"info","port":"9000","ip":"18.144.51.20","caller":"/home/ubuntu/harmony-one/harmony/node/node_explorer.go:185","time":"2020-03-        31T01:11:07.0790898Z","message":"[explorer] getstakingtransactionshistory"}                                                               30453 {"level":"error","port":"9000","ip":"18.144.51.20","error":"leveldb: not found","caller":"/home/ubuntu/harmony-one/harmony/node/                node_explorer.go:191","time":"2020-03-31T01:11:07.079125703Z","message":"[Explorer] Cannot get storage db instance"}   

On my own explorer node, having some trouble inserting new blocks to the db

@denniswon
Copy link
Contributor Author

denniswon commented Mar 31, 2020

if !node.Consensus.Decider.IsQuorumAchievedByMask(mask) {
			utils.Logger().Error().Msg("[Explorer] not have enough signature power")
			return
		}

I also see this when my explorer node is syncing.

cc: @fxfactorial any idea by chance?

[UPDATE] Resolved later during syncing. I guess this happens after hard refresh when not enough nodes are there on the network.

@gupadhyaya
Copy link
Contributor

@denniswon could you please fix the Jenkins build?

@denniswon
Copy link
Contributor Author

@gupadhyaya jenkins has been disabled.

@denniswon denniswon force-pushed the stakingtxn_history branch from 87d3a5b to 3649ffc Compare April 2, 2020 09:08
@denniswon denniswon force-pushed the stakingtxn_history branch from f181d32 to c8df6e3 Compare April 2, 2020 10:01
@denniswon
Copy link
Contributor Author

@gupadhyaya reverted the apiv1 change in transactionpool and jenkins/travis all passed. final review to be merged so that this pr can be included in tmrw's ostn upgrade.

Copy link
Contributor

@gupadhyaya gupadhyaya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@rlan35 rlan35 merged commit b9843ab into harmony-one:master Apr 2, 2020
fxfactorial added a commit that referenced this pull request Apr 2, 2020
* Added support for using AWS encrypted BLS key files (#2650)

* added bls support, cleaned the code

* fixed code per review

* [build] fix the release action for static build

Signed-off-by: Leo Chen <leo@harmony.one>

* [build] remove double / in FOLDER

Signed-off-by: Leo Chen <leo@harmony.one>

* [blsgen] LoadFromKey with passphrase more robustly (#2675)

* [project] Remove txgen, prefer stack based init & map literals (#2678)

* [project] Remove unused internal memprofiling, profiling, metrics service (#2679)

* [project] Remove unused internal memprofiling, profiling

* [internal] Unused attack package

* [internal] More dead metrics related code

* [offchain] Forgot removal of ! which did not write the reward-per-key to work (#2681)

* [project] Fix first round issue from static analysis check (#2682)

* RPC for staking txns and txns history queries (#2554)

* staking txn. look up by hash fix on api backend rawdb storage

* node explorer staking txn 'history' RPC layer support

* fix unit test

* add error log when explorer node db instance cannot be fetched

* revert unwanted merge changes during rebase

* use already encoded tx message fields for get staking txn rpc

* update explorer node storage service for staking txns

* use hex string for staking transaction data field

* revert transaction pool apiv1 changes

* [blockchain] More info in failed validator voting power update (#2685)

Co-authored-by: coolcottontail <50373379+coolcottontail@users.noreply.github.com>
Co-authored-by: Leo Chen <leo@harmony.one>
Co-authored-by: Edgar Aroutiounian <edgar.factorial@gmail.com>
Co-authored-by: Jong Hyuck Won <denniswon@users.noreply.github.com>
fxfactorial added a commit that referenced this pull request Apr 4, 2020
* Merge master t3 0402 (#2686)

* Added support for using AWS encrypted BLS key files (#2650)

* added bls support, cleaned the code

* fixed code per review

* [build] fix the release action for static build

Signed-off-by: Leo Chen <leo@harmony.one>

* [build] remove double / in FOLDER

Signed-off-by: Leo Chen <leo@harmony.one>

* [blsgen] LoadFromKey with passphrase more robustly (#2675)

* [project] Remove txgen, prefer stack based init & map literals (#2678)

* [project] Remove unused internal memprofiling, profiling, metrics service (#2679)

* [project] Remove unused internal memprofiling, profiling

* [internal] Unused attack package

* [internal] More dead metrics related code

* [offchain] Forgot removal of ! which did not write the reward-per-key to work (#2681)

* [project] Fix first round issue from static analysis check (#2682)

* RPC for staking txns and txns history queries (#2554)

* staking txn. look up by hash fix on api backend rawdb storage

* node explorer staking txn 'history' RPC layer support

* fix unit test

* add error log when explorer node db instance cannot be fetched

* revert unwanted merge changes during rebase

* use already encoded tx message fields for get staking txn rpc

* update explorer node storage service for staking txns

* use hex string for staking transaction data field

* revert transaction pool apiv1 changes

* [blockchain] More info in failed validator voting power update (#2685)

Co-authored-by: coolcottontail <50373379+coolcottontail@users.noreply.github.com>
Co-authored-by: Leo Chen <leo@harmony.one>
Co-authored-by: Edgar Aroutiounian <edgar.factorial@gmail.com>
Co-authored-by: Jong Hyuck Won <denniswon@users.noreply.github.com>

* [validator] Hide one field from JSON (#2705)

* [rpc] Show both latest header of beacon chain and shard chain (#2714)

Co-authored-by: Leo Chen <leo@harmony.one>
Co-authored-by: coolcottontail <50373379+coolcottontail@users.noreply.github.com>
Co-authored-by: Edgar Aroutiounian <edgar.factorial@gmail.com>
Co-authored-by: Jong Hyuck Won <denniswon@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants