From 3f71e3b6760d9782e9258f0b6a19eb9943fe842a Mon Sep 17 00:00:00 2001 From: marcello33 Date: Wed, 10 May 2023 22:52:14 +0200 Subject: [PATCH] mardizzone/merge-qa-to-master (#856) * Added checks to RPC requests and introduced new flags to customise the parameters (#657) * added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit) * added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit) * updated the config files and docs * chg : trieTimeout from 60 to 10 mins (#692) * chg : trieTimeout from 60 to 10 mins * chg : cache.timout to 10m from 1h in configs * internal/cli/server : fix : added triesInMemory in config (#691) * changed version from 0.3.0 to 0.3.4-beta (#693) * fix nil state-sync issue, increase grpc limit (#695) * Increase grpc message size limit in pprof * consensus/bor/bor.go : stateSyncs init fixed [Fix #686] * eth/filters: handle nil state-sync before notify * eth/filters: update check Co-authored-by: Jerry Co-authored-by: Daniil * core, tests/bor: add more tests for state-sync validation (#710) * core: add get state sync function for tests * tests/bor: add validation for state sync events post consensus * Arpit/temp bor sync (#701) * Increase grpc message size limit in pprof * ReadBorReceipts improvements * use internal function * fix tests * fetch geth upstread for ReadBorReceiptRLP * Only query bor receipt when the query index is equal to # tx in block body This change reduces the frequency of calling ReadBorReceipt and ReadBorTransaction, which are CPU and db intensive. * Revert "fetch geth upstread for ReadBorReceiptRLP" This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0. * Restore ReadBorReceiptRLP * fix bor receipts * remove unused * fix lints --------- Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720) This reverts commit 241843c7e7bb18e64d2e157fd6fbbd665f6ce9d9. * Arpit/add execution pool 2 (#719) * initial * linters * linters * remove timeout * update pool * change pool size function * check nil * check nil * fix tests * Use execution pool from server in all handlers * simplify things * test fix * add support for cli, config * add to cli and config * merge base branch * debug statements * fix bug * atomic pointer timeout * add apis * update workerpool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * fix tests * mutex * refactor flag and value names * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * debug statements * fix bug * update workerpool * atomic pointer timeout * add apis * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * merge base branch * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * mutex * fix tests * Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * Change default size of execution pool to 40 * refactor flag and value names * fix merge conflicts * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * fix linters * fix go.mod * change sec to ms * change default value for ep timeout * fix node api calls * comment setter for ep timeout --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Jerry Co-authored-by: Manav Darji * version change (#721) * Event based pprof (#732) * feature * Save pprof to /tmp --------- Co-authored-by: Jerry * Cherry-pick changes from develop (#738) * Check if block is nil to prevent panic (#736) * miner: use env for tracing instead of block object (#728) --------- Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * add max code init size check in txpool (#739) * Revert "Event based pprof" and update version (#742) * Revert "Event based pprof (#732)" This reverts commit 22fa4033e8fabb51c44e8d2a8c6bb695a6e9285e. * params: update version to 0.3.4-beta3 * packaging/templates: update bor version * internal/ethapi :: Fix : newRPCTransactionFromBlockIndex * Merge master to qa (#813) * Merge qa to master (#750) * Added checks to RPC requests and introduced new flags to customise the parameters (#657) * added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit) * added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit) * updated the config files and docs * chg : trieTimeout from 60 to 10 mins (#692) * chg : trieTimeout from 60 to 10 mins * chg : cache.timout to 10m from 1h in configs * internal/cli/server : fix : added triesInMemory in config (#691) * changed version from 0.3.0 to 0.3.4-beta (#693) * fix nil state-sync issue, increase grpc limit (#695) * Increase grpc message size limit in pprof * consensus/bor/bor.go : stateSyncs init fixed [Fix #686] * eth/filters: handle nil state-sync before notify * eth/filters: update check Co-authored-by: Jerry Co-authored-by: Daniil * core, tests/bor: add more tests for state-sync validation (#710) * core: add get state sync function for tests * tests/bor: add validation for state sync events post consensus * Arpit/temp bor sync (#701) * Increase grpc message size limit in pprof * ReadBorReceipts improvements * use internal function * fix tests * fetch geth upstread for ReadBorReceiptRLP * Only query bor receipt when the query index is equal to # tx in block body This change reduces the frequency of calling ReadBorReceipt and ReadBorTransaction, which are CPU and db intensive. * Revert "fetch geth upstread for ReadBorReceiptRLP" This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0. * Restore ReadBorReceiptRLP * fix bor receipts * remove unused * fix lints --------- Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720) This reverts commit 241843c7e7bb18e64d2e157fd6fbbd665f6ce9d9. * Arpit/add execution pool 2 (#719) * initial * linters * linters * remove timeout * update pool * change pool size function * check nil * check nil * fix tests * Use execution pool from server in all handlers * simplify things * test fix * add support for cli, config * add to cli and config * merge base branch * debug statements * fix bug * atomic pointer timeout * add apis * update workerpool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * fix tests * mutex * refactor flag and value names * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * debug statements * fix bug * update workerpool * atomic pointer timeout * add apis * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * merge base branch * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * mutex * fix tests * Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * Change default size of execution pool to 40 * refactor flag and value names * fix merge conflicts * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * fix linters * fix go.mod * change sec to ms * change default value for ep timeout * fix node api calls * comment setter for ep timeout --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Jerry Co-authored-by: Manav Darji * version change (#721) * Event based pprof (#732) * feature * Save pprof to /tmp --------- Co-authored-by: Jerry * Cherry-pick changes from develop (#738) * Check if block is nil to prevent panic (#736) * miner: use env for tracing instead of block object (#728) --------- Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * add max code init size check in txpool (#739) * Revert "Event based pprof" and update version (#742) * Revert "Event based pprof (#732)" This reverts commit 22fa4033e8fabb51c44e8d2a8c6bb695a6e9285e. * params: update version to 0.3.4-beta3 * packaging/templates: update bor version * params, packaging/templates: update bor version --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Pratik Patil Co-authored-by: Jerry Co-authored-by: Daniil Co-authored-by: Arpit Temani Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * core, miner: add sub-spans for tracing (#753) * core, miner: add sub-spans for tracing * fix linters * core: add logs for debugging * core: add more logs to print tdd while reorg * fix linters * core: minor fix * core: remove debug logs * core: use different span for write block and set head * core: use internal context for sending traces (#755) * core: add : impossible reorg block dump (#754) * add : impossible reorg block dump * chg : 3 seperate files for impossoble reorg dump * add : use exportBlocks method and RLP blocks before writing * chg : small changes * bump : go version from 1.19 to 1.20.1 (#761) * Revert "bump : go version from 1.19 to 1.20.1 (#761)" This reverts commit 4561012af9a31d20c2715ce26e4b39ca93420b8b. * core/vm: use optimized bigint (#26021) * Add holiman/big * Fix linter * Bump version to v0.3.5 * fix lints from develop (few lints decided to appear from code that was untouched, weird) * upgrade crypto lib version (#770) * bump dep : github.com/Masterminds/goutils to v1.1.1 (#769) * mardizzone/pos-1313: bump crypto dependency (#772) * dev: chg: bumd net dependency * dev: chg: bump crypto dependency * dev: chg: bump crypto dependency * bump dep : golang.org/x/net to v0.8.0 (#771) * Verify validator set against local contract on receiving an end-of-sprint block (#768) * Verify validator set against local contract on receiving an end-of-sprint block * Fix tests * Respect error returned by ParseValidators * Keep going back until a parent block presents * core/txpool: implement DoS defenses from geth (#778) * Hotfixes and deps bump (#776) * dev: chg: bump deps * internal/cli/server, rpc: lower down http readtimeout to 10s * dev: chg: get p2p adapter * dev: chg: lower down jsonrpc readtimeout to 10s * cherry-pick txpool optimisation changes * add check for empty lists in txpool (#704) * add check * linters * core, miner: add empty instrumentation name for tracing --------- Co-authored-by: Raneet Debnath Co-authored-by: SHIVAM SHARMA Co-authored-by: Evgeny Danilenko <6655321@bk.ru> Co-authored-by: Manav Darji * packaging,params: bump to v0.3.6 (#782) * v0.3.6 fix (#787) * Fix get validator set in header verifier * chg : commit tx logs from info to debug (#673) * chg : commit tx logs from info to debug * fix : minor changes * chg : miner : commitTransactions-stats moved from info to debug * lint : fix linters * refactor logging * miner : chg : UnauthorizedSignerError to debug * lint : fix lint * fix : log.Logger interface compatibility --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Remove unnecessary sorting of valset from header in verification * dev: chg: version bump --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: marcello33 * core: improve locks in txpool (#807) * added a write lock to the txs.filter method and a read lock to the txs.reheap method - both of which are called by Filter during reorg adjustments to txpool * txpool reorg locks * more locks * locks * linters * params, packaging: update version for v0.3.8-beta release * core: add logs in reheap --------- Co-authored-by: Alex Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Merge qa to master (#808) * Added checks to RPC requests and introduced new flags to customise the parameters (#657) * added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit) * added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit) * updated the config files and docs * chg : trieTimeout from 60 to 10 mins (#692) * chg : trieTimeout from 60 to 10 mins * chg : cache.timout to 10m from 1h in configs * internal/cli/server : fix : added triesInMemory in config (#691) * changed version from 0.3.0 to 0.3.4-beta (#693) * fix nil state-sync issue, increase grpc limit (#695) * Increase grpc message size limit in pprof * consensus/bor/bor.go : stateSyncs init fixed [Fix #686] * eth/filters: handle nil state-sync before notify * eth/filters: update check Co-authored-by: Jerry Co-authored-by: Daniil * core, tests/bor: add more tests for state-sync validation (#710) * core: add get state sync function for tests * tests/bor: add validation for state sync events post consensus * Arpit/temp bor sync (#701) * Increase grpc message size limit in pprof * ReadBorReceipts improvements * use internal function * fix tests * fetch geth upstread for ReadBorReceiptRLP * Only query bor receipt when the query index is equal to # tx in block body This change reduces the frequency of calling ReadBorReceipt and ReadBorTransaction, which are CPU and db intensive. * Revert "fetch geth upstread for ReadBorReceiptRLP" This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0. * Restore ReadBorReceiptRLP * fix bor receipts * remove unused * fix lints --------- Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720) This reverts commit 241843c7e7bb18e64d2e157fd6fbbd665f6ce9d9. * Arpit/add execution pool 2 (#719) * initial * linters * linters * remove timeout * update pool * change pool size function * check nil * check nil * fix tests * Use execution pool from server in all handlers * simplify things * test fix * add support for cli, config * add to cli and config * merge base branch * debug statements * fix bug * atomic pointer timeout * add apis * update workerpool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * fix tests * mutex * refactor flag and value names * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * debug statements * fix bug * update workerpool * atomic pointer timeout * add apis * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * merge base branch * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * mutex * fix tests * Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * Change default size of execution pool to 40 * refactor flag and value names * fix merge conflicts * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * fix linters * fix go.mod * change sec to ms * change default value for ep timeout * fix node api calls * comment setter for ep timeout --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Jerry Co-authored-by: Manav Darji * version change (#721) * Event based pprof (#732) * feature * Save pprof to /tmp --------- Co-authored-by: Jerry * Cherry-pick changes from develop (#738) * Check if block is nil to prevent panic (#736) * miner: use env for tracing instead of block object (#728) --------- Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * add max code init size check in txpool (#739) * Revert "Event based pprof" and update version (#742) * Revert "Event based pprof (#732)" This reverts commit 22fa4033e8fabb51c44e8d2a8c6bb695a6e9285e. * params: update version to 0.3.4-beta3 * packaging/templates: update bor version * internal/ethapi :: Fix : newRPCTransactionFromBlockIndex * fix: remove assignment for bor receipt --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Pratik Patil Co-authored-by: Jerry Co-authored-by: Daniil Co-authored-by: Arpit Temani Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Pratik Patil Co-authored-by: Jerry Co-authored-by: Daniil Co-authored-by: Arpit Temani Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> Co-authored-by: Martin Holst Swende Co-authored-by: marcello33 Co-authored-by: Raneet Debnath Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com> Co-authored-by: Alex Co-authored-by: Daniel Jones * merge v0.3.9-alpha to qa (#824) * commit logs * CI: test launch devnet without hardcoded sleep time (pos-534) * CI: try using checked out bor path * CI: fix missing ; * CI: fix assignment operator * CI: echo peers and block no. * CI: cleanup * minor chg: add new line * dev: add: pos-944: snyk and govuln integration (#578) * dev: add: pos-944 security ci and readme * dev: add: pos-944 remove linters as this is included already in build ci * dev: chg: pos-947 dependencies upgrade to solve snyk security issues * dev: chg: update security-ci * dev: chg: remove linter to allow replacements for security issues * dev: add: pos-944 verify path when updating metrics from config * dev: add: pos-944 fix linter * dev: add: pos-944 add .snyk policy file / fix snyk code vulnerabilities * dev: fix: pos-944 import common package / gitignore snyk dccache file * dev: fix: pos-944 verify canonical path for crashers * dev: fix: pos-944 linter * dev: add: pos-976 add govuln check * dev: add: pos-976 test upload with permissions * dev: add: pos-976 remove duplicated upload * dev: add: pos-976 report upload * dev: add: pos-976 remove upload * dev: fix: pos-944 fix govuln action * dev: fix: pos-944 move govulncheck to security-ci * dev: fix: pos-944 bump golvun action and golang versions * dev: fix: pos-944 remove persmissions and fix conflicts * dev: chg: restore err msg * dev: chg: remove duplicated function * dev: chg: sort import * dev: chg: fix linter * dev: add: use common VerifyCrasher function to avoid duplications / replace deprecated ioutils.ReadFile * dev: fix: typo * fix linters * upgrade grpc version * add ignore rule for net/http2 * Shivam/txpool tracing (#604) * lock, unlock to rlock, runlock * add : tracing Pending() and Locals() * Log time spent in committing a tx during mining * Remove data from logging * Move log into case where a tx completes without error * profile fillTransactions * fix conflict * bug fixes * add logs * txpool: add tracing in Pending() * rearrange tracing * add attributes * fix * fix * log error in profiling * update file mode and file path for profiling * full profiling * fix log * fix log * less wait * fix * fix * logs * worker: use block number for prof files * initial * txList add * fix gas calculation * fix * green tests * linters * prettify * allocate less * no locks between pending and reorg * no locks * no locks on locals * more tests * linters * less allocs * comment * optimize errors * linters * fix * fix * Linters * linters * linters * simplify errors * atomics for transactions * fix * optimize workers * fix copy * linters * txpool tracing * linters * fix tracing * duration in mcs * locks * metrics * fix * cache hit/miss * less locks on evict * remove once * remove once * switch off pprof * fix data race * fix data race * add : sealed total/empty blocks metric gauge * add : RPC debug_getTraceStack * fix : RPC debug_getTraceStack * fix : RPC debug_getTraceStack for all go-routines * linters * add data race test on txpool * fix concurrency * noleak * increase batch size * prettify * tests * baseFee mutex * panic fix * linters * fix gas fee data race * linters * more transactions * debug * debug * fix ticker * fix test * add cacheMu * more tests * fix test panic * linters * add statistics * add statistics * txitems data race * fix tx list Has * fix : lint Co-authored-by: Arpit Temani Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Reduce txArriveTimeout to 100ms * init : remove exit on keystore err * add : multiple keystore tolerance * lint : fix linters * chg : use standard logging * chg : logging strings * Added flags to run heimdall as a child process (#597) * Added flags to run heimdall as a child process * Fix: Lint * Fix btcd package dependency for CI * Update btcd package version * Try removing ambigious importts * dev: fix: lint fix for parallel tests * remove delete for ambigious import * Remove unwanted space * go mod tidy * try replace * try replace * use vendor * rename vendor * tidy * vendor btcec * clean up * remove submodule * remove submodule * remove submodule * remove submodule * remove vendor & added replacer in test * go mod tidy * added replacer * Update replacer * Update replacer * Merge branch 'develop' into run-heimdall-flags * Skip TestGolangBindings * Typo fix * Remove unwanted changes Co-authored-by: marcello33 Co-authored-by: Evgeny Danienko <6655321@bk.ru> * dev: chg: update PR template to include nodes audience check (#641) * dev: chg: update PR template to include nodes audience check * dev: chg: better description * dev: chg: add entry to changes too * sonarqube integration (#658) * dev: add: sonarqube integration into security-ci * dev: add: exclude java files from sonarqube analysis * Merge branch 'qa' and 'master' into develop (#663) * Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come) * Added changes per discussion in PR, more changes may be necessary * Adding prerelease true * Disabling goreleaser * Removing README swap file * change bor_dir and add bor user for v0.3.0 release * rollback bor user and use root * metrics: handle equal to separated config flag (#596) * metrics: handle based config path * internal/cli/server: add more context to logs * use space separated flag and value in bor.service * fixed static-nodes related buf (os independent) (#598) * fixed static-nodes related buf (os independent) * taking static-nodes as input if default not present * Update default flags (#600) * internal/cli/server: use geth's default for txpool.pricelimit and add comments * builder/files: update config.toml for mainnet * packaging/templates: update defaults for mainnet and mumbai * internal/cli/server: skip overriding cache * packaging/templates: update cache value for mainnet * packaging/templates: update gcmode for archive mumbai node * metrics: handle nil telemetry config (#601) * resolve merge conflicts * update go version in release.yml * update goversion in makefile * update Docker login for goreleaser-cross v1.19 * Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist * added a toml configuration file with comments describing each flag (#607) * added a toml configuration file with comments describing each flag * internal/cli/server: update flag description * docs/cli: update example config and description of flags * docs: update new-cli docs Co-authored-by: Manav Darji * Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update * added ancient datadir flag and toml field, need to decide on default value and update the conversion script * updated toml files with ancient field * Add support for new flags in new config.toml, which were present in old config.toml (#612) * added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml * added RAW fields for these time.Duration flags * updated the conversion script to support these extra 4 flags * removed hcl and json config tests as we are only supporting toml config files * updated toml files with cache.timeout field * updated toml files with jsonrpc.timeouts field * tests/bor: expect a call for latest checkpoint * tests/bor: expect a call for latest checkpoint * packaging/templates: update cache values for archive nodes Co-authored-by: Manav Darji * remove unwanted code * Fix docker publish authentication issue In gorelease-cross 1.19+, dockerhub authentication will require docker logion action followed by mounting docker config file. See https://github.com/goreleaser/goreleaser-cross#github-actions. * Revert "update Docker login for goreleaser-cross v1.19" This reverts commit 4d19cf5342a439d98cca21b03c63a0bc075769cf. * Bump version to stable * Revert "Merge pull request #435 from maticnetwork/POS-553" This reverts commit 657d262defc9c94e9513b3d45230492d8b20eac7, reversing changes made to 88dbfa1c13c15464d3c1a3085a9f12d0ffb9b218. * revert change for release for go1.19 * Add default values to CLI helper and docs This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value. * Add a summary of new CLI in docs * Updating packager as binutils changed version so that apt-get installs current versions * Add state pruning to new CLI * Minor wording fix in prune state description * Bumping control file versions * Mainnet Delhi fork * Set version to stable * change delhi hardfork block number * handle future chain import and skip peer drop (#650) * handle future chain import and skip peer drop * add block import metric * params: bump version to v0.3.3-stable * Bump bor version in control files for v0.3.3 mainnet release Co-authored-by: Daniel Jones Co-authored-by: Will Button Co-authored-by: Will Button Co-authored-by: Manav Darji Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com> Co-authored-by: Pratik Patil Co-authored-by: Arpit Temani Co-authored-by: Jerry * CI: use matic-cli master branch * trigger ci * internal/cli/server : fix : added triesInMemory in config (#677) * update requirements in README (#681) * consensus/bor : add : devFakeAuthor flag * core,eth,internal/cli,internal/ethapi: add --rpc.allow-unprotected-txs flag to allow txs to get replayed (for shadow node) * internal/cli: add `skiptrace` flag for profiling (#715) * internal/cli: add skiptrace flag * docs: update docs for skiptrace flag * Added flag in Status to wait for backend, and fixed panic issue. (#708) * checking if backend is available during status call, and added a flag to wait if backend is not available * added test for status command (does not cover the whole code) * Revert "Reduce txArriveTimeout to 100ms" (#707) This reverts commit 243d231fe45bc02f33678bb4f69e941167d7f466. * consensus/bor : add : devFakeAuthor flag (#697) * add check for empty lists in txpool (#704) * add check * linters * dev: chg: POS-215 move sonarqube to own ci (#733) * Added verbosity flag, supports log-level as well, but will remove that in future. (#722) * changed log-level flag back to verbosity, and updated the conversion script * supporting both verbosity and log-level, with a message to deprecat log-level * converted verbosity to a int value * Check if block is nil to prevent panic (#736) * miner: use env for tracing instead of block object (#728) * Add : mutex pprof profile (#731) * add : mutex pprof profile * rm : remove trace from default pprof profiles * chg : commit tx logs from info to debug (#673) * chg : commit tx logs from info to debug * fix : minor changes * chg : miner : commitTransactions-stats moved from info to debug * lint : fix linters * refactor logging * miner : chg : UnauthorizedSignerError to debug * lint : fix lint * fix : log.Logger interface compatibility --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Add : commit details to bor version (#730) * add : commit details to bor version * fix : MAKEFILE * undo : rm test-txpool-race * rm : params/build_date * rm : params/gitbranch and params/gitdate * core,docs/cli,internal/cli/server: make docs * builder,docs/cli,packaging: update toml files * mardizzone/hotfix-snyk: remove vcs build when running snyk (#745) * Feat : SetMaxPeers (#726) * init : add admin.setMaxPeers and getMaxPeers * lint : fix lint * add : some comments * Update wiki link (#762) * Heimdall App implementation (#646) * added support for miner.recommit flag (#743) * interrupting commit experiment (#556) * initial * delete * linters * big benchmark * benchmark big ints * delay * fix generate * remove debug * miner : chg : remove noempty check * fix lints * consensus/bor: handle unauthorized signer in consensus.Prepare (#651) * fix : break loop fix * lint : fix * lint : more lint fix * fix : skip TestEmptyWorkEthash and TestEmptyWorkClique * add : metrics commitInterruptCounter --------- Co-authored-by: Shivam Sharma Co-authored-by: Arpit Temani Co-authored-by: Manav Darji * internal/cli: added missing flags (#744) * minor comment update * added support for rpc.evmtimeout flag * added support for vmdebug (EnablePreimageRecording) flag * added support for jsonrpc.auth.(jwtsecret, addr, port, vhosts) flags * added support for miner.recommit flag * added support for gpo.maxheaderhistory and gpo.maxblockhistory flag * Revert "added support for miner.recommit flag" This reverts commit fcb722e4bf5bdda23d4c91da40ab95c9023a80c3. * added pprof related flags (expect --pprof.cpuprofile - Write CPU profile to the given file) * added support for --dev.gaslimit flag * added support for --fdlimit flag * added support for --netrestrict flag * added support for --nodekey and --nodekeyhex flag * added support for --vmodule, --log.json, --log.backtrace, and --log.debug flags * fixed related lint errors * fix lints from develop (few lints decided to appear from code that was untouched, weird) * more weird lints from develop * small precautionary fix * small bug ;) fix in NetRestrict * weird lints * change vmdebug = true to vmdebug = false * internal/ethapi: set rpc gas cap as default gas limit when creating access list * add : testcase for CommitInterruptExperiment (#792) * add : TestCommitInterruptExperimentBor * lint : fix lint * chg : add waitgroup * dev: chg: continue on error when uploading snyk results to GH (#795) * consensus/bor: revert handle unauthorized signer in consensus.Prepare * Revert "consensus/bor: handle unauthorized signer in consensus.Prepare (#651)" This reverts commit 9ce8c7de75a1b5021bb6f2b8086e813c8244b22c. * eth/downloader/whitelist: skip future chain validation (#796) * eth/downloader/whitelist: skip future chain validation * eth/downloader/whitelist: fix tests for future chain import * Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life * Merge qa to develop (#814) * Adding in Mumbai/Mainnet precursor deb packaging for tests to use during upgrade(iterations to come) * Added changes per discussion in PR, more changes may be necessary * Adding prerelease true * Disabling goreleaser * Removing README swap file * change bor_dir and add bor user for v0.3.0 release * rollback bor user and use root * metrics: handle equal to separated config flag (#596) * metrics: handle based config path * internal/cli/server: add more context to logs * use space separated flag and value in bor.service * fixed static-nodes related buf (os independent) (#598) * fixed static-nodes related buf (os independent) * taking static-nodes as input if default not present * Update default flags (#600) * internal/cli/server: use geth's default for txpool.pricelimit and add comments * builder/files: update config.toml for mainnet * packaging/templates: update defaults for mainnet and mumbai * internal/cli/server: skip overriding cache * packaging/templates: update cache value for mainnet * packaging/templates: update gcmode for archive mumbai node * metrics: handle nil telemetry config (#601) * resolve merge conflicts * update go version in release.yml * update goversion in makefile * update Docker login for goreleaser-cross v1.19 * Cleanup for the packager to use git tag in the package profile naming. Added conditional check for directory structure, this is in prep for v0.3.1, as this will create a failure on upgrade path in package due to file exist * added a toml configuration file with comments describing each flag (#607) * added a toml configuration file with comments describing each flag * internal/cli/server: update flag description * docs/cli: update example config and description of flags * docs: update new-cli docs Co-authored-by: Manav Darji * Adding of 0.3.0 package changes, control file updates, postinst changes, and packager update * added ancient datadir flag and toml field, need to decide on default value and update the conversion script * updated toml files with ancient field * Add support for new flags in new config.toml, which were present in old config.toml (#612) * added HTTPTimeouts, and TrieTimeout flag in new tol, from old toml * added RAW fields for these time.Duration flags * updated the conversion script to support these extra 4 flags * removed hcl and json config tests as we are only supporting toml config files * updated toml files with cache.timeout field * updated toml files with jsonrpc.timeouts field * tests/bor: expect a call for latest checkpoint * tests/bor: expect a call for latest checkpoint * packaging/templates: update cache values for archive nodes Co-authored-by: Manav Darji * remove unwanted code * Fix docker publish authentication issue In gorelease-cross 1.19+, dockerhub authentication will require docker logion action followed by mounting docker config file. See https://github.com/goreleaser/goreleaser-cross#github-actions. * Revert "update Docker login for goreleaser-cross v1.19" This reverts commit 4d19cf5342a439d98cca21b03c63a0bc075769cf. * Bump version to stable * Revert "Merge pull request #435 from maticnetwork/POS-553" This reverts commit 657d262defc9c94e9513b3d45230492d8b20eac7, reversing changes made to 88dbfa1c13c15464d3c1a3085a9f12d0ffb9b218. * revert change for release for go1.19 * Add default values to CLI helper and docs This commit adds default values to CLI helper and docs. When the default value of a string flag, slice string flag, or map string flag is empty, its helper message won't show any default value. * Add a summary of new CLI in docs * Updating packager as binutils changed version so that apt-get installs current versions * Add state pruning to new CLI * Minor wording fix in prune state description * Bumping control file versions * Mainnet Delhi fork * Set version to stable * change delhi hardfork block number * handle future chain import and skip peer drop (#650) * handle future chain import and skip peer drop * add block import metric * params: bump version to v0.3.3-stable * Bump bor version in control files for v0.3.3 mainnet release * Added checks to RPC requests and introduced new flags to customise the parameters (#657) * added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit) * added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit) * updated the config files and docs * chg : trieTimeout from 60 to 10 mins (#692) * chg : trieTimeout from 60 to 10 mins * chg : cache.timout to 10m from 1h in configs * internal/cli/server : fix : added triesInMemory in config (#691) * changed version from 0.3.0 to 0.3.4-beta (#693) * fix nil state-sync issue, increase grpc limit (#695) * Increase grpc message size limit in pprof * consensus/bor/bor.go : stateSyncs init fixed [Fix #686] * eth/filters: handle nil state-sync before notify * eth/filters: update check Co-authored-by: Jerry Co-authored-by: Daniil * core, tests/bor: add more tests for state-sync validation (#710) * core: add get state sync function for tests * tests/bor: add validation for state sync events post consensus * Arpit/temp bor sync (#701) * Increase grpc message size limit in pprof * ReadBorReceipts improvements * use internal function * fix tests * fetch geth upstread for ReadBorReceiptRLP * Only query bor receipt when the query index is equal to # tx in block body This change reduces the frequency of calling ReadBorReceipt and ReadBorTransaction, which are CPU and db intensive. * Revert "fetch geth upstread for ReadBorReceiptRLP" This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0. * Restore ReadBorReceiptRLP * fix bor receipts * remove unused * fix lints --------- Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720) This reverts commit 241843c7e7bb18e64d2e157fd6fbbd665f6ce9d9. * Arpit/add execution pool 2 (#719) * initial * linters * linters * remove timeout * update pool * change pool size function * check nil * check nil * fix tests * Use execution pool from server in all handlers * simplify things * test fix * add support for cli, config * add to cli and config * merge base branch * debug statements * fix bug * atomic pointer timeout * add apis * update workerpool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * fix tests * mutex * refactor flag and value names * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * debug statements * fix bug * update workerpool * atomic pointer timeout * add apis * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * merge base branch * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * mutex * fix tests * Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * Change default size of execution pool to 40 * refactor flag and value names * fix merge conflicts * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * fix linters * fix go.mod * change sec to ms * change default value for ep timeout * fix node api calls * comment setter for ep timeout --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Jerry Co-authored-by: Manav Darji * version change (#721) * Event based pprof (#732) * feature * Save pprof to /tmp --------- Co-authored-by: Jerry * Cherry-pick changes from develop (#738) * Check if block is nil to prevent panic (#736) * miner: use env for tracing instead of block object (#728) --------- Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * add max code init size check in txpool (#739) * Revert "Event based pprof" and update version (#742) * Revert "Event based pprof (#732)" This reverts commit 22fa4033e8fabb51c44e8d2a8c6bb695a6e9285e. * params: update version to 0.3.4-beta3 * packaging/templates: update bor version * internal/ethapi :: Fix : newRPCTransactionFromBlockIndex * Merge master to qa (#813) * Merge qa to master (#750) * Added checks to RPC requests and introduced new flags to customise the parameters (#657) * added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit) * added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit) * updated the config files and docs * chg : trieTimeout from 60 to 10 mins (#692) * chg : trieTimeout from 60 to 10 mins * chg : cache.timout to 10m from 1h in configs * internal/cli/server : fix : added triesInMemory in config (#691) * changed version from 0.3.0 to 0.3.4-beta (#693) * fix nil state-sync issue, increase grpc limit (#695) * Increase grpc message size limit in pprof * consensus/bor/bor.go : stateSyncs init fixed [Fix #686] * eth/filters: handle nil state-sync before notify * eth/filters: update check Co-authored-by: Jerry Co-authored-by: Daniil * core, tests/bor: add more tests for state-sync validation (#710) * core: add get state sync function for tests * tests/bor: add validation for state sync events post consensus * Arpit/temp bor sync (#701) * Increase grpc message size limit in pprof * ReadBorReceipts improvements * use internal function * fix tests * fetch geth upstread for ReadBorReceiptRLP * Only query bor receipt when the query index is equal to # tx in block body This change reduces the frequency of calling ReadBorReceipt and ReadBorTransaction, which are CPU and db intensive. * Revert "fetch geth upstread for ReadBorReceiptRLP" This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0. * Restore ReadBorReceiptRLP * fix bor receipts * remove unused * fix lints --------- Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720) This reverts commit 241843c7e7bb18e64d2e157fd6fbbd665f6ce9d9. * Arpit/add execution pool 2 (#719) * initial * linters * linters * remove timeout * update pool * change pool size function * check nil * check nil * fix tests * Use execution pool from server in all handlers * simplify things * test fix * add support for cli, config * add to cli and config * merge base branch * debug statements * fix bug * atomic pointer timeout * add apis * update workerpool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * fix tests * mutex * refactor flag and value names * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * debug statements * fix bug * update workerpool * atomic pointer timeout * add apis * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * merge base branch * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * mutex * fix tests * Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * Change default size of execution pool to 40 * refactor flag and value names * fix merge conflicts * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * fix linters * fix go.mod * change sec to ms * change default value for ep timeout * fix node api calls * comment setter for ep timeout --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Jerry Co-authored-by: Manav Darji * version change (#721) * Event based pprof (#732) * feature * Save pprof to /tmp --------- Co-authored-by: Jerry * Cherry-pick changes from develop (#738) * Check if block is nil to prevent panic (#736) * miner: use env for tracing instead of block object (#728) --------- Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * add max code init size check in txpool (#739) * Revert "Event based pprof" and update version (#742) * Revert "Event based pprof (#732)" This reverts commit 22fa4033e8fabb51c44e8d2a8c6bb695a6e9285e. * params: update version to 0.3.4-beta3 * packaging/templates: update bor version * params, packaging/templates: update bor version --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Pratik Patil Co-authored-by: Jerry Co-authored-by: Daniil Co-authored-by: Arpit Temani Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * core, miner: add sub-spans for tracing (#753) * core, miner: add sub-spans for tracing * fix linters * core: add logs for debugging * core: add more logs to print tdd while reorg * fix linters * core: minor fix * core: remove debug logs * core: use different span for write block and set head * core: use internal context for sending traces (#755) * core: add : impossible reorg block dump (#754) * add : impossible reorg block dump * chg : 3 seperate files for impossoble reorg dump * add : use exportBlocks method and RLP blocks before writing * chg : small changes * bump : go version from 1.19 to 1.20.1 (#761) * Revert "bump : go version from 1.19 to 1.20.1 (#761)" This reverts commit 4561012af9a31d20c2715ce26e4b39ca93420b8b. * core/vm: use optimized bigint (#26021) * Add holiman/big * Fix linter * Bump version to v0.3.5 * fix lints from develop (few lints decided to appear from code that was untouched, weird) * upgrade crypto lib version (#770) * bump dep : github.com/Masterminds/goutils to v1.1.1 (#769) * mardizzone/pos-1313: bump crypto dependency (#772) * dev: chg: bumd net dependency * dev: chg: bump crypto dependency * dev: chg: bump crypto dependency * bump dep : golang.org/x/net to v0.8.0 (#771) * Verify validator set against local contract on receiving an end-of-sprint block (#768) * Verify validator set against local contract on receiving an end-of-sprint block * Fix tests * Respect error returned by ParseValidators * Keep going back until a parent block presents * core/txpool: implement DoS defenses from geth (#778) * Hotfixes and deps bump (#776) * dev: chg: bump deps * internal/cli/server, rpc: lower down http readtimeout to 10s * dev: chg: get p2p adapter * dev: chg: lower down jsonrpc readtimeout to 10s * cherry-pick txpool optimisation changes * add check for empty lists in txpool (#704) * add check * linters * core, miner: add empty instrumentation name for tracing --------- Co-authored-by: Raneet Debnath Co-authored-by: SHIVAM SHARMA Co-authored-by: Evgeny Danilenko <6655321@bk.ru> Co-authored-by: Manav Darji * packaging,params: bump to v0.3.6 (#782) * v0.3.6 fix (#787) * Fix get validator set in header verifier * chg : commit tx logs from info to debug (#673) * chg : commit tx logs from info to debug * fix : minor changes * chg : miner : commitTransactions-stats moved from info to debug * lint : fix linters * refactor logging * miner : chg : UnauthorizedSignerError to debug * lint : fix lint * fix : log.Logger interface compatibility --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Remove unnecessary sorting of valset from header in verification * dev: chg: version bump --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: marcello33 * core: improve locks in txpool (#807) * added a write lock to the txs.filter method and a read lock to the txs.reheap method - both of which are called by Filter during reorg adjustments to txpool * txpool reorg locks * more locks * locks * linters * params, packaging: update version for v0.3.8-beta release * core: add logs in reheap --------- Co-authored-by: Alex Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Merge qa to master (#808) * Added checks to RPC requests and introduced new flags to customise the parameters (#657) * added a check to reject rpc requests with batch size > the one set using a newly added flag (rpcbatchlimit) * added a check to reject rpc requests whose result size > the one set using a newly added flag (rpcreturndatalimit) * updated the config files and docs * chg : trieTimeout from 60 to 10 mins (#692) * chg : trieTimeout from 60 to 10 mins * chg : cache.timout to 10m from 1h in configs * internal/cli/server : fix : added triesInMemory in config (#691) * changed version from 0.3.0 to 0.3.4-beta (#693) * fix nil state-sync issue, increase grpc limit (#695) * Increase grpc message size limit in pprof * consensus/bor/bor.go : stateSyncs init fixed [Fix #686] * eth/filters: handle nil state-sync before notify * eth/filters: update check Co-authored-by: Jerry Co-authored-by: Daniil * core, tests/bor: add more tests for state-sync validation (#710) * core: add get state sync function for tests * tests/bor: add validation for state sync events post consensus * Arpit/temp bor sync (#701) * Increase grpc message size limit in pprof * ReadBorReceipts improvements * use internal function * fix tests * fetch geth upstread for ReadBorReceiptRLP * Only query bor receipt when the query index is equal to # tx in block body This change reduces the frequency of calling ReadBorReceipt and ReadBorTransaction, which are CPU and db intensive. * Revert "fetch geth upstread for ReadBorReceiptRLP" This reverts commit 2e838a6b1313d26674f3a8df4b044e35dcbf35a0. * Restore ReadBorReceiptRLP * fix bor receipts * remove unused * fix lints --------- Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: Evgeny Danienko <6655321@bk.ru> * Revert "chg : trieTimeout from 60 to 10 mins (#692)" (#720) This reverts commit 241843c7e7bb18e64d2e157fd6fbbd665f6ce9d9. * Arpit/add execution pool 2 (#719) * initial * linters * linters * remove timeout * update pool * change pool size function * check nil * check nil * fix tests * Use execution pool from server in all handlers * simplify things * test fix * add support for cli, config * add to cli and config * merge base branch * debug statements * fix bug * atomic pointer timeout * add apis * update workerpool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * fix tests * mutex * refactor flag and value names * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * debug statements * fix bug * update workerpool * atomic pointer timeout * add apis * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * fix issues * change params * fix issues * fix ipc issue * remove execution pool from IPC * revert * merge base branch * Merge branch 'add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * mutex * fix tests * Merge branch 'arpit/add-execution-pool' of github.com:maticnetwork/bor into arpit/add-execution-pool * Change default size of execution pool to 40 * refactor flag and value names * fix merge conflicts * ordering fix * refactor flag and value names * update default ep size to 40 * fix bor start issues * revert file changes * fix linters * fix go.mod * change sec to ms * change default value for ep timeout * fix node api calls * comment setter for ep timeout --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Jerry Co-authored-by: Manav Darji * version change (#721) * Event based pprof (#732) * feature * Save pprof to /tmp --------- Co-authored-by: Jerry * Cherry-pick changes from develop (#738) * Check if block is nil to prevent panic (#736) * miner: use env for tracing instead of block object (#728) --------- Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * add max code init size check in txpool (#739) * Revert "Event based pprof" and update version (#742) * Revert "Event based pprof (#732)" This reverts commit 22fa4033e8fabb51c44e8d2a8c6bb695a6e9285e. * params: update version to 0.3.4-beta3 * packaging/templates: update bor version * internal/ethapi :: Fix : newRPCTransactionFromBlockIndex * fix: remove assignment for bor receipt --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Pratik Patil Co-authored-by: Jerry Co-authored-by: Daniil Co-authored-by: Arpit Temani Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> * Setting up bor to use hosted 18.04 runner as ubuntu provided 18.04 runner is end of life --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Pratik Patil Co-authored-by: Jerry Co-authored-by: Daniil Co-authored-by: Arpit Temani Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> Co-authored-by: Martin Holst Swende Co-authored-by: marcello33 Co-authored-by: Raneet Debnath Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com> Co-authored-by: Alex Co-authored-by: Daniel Jones * core: remove duplicate tests * miner: use get validators by hash in tests --------- Co-authored-by: Daniel Jones Co-authored-by: Will Button Co-authored-by: Will Button Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com> Co-authored-by: Pratik Patil Co-authored-by: Arpit Temani Co-authored-by: Jerry Co-authored-by: SHIVAM SHARMA Co-authored-by: Daniil Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> Co-authored-by: Martin Holst Swende Co-authored-by: marcello33 Co-authored-by: Raneet Debnath Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com> Co-authored-by: Alex --------- Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Raneet Debnath Co-authored-by: Arpit Temani Co-authored-by: SHIVAM SHARMA Co-authored-by: Jerry Co-authored-by: Manav Darji Co-authored-by: builder90210 Co-authored-by: Krishna Upadhyaya Co-authored-by: Daniel Jones Co-authored-by: Will Button Co-authored-by: Will Button Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com> Co-authored-by: Pratik Patil Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com> Co-authored-by: Didi Co-authored-by: ephess Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> Co-authored-by: Daniil Co-authored-by: Martin Holst Swende Co-authored-by: Alex * dev: chg: version bump for v0.3.9-beta (#825) * eth/fetcher: if peers never respond, drop them (#837) (#844) * dev: chg: bump version to beta-2 for v0.3.9 (#845) * rm : disable interruptCommit on txsCh select case (#848) * dev: chg: version bump (#849) * fix : go-routine leak in commitInterrupt channel (#851) * fix : go-routine leak in commitInterrupt channel * fix : fix lint * rm : remove t.Parallel() from TestGenerateBlockAndImport tests * fix : minor optimisations in test * fix : BenchmarkBorMining * dev: chg: beta version bump (#853) * dev: chg: bump version to stable * dev: chg: contracts branch * dev: fix: linter * dev: chg: update templum/govulncheck-action to fix execution --------- Co-authored-by: SHIVAM SHARMA Co-authored-by: Pratik Patil Co-authored-by: Manav Darji Co-authored-by: Jerry Co-authored-by: Daniil Co-authored-by: Arpit Temani Co-authored-by: Evgeny Danienko <6655321@bk.ru> Co-authored-by: Dmitry <46797839+dkeysil@users.noreply.github.com> Co-authored-by: Martin Holst Swende Co-authored-by: Raneet Debnath Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com> Co-authored-by: Alex Co-authored-by: Daniel Jones Co-authored-by: builder90210 Co-authored-by: Krishna Upadhyaya Co-authored-by: Will Button Co-authored-by: Will Button Co-authored-by: Daniel Jones <105369507+djpolygon@users.noreply.github.com> Co-authored-by: Didi Co-authored-by: ephess --- .github/ISSUE_TEMPLATE/bug.md | 2 +- .github/matic-cli-config.yml | 17 +- .github/pull_request_template.md | 7 +- .github/workflows/ci.yml | 14 +- .github/workflows/security-ci.yml | 67 ++ .github/workflows/security-sonarqube-ci.yml | 32 + .gitignore | 2 + .golangci.yml | 10 +- .snyk | 41 ++ Makefile | 8 +- README.md | 4 +- SECURITY.md | 181 +---- accounts/abi/bind/bind_test.go | 1 + build/ci.go | 38 +- builder/files/config.toml | 35 +- cmd/faucet/faucet.go | 11 +- cmd/geth/chaincmd.go | 3 + cmd/geth/main.go | 18 + cmd/utils/bor_flags.go | 24 + cmd/utils/flags.go | 3 + common/path.go | 29 + consensus/bor/bor.go | 18 +- consensus/bor/heimdallapp/checkpoint.go | 52 ++ consensus/bor/heimdallapp/client.go | 27 + consensus/bor/heimdallapp/span.go | 86 +++ consensus/bor/heimdallapp/state_sync.go | 64 ++ consensus/bor/valset/validator_set.go | 27 +- core/tests/blockchain_repair_test.go | 2 +- core/tx_pool.go | 16 +- core/tx_pool_test.go | 91 +++ core/types/transaction_signing.go | 36 + core/vm/contracts.go | 1 + docs/cli/bootnode.md | 6 +- docs/cli/debug_pprof.md | 6 +- docs/cli/example_config.toml | 75 +- docs/cli/server.md | 66 +- eth/backend.go | 4 +- eth/downloader/whitelist/service.go | 18 +- eth/downloader/whitelist/service_test.go | 11 +- eth/ethconfig/config.go | 24 +- eth/fetcher/block_fetcher.go | 39 +- eth/filters/bor_api.go | 1 + ethclient/ethclient_test.go | 141 ++-- go.mod | 118 ++- go.sum | 687 +++++++++++++++++- integration-tests/bor_health.sh | 15 + internal/cli/bootnode.go | 23 +- internal/cli/debug_pprof.go | 27 +- internal/cli/debug_test.go | 3 +- internal/cli/dumpconfig.go | 2 + internal/cli/server/chains/developer.go | 4 +- internal/cli/server/command.go | 43 ++ internal/cli/server/config.go | 342 ++++++++- internal/cli/server/flags.go | 204 +++++- internal/cli/server/pprof/pprof.go | 22 + internal/cli/server/proto/server.pb.go | 418 ++++++----- internal/cli/server/proto/server.proto | 6 +- internal/cli/server/proto/server_grpc.pb.go | 16 +- internal/cli/server/server.go | 79 +- internal/cli/server/service.go | 31 +- internal/cli/snapshot.go | 2 +- internal/cli/status.go | 22 +- internal/cli/status_test.go | 42 ++ internal/cli/version.go | 2 +- internal/ethapi/api.go | 25 +- internal/web3ext/web3ext.go | 10 +- metrics/metrics.go | 11 +- miner/fake_miner.go | 4 +- miner/test_backend.go | 619 +++++++++++++++- miner/worker.go | 73 +- miner/worker_test.go | 147 ++-- node/api.go | 24 + p2p/server.go | 31 + .../templates/mainnet-v1/archive/config.toml | 32 +- .../mainnet-v1/sentry/sentry/bor/config.toml | 40 +- .../sentry/validator/bor/config.toml | 40 +- .../mainnet-v1/without-sentry/bor/config.toml | 40 +- packaging/templates/package_scripts/control | 2 +- .../templates/package_scripts/control.arm64 | 2 +- .../package_scripts/control.profile.amd64 | 2 +- .../package_scripts/control.profile.arm64 | 2 +- .../package_scripts/control.validator | 2 +- .../package_scripts/control.validator.arm64 | 2 +- .../templates/testnet-v4/archive/config.toml | 32 +- .../testnet-v4/sentry/sentry/bor/config.toml | 32 +- .../sentry/validator/bor/config.toml | 39 +- .../testnet-v4/without-sentry/bor/config.toml | 32 +- params/version.go | 19 +- rlp/rlpgen/main.go | 13 +- rpc/handler.go | 1 + scripts/getconfig.go | 208 +++--- sonar-project.properties | 2 + tests/fuzzers/difficulty/debug/main.go | 11 +- tests/fuzzers/les/debug/main.go | 11 +- tests/fuzzers/rangeproof/debug/main.go | 11 +- tests/fuzzers/snap/debug/main.go | 11 +- tests/fuzzers/stacktrie/debug/main.go | 11 +- tests/fuzzers/vflux/debug/main.go | 11 +- 98 files changed, 4144 insertions(+), 874 deletions(-) create mode 100644 .github/workflows/security-ci.yml create mode 100644 .github/workflows/security-sonarqube-ci.yml create mode 100644 .snyk create mode 100644 consensus/bor/heimdallapp/checkpoint.go create mode 100644 consensus/bor/heimdallapp/client.go create mode 100644 consensus/bor/heimdallapp/span.go create mode 100644 consensus/bor/heimdallapp/state_sync.go create mode 100644 integration-tests/bor_health.sh create mode 100644 internal/cli/status_test.go create mode 100644 sonar-project.properties diff --git a/.github/ISSUE_TEMPLATE/bug.md b/.github/ISSUE_TEMPLATE/bug.md index 7d34216478..8c23f0bd9f 100644 --- a/.github/ISSUE_TEMPLATE/bug.md +++ b/.github/ISSUE_TEMPLATE/bug.md @@ -6,7 +6,7 @@ labels: 'type:bug' assignees: '' --- -Our support team has aggregated some common issues and their solutions from past which are faced while running or interacting with a bor client. In order to prevent redundant efforts, we would encourage you to have a look at the [FAQ's section](https://docs.polygon.technology/docs/faq/technical-faqs) of our documentation mentioning the same, before filing an issue here. In case of additional support, you can also join our [discord](https://discord.com/invite/zdwkdvMNY2) server +Our support team has aggregated some common issues and their solutions from past which are faced while running or interacting with a bor client. In order to prevent redundant efforts, we would encourage you to have a look at the [FAQ's section](https://wiki.polygon.technology/docs/faq/technical-faqs/) of our documentation mentioning the same, before filing an issue here. In case of additional support, you can also join our [discord](https://discord.com/invite/zdwkdvMNY2) server