[syncer/storage] Pre-store Blocks

[patrick-ogrady]

To increase sync speed, we modified the storage/modules package to pre-store block data before it is sequenced (which can be done concurrently). This significantly reduces the amount of work that must be done serially (i.e. updating balances, updating coins, etc).

WARNING: THIS PR INCLUDES BREAKING CHANGES TO STORAGE! YOU MUST RESYNC ANY DATA STORED USING PREVIOUS RELEASES!

Changes

• Pre-store blocks as soon as they are seen by the syncer (instead of waiting to store until blocks are sequenced)
• Replace accountEntry with *types.AccountCurrency
• Manually encode/decode *types.AccountCurrency
• Ensure ctx changes in syncer do not introduce regression in rosetta-cli
• Update syncer tests to handle BlockSeen
• limit concurrent seen calls to runtime.NumCPU (tend to be more intensive than fetch calls so allow for separate configuration)
  • add ability to override concurrency in config (SeenConcurrency)

Future Work

• Add configuration option to rosetta-cli to determine SeeBlock concurrency

[coveralls]

Pull Request Test Coverage Report for Build 13474

• 208 of 289 (71.97%) changed or added relevant lines in 4 files are covered.
• 4 unchanged lines in 2 files lost coverage.
• Overall coverage decreased (-0.3%) to 78.078%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
storage/modules/balance_storage.go	23	25	92.0%
syncer/syncer.go	55	60	91.67%
storage/modules/block_storage.go	28	49	57.14%
storage/encoder/encoder.go	102	155	65.81%

Files with Coverage Reduction	New Missed Lines	%
syncer/syncer.go	1	91.18%
storage/modules/block_storage.go	3	76.37%

Totals
Change from base Build 13199:	-0.3%
Covered Lines:	8074
Relevant Lines:	10341

---

💛 - Coveralls

[patrick-ogrady]

the double context.Context in sequenceBlocks causes a linting issue that can't be bypassed 😢

[patrick-ogrady]

This is needed to spawn up more fetcher threads.

[yfl92]

LGTM