feat: chain: make chain tipset fetching 1000x faster

[arajasek]

Related Issues

@jennijuju flagged that we need faster fetching of tipsets by height for Eth tooling support.

Proposed Changes

• Increase the skipcache size to 2^20 entries
• Drop the tipset itself from the skipcache entries. This is a lot of information to hold in memory that we don't actually need.
• Drop the parentHeight from the skipcache entries. This is just 8 bytes, but I don't think we need it.
• With that, we can easily rely on the skip cache to load 20 million epochs (so good until 2040).
• Each cache entry itself is ~308 bytes (an int64 + a TSK, using ~5 CIDs per TSK, which is aggressive), which means the total cache size when we get to 20M epochs will be ~0.3 GiB.

Additional Info

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title is in the form of of <PR type>: <area>: <change being made>
  • example: fix: mempool: Introduce a cache for valid signatures
  • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
  • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[arajasek]

Timing results on mainnet:

aayush@aayush-desktop ~/p/lotus (asr/fast-index)> time ./lotus chain list --count=3 --height=200
198: (Aug 24 19:39:00) [ bafy2bzacear4yqnylpchlufj5nccvdv56zi7655jtich2chbk2w7oci4fv4b6: f01002, ]
199: (Aug 24 19:39:30) [ bafy2bzacecrlj6vujyqwm5daygo27ksaibr37wyklcby6ihb4kiejlfludzyi: f01002, ]
200: (Aug 24 19:40:00) [ bafy2bzacedgt32ubhpx4hu3hrbu663djcykuplt4jc5mnyii2wlzx5ovsj7cs: f01000,bafy2bzaceb6ksh75v3m6dydsutfrcaoe542d44rotznniq5poi2bfirthl6se: f01002, ]

________________________________________________________
Executed in  342.67 secs      fish           external
   usr time  663.49 millis  572.00 micros  662.92 millis
   sys time   60.26 millis    0.00 micros   60.26 millis

aayush@aayush-desktop ~/p/lotus (asr/fast-index)> time ./lotus chain list --count=3 --height=200
198: (Aug 24 19:39:00) [ bafy2bzacear4yqnylpchlufj5nccvdv56zi7655jtich2chbk2w7oci4fv4b6: f01002, ]
199: (Aug 24 19:39:30) [ bafy2bzacecrlj6vujyqwm5daygo27ksaibr37wyklcby6ihb4kiejlfludzyi: f01002, ]
200: (Aug 24 19:40:00) [ bafy2bzacedgt32ubhpx4hu3hrbu663djcykuplt4jc5mnyii2wlzx5ovsj7cs: f01000,bafy2bzaceb6ksh75v3m6dydsutfrcaoe542d44rotznniq5poi2bfirthl6se: f01002, ]

________________________________________________________
Executed in  287.58 millis    fish           external
   usr time  475.32 millis  345.00 micros  474.98 millis
   sys time   37.61 millis  115.00 micros   37.50 millis

Warming the cache takes ~6 minutes, subsequent queries are less than a second.

[Stebalien]

LGTM. A couple of comments:

1. I'd just drop the ARC cache and just use a map. We're likely wasting a lot of time updating the "recency" metrics on that cache every time we use it (and the memory locality is likely really bad).
2. I'm somewhat concerned that a single query might take 6 minutes (will timeout on most services). We might want some form of background job to warm the cache. But (a) that can happen later and (b) service providers can likely work around this issue themselves.

[Stebalien]

If this isn't enough, we can consider implementing a real skiplist (with multiple levels).

[arajasek]

@Stebalien Thanks for the review and the suggestions!

I switched to using a map. It appears to have made the warmup slower, which is somewhat unexpected, but subsequent lookups faster. I think that's still a win, in addition to the overall simplicity, so I'm gonna push that up.

[Stebalien]

It's slower because you're not initializing the map to DefaultChainIndexCacheSize.

[Stebalien]

Also note: I think you can just use lbEntry by value to save some allocations which should speed things up even more.