Scan data by time range

[evenyag]

What type of enhancement is this?

Performance, Refactor

What does the enhancement do?

A region partitions data by time ranges. The region can scan data by time range so it only reads a small number of files each time. This can reduce the latency since downloading files from all ranges may take longer.

Implementation challenges

We need to postpone the pruning phase and reimplement scanners. UnorderedScan and SeqScan have different requirements for ranges.

Implementation history

• feat: unordered scanner scans data by time ranges #4757
• feat: Seq scanner scans data by time range #4809
• feat: yields empty batch after reading a range #4845
• Remove the scan parallelism. fix!: fix regression caused by unbalanced partitions and splitting ranges #5090
• Support splitting multiple row groups in SeqScan. feat: get row group time range from cached metadata #4869
• Support field pruning in last_non_null mode if the time range only has one file.
• More tests for RangeMeta and StreamContext
• Try to get time range from cached metadata for each partition range feat: get row group time range from cached metadata #4869
• Memtable should prune by timestamp fix: prune batches from memtable by time range #4913
• fix: prune memtable/files range independently in each partition #4998
• Split memtable into multiple partitions