Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: metadata cache provider and datafusion trait impls #25566

Merged
merged 3 commits into from
Nov 22, 2024

Commits on Nov 21, 2024

  1. feat: add metadata cache provider with APIs for write and query

    This adds the MetaDataCacheProvider for managing metadata caches in the
    influxdb3 instance. This includes APIs to create caches through the WAL
    as well as from a catalog on initialization, to write data into the
    managed caches, and to query data out of them.
    
    The query side is fairly involved, relying on Datafusion's TableFunctionImpl
    and TableProvider traits to make querying the cache using a user-defined
    table function (UDTF) possible.
    
    The predicate code was modified to only support two kinds of predicates:
    IN and NOT IN, which simplifies the code, and maps nicely with the DataFusion
    LiteralGuarantee which we leverage to derive the predicates from the
    incoming queries.
    
    A custom ExecutionPlan implementation was added specifically for the
    metadata cache that can report the predicates that are pushed down to
    the cache during query planning/execution.
    
    A big set of tests was added to to check that queries are working, and
    that predicates are being pushed down properly.
    hiltontj committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    088df39 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    20c74a7 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2024

  1. Configuration menu
    Copy the full SHA
    20d09a8 View commit details
    Browse the repository at this point in the history