Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delta Lake metadata query detection logic should be moved into DeltaProvider #11166

Open
jlowe opened this issue Jul 10, 2024 · 0 comments
Open
Labels
good first issue Good for newcomers performance A performance related task/issue

Comments

@jlowe
Copy link
Member

jlowe commented Jul 10, 2024

Is your feature request related to a problem? Please describe.
Currently Delta Lake metadata query detection searches the entire query plan for nodes matching the rule, but this is wasteful when Delta Lake isn't available in the runtime environment.

Describe the solution you'd like
This logic should be moved to the code specific to supporting Delta Lake and the predicate function exposed through something like ExternalSource.

@jlowe jlowe added the ? - Needs Triage Need team to review and classify label Jul 10, 2024
@mattahrens mattahrens added good first issue Good for newcomers performance A performance related task/issue and removed ? - Needs Triage Need team to review and classify labels Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers performance A performance related task/issue
Projects
None yet
Development

No branches or pull requests

2 participants