-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Eliminate pushed down filters in JDBC connectors #4111
Comments
From offline conversation with @martint :
I agree. We need JdbcMetadata / JdbcClient / ColumnMapping change allowing the connector to opt out¹ from Today we have ¹) "opt in" into "full pushdown" is safer from backward compatibility perspective, but it is IMO wrong choice |
PR: #4565 |
In JDBC connectors, pushed down filters are retained in the plan, because we do not know whether and which portion of the filter will actually be applied.
For example:
varbinary
)IN
predicates in SQL Server connector)Introduce the base-jdbc level API to let JdbcClient provide feedback whether predicate is fully pushed down.
The goal is to eliminate filters to enable subsequent aggregation pushdown.
Motivation
Agg pushdown is only possible when aggregation is direct above TS.
So, for a query like
{code}
SELECT sale_date, count(*)
FROM sales
WHERE sale_date >= DATE '2020-01-01'
{code}
we would need not only to push the filter into JDBC connector ((this is the case today)
but also eliminate it from above the TS (this is not the case today)
The text was updated successfully, but these errors were encountered: