Fix 'Multiple entries with same key' error with dynamic filters

[lxqfy]

Description

Trino will throw Exceptions "Multiple entries with same key", when the same dynamic filter applies to the same partitions key column multiple times under different conditions. E.g.

SELECT *
FROM tbl_a a
JOIN tbl_b b ON b.partition_key_date >= a.start_date AND b.partition_key_date <= a.start_date

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# General
* Fix query failures with "Multiple entries with same key" error encountered with joins on partitioned tables. ({issue}`20917`)

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Li Xiangqun.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[raunaqmorarka]

Please add a test in BaseDynamicPartitionPruningTest

[lxqfy]

Noted will do.

[sopel39]

lgtm % @raunaqmorarka comments.

SELECT *
FROM tbl_a a
JOIN tbl_b b ON b.partition_key_date >= a.start_date AND b.partition_key_date <= a.start_date

that query really simplifies to:

SELECT *
FROM tbl_a a
JOIN tbl_b b ON b.partition_key_date = a.start_date

and inequality join is pretty inefficient.

Is there a better example where it fails?

[sopel39]

nit: commit message too long

[lxqfy]

lgtm % @raunaqmorarka comments.

SELECT *
FROM tbl_a a
JOIN tbl_b b ON b.partition_key_date >= a.start_date AND b.partition_key_date <= a.start_date

that query really simplifies to:

SELECT *
FROM tbl_a a
JOIN tbl_b b ON b.partition_key_date = a.start_date

and inequality join is pretty inefficient.

Is there a better example where it fails?

This is just a simplified query to reproduce the issue.

[sopel39]

This is just a simplified query to reproduce the issue.

I understand. However, the issue that this PR fixes is rather basic, but wasn't discovered before. Hence I wonder what practical query could trigger it, because queries like ON b.partition_key_date >= a.start_date AND b.partition_key_date <= a.start_date are bad for performance anyway.

@lxqfy What kind of production query did hit the issue?

[lxqfy]

This is just a simplified query to reproduce the issue.

I understand. However, the issue that this PR fixes is rather basic, but wasn't discovered before. Hence I wonder what practical query could trigger it, because queries like ON b.partition_key_date >= a.start_date AND b.partition_key_date <= a.start_date are bad for performance anyway.

@lxqfy What kind of production query did hit the issue?

The actual query spans 200 lines and is overly complex. Generally, The query owner creates a CTE UNION data from various granularities such as "Daily", "Weekly", and "Monthly" with start_date and end_date. For "Daily", they just did SELECT report_date AS start_date, report_date AS end_date. and JOIN another table ON event_date between start_date and end_date

[sopel39]

@lxqfy but does that prod query simplify to ON b.partition_key_date >= a.start_date AND b.partition_key_date <= a.start_date?

The reason I'm asking is maybe we should have a rule that transforms ON b.partition_key_date >= a.start_date AND b.partition_key_date <= a.start_date into ON b.partition_key_date = a.start_date?