[Bug] [seatunnel-connector-flink-jdbc] Data loss occurs when partition_column (partition mode) is set

[ic4y]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

After setting partition_column, it will cause data loss, and the data loss is equal to the number of partitions.

The reason is that the sql partition read uses partitionColumn >= ? AND partitionColumn + < ? , which does not include the upper bound.

But the upper and lower bounds are not repeated when splitting ParameterValues.

For example

partition_id=(1,2,3,4,5,6,7,8,9,10)
partition number is 2

Then ParameterValues=[[1,5][6,10]]

Two SQL statements will be generated
1、partition_id >= 1 AND partition_id < 5
2、partition_id >= 6 AND partition_id < 10

In this way, two data of id=5 and 10 will be lost. The lost data is equal to the number of partitions. The larger the number of partitions, the more data is lost.

So sql also needs to include the upper bound，like partitionColumn >= ? AND partitionColumn + <= ?

SeaTunnel Version

dev

SeaTunnel Config

None

Running Command

None

Error Exception

None

Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct