[Feature][cdc sink clickhouse] Can timed writes be added to the cdc sink clickhouse

[zhaoyvli]

Search before asking

• I had searched in the feature and found no similar feature requirement.

Description

Can a timing configuration be added to write data to the clickhouse even if the number of data changes does not reach bulk_size within the specified time.

Usage Scenario

MySQL cdc write clickhouse. If the MySQL record update frequency is low and the number of records reaches bulk_size, data writing will be delayed. However, if bulk_size is reduced, frequent write operations will occur when the record update frequency is high. Therefore, even if the number of records in the specified time period does not reach the value of bulk_size, a timing configuration can be added to write clickhouse.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Carl-Zhou-CN]

can assign it to me?

[zhaoyvli]

sure, thanks.

[hailin0]

can assign it to me?

@Carl-Zhou-CN Assigned

[zhilinli123]

can assign it to me?

Do you have any good ideas or solutions for this timing writing? My previous implementation using a timed thread pool has completed a version but eventually the community did not choose to merge due to some resource issues!

CC: @Carl-Zhou-CN @Hisoka-X
close pr:#4328

[Carl-Zhou-CN]

can assign it to me?

Do you have any good ideas or solutions for this timing writing? My previous implementation using a timed thread pool has completed a version but eventually the community did not choose to merge due to some resource issues!

CC: @Carl-Zhou-CN @Hisoka-X close pr:#4328

@zhilinli123 Thank you very much. My idea is to trigger a refresh based on the division of the checkpoints

[zhilinli123]

can assign it to me?

Do you have any good ideas or solutions for this timing writing? My previous implementation using a timed thread pool has completed a version but eventually the community did not choose to merge due to some resource issues!
CC: @Carl-Zhou-CN @Hisoka-X close pr:#4328

@zhilinli123 Thank you very much. My idea is to trigger a refresh based on the division of the checkpoints

This is a nice idea to look forward to!