[Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 #7840

XenosK · 2024-10-15T06:34:11Z

Purpose of this pull request

fix: #7794

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update plugin-mapping.properties and add new connector information in it
2. Update the pom file of seatunnel-dist
3. Add ci label in label-scope-conf
4. Add e2e testcase in seatunnel-e2e
5. Update connector plugin_config
Update the release-note.

…arams apache#7794

# Conflicts: # plugin-mapping.properties

…arams apache#7794

…t.column apache#7794

Hisoka-X

Please add test case.

… update test case

XenosK · 2024-10-15T14:00:08Z

Please add test case.

Test case have been added, take a look.

Hisoka-X

cc @hailin0

...se/src/main/java/org/apache/seatunnel/connectors/cdc/base/dialect/JdbcDataSourceDialect.java

...eatunnel/connectors/cdc/base/source/enumerator/splitter/AbstractJdbcSourceChunkSplitter.java

...se/src/main/java/org/apache/seatunnel/connectors/cdc/base/dialect/JdbcDataSourceDialect.java

...eatunnel/connectors/cdc/base/source/enumerator/splitter/AbstractJdbcSourceChunkSplitter.java

…ain/java/org/apache/seatunnel/connectors/cdc/base/source/enumerator/splitter/AbstractJdbcSourceChunkSplitter.java Co-authored-by: Jia Fan <fanjiaeminem@qq.com>

…ain/java/org/apache/seatunnel/connectors/cdc/base/dialect/JdbcDataSourceDialect.java Co-authored-by: Jia Fan <fanjiaeminem@qq.com>

docs/en/connector-v2/source/MySQL-CDC.md

...eatunnel/connectors/cdc/base/source/enumerator/splitter/AbstractJdbcSourceChunkSplitter.java

Hisoka-X

Thanks @XenosK !

XenosK added 8 commits October 14, 2024 11:18

[Improve][Sls] Add sls sink connector、e2e、doc

fc75772

[Improve][Sls] Add sls sink connector、e2e、doc

1576162

Merge branch 'dev' of https://github.com/apache/seatunnel into dev

1a636d3

[Improve][Sls] Update sls sink e2e code style

87d749a

[Improve][Sls] Add sls sink license header

69bf03c

[Feature][Connector-V2]Jdbc chunk split add “snapshot.split.column” p…

1165447

…arams apache#7794

Merge branch 'dev' of https://github.com/apache/seatunnel into dev

0c2676b

# Conflicts: # plugin-mapping.properties

[Feature][Connector-V2]Jdbc chunk split add “snapshot.split.column” p…

7165bad

…arams apache#7794

github-actions bot added connectors-v2 cdc labels Oct 15, 2024

[Feature][Connector-V2]Update mysql-cdc doc with params snapshot.spli…

f5c5ee8

…t.column apache#7794

github-actions bot added the document label Oct 15, 2024

Hisoka-X reviewed Oct 15, 2024

View reviewed changes

[Feature][Connector-V2]Update snapshot.split.column type with map and…

8a0b6ba

… update test case

github-actions bot added the e2e label Oct 15, 2024

XenosK added 6 commits October 15, 2024 16:12

[Feature][Connector-V2]Remove local test run println

401723e

[Feature][Connector-V2] code style

f9c973a

[Feature][Connector-V2] SourceConfig remove getSplitColumn

83020fd

[Feature][Connector-V2] uniq_key_f add id

9a87c8b

[Feature][Connector-V2] add split column type check

7be5d6d

[Feature][Connector-V2] add split column type check

afd9cd9

Hisoka-X reviewed Oct 16, 2024

View reviewed changes

XenosK and others added 7 commits October 16, 2024 11:01

Update seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/m…

2cae910

…ain/java/org/apache/seatunnel/connectors/cdc/base/source/enumerator/splitter/AbstractJdbcSourceChunkSplitter.java Co-authored-by: Jia Fan <fanjiaeminem@qq.com>

Update seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/m…

edc6f70

…ain/java/org/apache/seatunnel/connectors/cdc/base/dialect/JdbcDataSourceDialect.java Co-authored-by: Jia Fan <fanjiaeminem@qq.com>

Update seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/m…

275520d

…ain/java/org/apache/seatunnel/connectors/cdc/base/dialect/JdbcDataSourceDialect.java Co-authored-by: Jia Fan <fanjiaeminem@qq.com>

[Feature][Connector-V2] boolean isUniqueKey

fe8f7d3

[Feature][Connector-V2] StringUtils.isNotEmpty

768ca66

[Feature][Connector-V2] code style

1973c18

[Feature][Connector-V2] &&!xx replace &&xx==false

08bd831

[Feature][Connector-V2] update doc

1388c01

XenosK dismissed Hisoka-X’s stale review via 1388c01 October 23, 2024 11:22

github-actions bot removed approved reviewed labels Oct 23, 2024

[Feature][Connector-V2] update snapshotSplitColumn config e2e

fa9ec16

hailin0 reviewed Nov 8, 2024

View reviewed changes

docs/en/connector-v2/source/MySQL-CDC.md Show resolved Hide resolved

[Feature][Connector-V2] update snapshotSplitColumn doc

5b2056f

XenosK changed the title ~~[Feature][Connector-V2]Jdbc chunk split add “snapshot.split.column” params #7794~~ [Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 Nov 11, 2024

Merge branch 'dev' of https://github.com/apache/seatunnel into dev

307a617

hailin0 previously approved these changes Nov 12, 2024

View reviewed changes

github-actions bot added approved reviewed labels Nov 12, 2024

hailin0 reviewed Nov 12, 2024

View reviewed changes

...eatunnel/connectors/cdc/base/source/enumerator/splitter/AbstractJdbcSourceChunkSplitter.java Outdated Show resolved Hide resolved

[Feature][Connector-V2] update unique column function

59730d0

XenosK dismissed hailin0’s stale review via 59730d0 November 13, 2024 02:47

github-actions bot removed approved reviewed labels Nov 13, 2024

Hisoka-X previously approved these changes Nov 13, 2024

View reviewed changes

github-actions bot added approved reviewed labels Nov 13, 2024

Hisoka-X requested a review from hailin0 November 14, 2024 03:28

[Feature][Connector-V2] update unique column function

4556bc6

XenosK dismissed Hisoka-X’s stale review via 4556bc6 November 15, 2024 06:39

github-actions bot removed approved reviewed labels Nov 15, 2024

hailin0 approved these changes Nov 19, 2024

View reviewed changes

github-actions bot added approved reviewed labels Nov 19, 2024

Hisoka-X approved these changes Nov 19, 2024

View reviewed changes

Hisoka-X merged commit b6c6dc0 into apache:dev Nov 19, 2024
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 #7840

[Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 #7840

XenosK commented Oct 15, 2024 •

edited

Loading

Hisoka-X left a comment

XenosK commented Oct 15, 2024

Hisoka-X left a comment

Hisoka-X left a comment

[Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 #7840

[Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 #7840

Conversation

XenosK commented Oct 15, 2024 • edited Loading

Purpose of this pull request

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

Hisoka-X left a comment

Choose a reason for hiding this comment

XenosK commented Oct 15, 2024

Hisoka-X left a comment

Choose a reason for hiding this comment

Hisoka-X left a comment

Choose a reason for hiding this comment

XenosK commented Oct 15, 2024 •

edited

Loading