-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature][Connector-V2] Jdbc DB2 support upsert SQL #7879
[Feature][Connector-V2] Jdbc DB2 support upsert SQL #7879
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @shashwatsai , LGTM just except one minior problem.
database = "E2E" | ||
table = "SINK" | ||
# The primary keys of the table, which will be used to generate the upsert sql | ||
generate_sink_sql = true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please force set enable_upsert
to true
to avoid the test case broken when the default value changed.
Add E2E for upsert db2 scenario
ce8ab9b
to
a6e8108
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM if ci passes. Thanks @shashwatsai !
@Hisoka-X , thanks for your time in reviewing the same. @hailin0 , @liugddx, @arshadmohammad can you please help with the review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM +1
Purpose of this pull request
Currently Seatunnel JDBC Connector with DB2 Dialect, does not support upsert scenarios, as the implementation is unavailable. While analysing our requirements within Visa's use-cases, we have approximately 60% of jobs that heavily depend on this.
We want to introduce the implementation for upsert support for DB2 dialect, along with the e2e Integration test case.
solves: #4434, as Dameng DB, implementation is already present.
How do I reopen the issue?
Does this PR introduce any user-facing change?
No,
User does not have to provide the query parameter in sink, user has to provide database, table and generate_sink_sql, which are already available in the configurations. Please refer corresponding [Seatunnel-web] PR: apache/seatunnel-web#233
Sample:
How was this patch tested?
We internally deployed, and tested the scenarios, we need to introduce a minor change in Seatunnel-web module as well, to delegate generation of sink sql to the engine, as required for the upsert use-case.
Check list
New License Guide
release-note
.