🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC #17815

akashkulk · 2022-10-11T00:37:23Z

What

Currently, CDC mode does not work when when the source MySQL db is set up with a server timezone that is not recognized by the ZoneID Java library (e.g. CEST, PDT).

The fix here is to expose an server timezone option in the CDC replication methods UI and add it to the debezium serverTimezone property while reading from MySQL in CDC mode.

🚨 User Impact 🚨

User will see a new option to add server timezone, with messaging that this is an 'advanced' option - only to be used when the server timezone is not IANNA compliant.

https://www.loom.com/i/2c1d080ab76541419782d508a22ab98b

github-actions · 2022-10-11T00:38:13Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

akashkulk · 2022-10-11T00:39:20Z

/test connector=connectors/source-mysql-strict-encrypt

🕑 connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223298892
❌ connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223298892
🐛 https://gradle.com/s/gfrkyuq7cl2oa

Build Failed

Test summary info:

Could not find result summary

akashkulk · 2022-10-11T00:39:28Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223299137
❌ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223299137
🐛 https://gradle.com/s/23ht6kdibqe26

Build Failed

Test summary info:

Could not find result summary

github-actions · 2022-10-11T03:30:08Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

akashkulk · 2022-10-11T03:31:08Z

/test connector=connectors/source-mysql-strict-encrypt

🕑 connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223916999
✅ connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3223916999
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-10-11T03:31:22Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223918088
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3223918088
No Python unittests run

Build Passed

Test summary info:

All Passed

github-actions · 2022-10-11T16:03:35Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T16:23:11Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

github-actions · 2022-10-11T16:23:27Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T16:26:01Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

github-actions · 2022-10-11T16:27:15Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T16:27:58Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-11T23:56:35Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-12T00:02:30Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

akashkulk · 2022-10-12T00:03:32Z

/test connector=connectors/source-mysql-strict-encrypt

🕑 connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3231075194
✅ connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3231075194
No Python unittests run

Build Passed

Test summary info:

All Passed

github-actions · 2022-10-12T00:03:33Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

github-actions · 2022-10-12T00:03:56Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

akashkulk · 2022-10-12T15:11:20Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3235809223
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3235809223
No Python unittests run

Build Passed

Test summary info:

All Passed

github-actions · 2022-10-12T15:12:04Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql
source-mysql-strict-encrypt

akashkulk · 2022-10-12T16:37:54Z

/publish connector=connectors/source-mysql run-tests=false

🕑 Publishing the following connectors:
connectors/source-mysql
https://github.com/airbytehq/airbyte/actions/runs/3236373815

Connector	Did it publish?	Were definitions generated?
connectors/source-mysql	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

akashkulk · 2022-10-12T16:38:04Z

/publish connector=connectors/source-mysql-strict-encrypt run-tests=false

🕑 Publishing the following connectors:
connectors/source-mysql-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/3236375145

Connector	Did it publish?	Were definitions generated?
connectors/source-mysql-strict-encrypt	✅	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

github-actions · 2022-10-12T16:50:32Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

github-actions · 2022-10-12T16:52:42Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mysql-strict-encrypt
source-mysql

…ce spec for CDC (airbytehq#17815) * Expose serverTimezone debezium option via MySQL Source spec for CDC * Fix tests * Bump dockerfile version * Update Dockerfile * Update mysql.md * Update mysql.md * Documentation * Code Review Comments * Addressing comments * Addressing comments * Addressing comments * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

Expose serverTimezone debezium option via MySQL Source spec for CDC

9e5fe30

github-actions bot added the area/connectors Connector related issues label Oct 11, 2022

octavia-squidington-iv added the connectors/source/mysql label Oct 11, 2022

akashkulk changed the title ~~Expose serverTimezone debezium option via MySQL Source spec for CDC~~ 🎉 Expose serverTimezone debezium option via MySQL Source spec for CDC Oct 11, 2022

Fix tests

9df57a6

octavia-squidington-iv added the connectors/source/mysql-strict-encrypt label Oct 11, 2022

Merge branch 'master' into akash/cdc-tz

24c591c

akashkulk added 2 commits October 11, 2022 09:22

Bump dockerfile version

e494bba

Update Dockerfile

7553d02

Update mysql.md

9767336

github-actions bot added the area/documentation Improvements or additions to documentation label Oct 11, 2022

akashkulk added 2 commits October 11, 2022 09:26

Update mysql.md

8620f41

Merge branch 'master' into akash/cdc-tz

aff43fa

akashkulk marked this pull request as ready for review October 11, 2022 16:27

akashkulk requested a review from a team as a code owner October 11, 2022 16:27

akashkulk requested review from subodh1810 and rodireich October 11, 2022 16:27

Documentation

068fd16

Addressing comments

0bf8b88

Addressing comments

163a0dc

akashkulk added 2 commits October 11, 2022 17:02

Addressing comments

f096c08

Merge branch 'master' into akash/cdc-tz

7945e53

Merge branch 'master' into akash/cdc-tz

5dc920a

akashkulk changed the title ~~🎉 Expose serverTimezone debezium option via MySQL Source spec for CDC~~ 🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC Oct 12, 2022

octavia-squidington-iii and others added 2 commits October 12, 2022 16:48

auto-bump connector version [ci skip]

93ad9cd

Merge branch 'master' into akash/cdc-tz

3cace3e

akashkulk temporarily deployed to more-secrets October 12, 2022 16:51 Inactive

Merge branch 'master' into akash/cdc-tz

9f372ee

akashkulk merged commit e1701c9 into master Oct 12, 2022

akashkulk deleted the akash/cdc-tz branch October 12, 2022 16:53

akashkulk temporarily deployed to more-secrets October 12, 2022 16:53 Inactive

This was referenced Oct 13, 2022

Bump Airbyte version from 0.40.14 to 0.40.15 #17961

Closed

Bump Airbyte version from 0.40.14 to 0.40.15 #17970

Merged

michaelsonnle mentioned this pull request Mar 29, 2023

Source MySQL - sync CDC does not work because of changed mysql server timezone #24659

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC #17815

🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC #17815

akashkulk commented Oct 11, 2022 •

edited

Loading

github-actions bot commented Oct 11, 2022

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Oct 11, 2022

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 12, 2022

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Oct 12, 2022

github-actions bot commented Oct 12, 2022

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Oct 12, 2022

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Oct 12, 2022

github-actions bot commented Oct 12, 2022

🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC #17815

🎉 MySQL Source : Expose serverTimezone debezium option via MySQL Source spec for CDC #17815

Conversation

akashkulk commented Oct 11, 2022 • edited Loading

What

🚨 User Impact 🚨

github-actions bot commented Oct 11, 2022

akashkulk commented Oct 11, 2022 • edited by github-actions bot Loading

Build Failed

akashkulk commented Oct 11, 2022 • edited by github-actions bot Loading

Build Failed

github-actions bot commented Oct 11, 2022

akashkulk commented Oct 11, 2022 • edited by github-actions bot Loading

Build Passed

akashkulk commented Oct 11, 2022 • edited by github-actions bot Loading

Build Passed

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 11, 2022

github-actions bot commented Oct 12, 2022

akashkulk commented Oct 12, 2022 • edited by github-actions bot Loading

Build Passed

github-actions bot commented Oct 12, 2022

github-actions bot commented Oct 12, 2022

akashkulk commented Oct 12, 2022 • edited by github-actions bot Loading

Build Passed

github-actions bot commented Oct 12, 2022

akashkulk commented Oct 12, 2022 • edited by github-actions bot Loading

akashkulk commented Oct 12, 2022 • edited by github-actions bot Loading

github-actions bot commented Oct 12, 2022

github-actions bot commented Oct 12, 2022

akashkulk commented Oct 11, 2022 •

edited

Loading

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 11, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading

akashkulk commented Oct 12, 2022 •

edited by github-actions bot

Loading