🐛 Destination MySQL: fix problem with data containing nested JSON

[julianopiovezan]

What

Fix #4583.

How

When a source has a text field containing a JSON, the double quotes of this field in the AirbyteRecordMessage must be escaped with a backslash, following the spec.

Here is a sample of a temp file created by writeBatchToFile

ee9e9984-083a-439b-8a1d-f9178181fb01,"{""id"":1,""topic"":""install"",""timestamp"":""2017-08-22T17:22:15Z"",""model"":""install"",""details"":""{\""name\"":\""Conferência Faturamento - Custo - Taxas - Margem - Resumo ano inicial até -2\"",\""description\"":null}""}",2021-07-08 00:58:59.889

The proposed solution is to use the ESCAPED BY clause of the MySQL LOAD DATA statement so the sequence \" will be treated as part of the field value.

Recommended reading order

1. x.java
2. y.python

Pre-merge Checklist

Expand the checklist which is relevant for this PR.

Connector checklist

• Issue acceptance criteria met
• PR name follows PR naming conventions
• Secrets are annotated with airbyte_secret in the connector's spec
• Credentials added to Github CI if needed and not already present. instructions for injecting secrets into CI.
• Unit & integration tests added as appropriate (and are passing)
  • Community members: please provide proof of this succeeding locally e.g: screenshot or copy-paste acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• /test connector=connectors/<name> command as documented here is passing.
  • Community members can skip this, Airbyters will run this for you.
• Code reviews completed
• Documentation updated
  • README.md
  • docs/SUMMARY.md if it's a new connector
  • Created or updated reference docs in docs/integrations/<source or destination>/<name>.
  • Changelog in the appropriate page in docs/integrations/.... See changelog example
  • docs/integrations/README.md contains a reference to the new connector
  • Build status added to build page
• Build is successful
• Connector version bumped like described here
• New Connector version released on Dockerhub by running the /publish command described here
• No major blockers
• PR merged into master branch
• Follow up tickets have been created
• Associated tickets have been closed & stakeholders notified

Connector Generator checklist

• Issue acceptance criteria met
• PR name follows PR naming conventions
• If adding a new generator, add it to the list of scaffold modules being tested
• The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
• Documentation which references the generator is updated as needed.

[marcosmarxm]

Thanks for the contribution @julianopiovezan! LGTM to me, @subodh1810 can you validate it too?

[marcosmarxm]

@julianopiovezan I also create #4671 to insert a unit test to validate and ensure this problem doesnt happen again for all jdbc-destination connector

[marcosmarxm]

Also already create PR #4671 to test and bump version of this modification.

[julianopiovezan]

@julianopiovezan I also create #4671 to insert a unit test to validate and ensure this problem doesnt happen again for all jdbc-destination connector

Nice, thanks @marcosmarxm.

[subodh1810]

This looks fine. It would be great if we can add a test case for this

[dislazy]

This looks fine. It would be great if we can add a test case for this

I also encountered the same problem, I hope I can help solve it as soon as possible, thank you

[etsybaev]

Hope that will be fixed as part of #4825

[marcosmarxm]

@etsybaev can we merge this and add test in #4825?

[etsybaev]

Hi @marcosmarxm this PR has been created 10 days ago, but probably author doesn't have time to finish it. My PR contains both fix, test and it's based on the latest master. I also have a bumped versions. So I would rather cancel this PR and merge it all at one shot to have a git history consistent (in my PR). What do you think?

[marcosmarxm]

@etsybaev could we merge this and you pull master/solve conflicts in your side?

[etsybaev]

Hi @marcosmarxm. My PR with the fix, test and bumped versions was merged to master 2 days ago.#4825
So I assume we may close this one without future actions.

[marcosmarxm]

@julianopiovezan thanks a lot for the contribution and guide the solution for the problem. I'm closing your PR because #4825 add tests to the problem. Hope for future contributions!

[julianopiovezan]

Sure, no problem @marcosmarxm. I'm glad to help.