Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966) #6318

Merged
merged 5 commits into from
Sep 21, 2021

Conversation

alexandr-shegeda
Copy link
Contributor

@alexandr-shegeda alexandr-shegeda commented Sep 20, 2021

What

The goal of this PR is to add SSH only for the MSSQL Source. It does not try to deal with how it can be reused for other sources or destinations.
Took concept of how it was done for Postres

How

Handles injecting the SSH tunnel in the check, discover, read methods of MSSQL source directly. This makes it relatively straightforward to inject without worrying about touching other jdbc dbs.

Tested with AWS EC2 bastion and RDS MSSQL.

Recommended reading order

  1. MssqlSource.java
  2. AbstractSshMssqlSourceAcceptanceTest.java
  3. CdcMssqlSourceAcceptanceTest.java
  4. MssqlSourceAcceptanceTest.java
  5. etc.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

  • Community member? Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • docs/SUMMARY.md
    • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
    • docs/integrations/README.md
    • airbyte-integrations/builds.md
  • PR name follows PR naming conventions
  • Connector added to connector index like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions
  • Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

@github-actions github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Sep 20, 2021
@alexandr-shegeda
Copy link
Contributor Author

alexandr-shegeda commented Sep 20, 2021

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1254262473
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1254262473

@jrhizor jrhizor temporarily deployed to more-secrets September 20, 2021 16:06 Inactive
# Conflicts:
#	.github/workflows/publish-command.yml
#	.github/workflows/test-command.yml
@alexandr-shegeda alexandr-shegeda temporarily deployed to more-secrets September 20, 2021 16:54 Inactive
@alexandr-shegeda alexandr-shegeda linked an issue Sep 20, 2021 that may be closed by this pull request
@alexandr-shegeda alexandr-shegeda temporarily deployed to more-secrets September 20, 2021 19:02 Inactive
docs/integrations/sources/mssql.md Outdated Show resolved Hide resolved
@alexandr-shegeda alexandr-shegeda temporarily deployed to more-secrets September 20, 2021 19:10 Inactive
@alexandr-shegeda alexandr-shegeda marked this pull request as ready for review September 21, 2021 08:52
@alexandr-shegeda
Copy link
Contributor Author

alexandr-shegeda commented Sep 21, 2021

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1256898815
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1256898815
No Python unittests run

@alexandr-shegeda alexandr-shegeda temporarily deployed to more-secrets September 21, 2021 08:55 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets September 21, 2021 08:57 Inactive
@alexandr-shegeda
Copy link
Contributor Author

alexandr-shegeda commented Sep 21, 2021

/publish connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1257533989
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1257533989

@jrhizor jrhizor temporarily deployed to more-secrets September 21, 2021 12:17 Inactive
@alexandr-shegeda alexandr-shegeda merged commit 59cd5ca into master Sep 21, 2021
@alexandr-shegeda alexandr-shegeda deleted the asheg/5966-mssql-source-add-ssh branch September 21, 2021 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support & test cases for MSSQL Source via SSH tunnels
5 participants