🎉 Destination MSSQL: Added support for connection via SSH tunnels (#5…

[sashaNeshcheret]

What

Improve the process of acceptance testing by allowing doing acceptance / integration testing using a bastion running in a docker container. Right now we have a dedicated host in AWS that we use.

Why do we care?

1. We'd rather not maintain the bastion infra if we don't have to

2. It means that the tests require secrets as an input which adds a bit more friction. If we use docker that would not be necessary.

3. It requires having a static instance of the database we trying to test running! Because the bastion is static it also needs to be forwarding to some database in our VPC so we need a statically running db as well. This limits the tests we can right and means we're paying for and maintaining a database for no good reason. Ideally we could spin up dbs and bastion all in docker!

How

Used existing SshBastion class, which allows you to initialize and run a bastion ssh server created in a docker test container. The test database and bastion run in containers on the same network. Connection configuration for integration tests is now taken directly from container settings and does not require a real database connection. Establishing an ssh tunnel implies 2 authentication options - using a password or using a private key

Recommended reading order

1. x.java
2. y.python

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

• Community member? Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • docs/SUMMARY.md
  • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • docs/integrations/README.md
  • airbyte-integrations/builds.md
• PR name follows PR naming conventions
• Connector added to connector index like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• Credentials added to Github CI. Instructions.
• /test connector=connectors/<name> command is passing.
• New Connector version released on Dockerhub by running the /publish command described here

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
• PR name follows PR naming conventions
• Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• Credentials added to Github CI. Instructions.
• /test connector=connectors/<name> command is passing.
• New Connector version released on Dockerhub by running the /publish command described here

Connector Generator

• Issue acceptance criteria met
• PR name follows PR naming conventions
• If adding a new generator, add it to the list of scaffold modules being tested
• The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
• Documentation which references the generator is updated as needed.

[CLAassistant]

All committers have signed the CLA.

[alexandr-shegeda]

please, update the changelog as well

[sashaNeshcheret]

/test connector=connectors/destination-mssql

🕑 connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/1286034362
✅ connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/1286034362
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    11      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     117      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 10      0   100%
	 normalization/transform_catalog/stream_processor.py                 349    199    43%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         141     36    74%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1037    391    62%

[sashaNeshcheret]

/publish connector=connectors/destination-mssql

🕑 connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/1286036502
✅ connectors/destination-mssql https://github.com/airbytehq/airbyte/actions/runs/1286036502