-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🎉 Redshift Source: create secure-only version #7040
Conversation
# Conflicts: # docs/integrations/destinations/redshift.md # docs/integrations/sources/redshift.md
vmaltsev seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
# Conflicts: # airbyte-integrations/connectors/destination-redshift/src/main/java/io/airbyte/integrations/destination/redshift/RedshiftCopyS3Destination.java # airbyte-integrations/connectors/destination-redshift/src/main/resources/spec.json # airbyte-integrations/connectors/source-redshift/src/main/java/io/airbyte/integrations/source/redshift/RedshiftSource.java # airbyte-integrations/connectors/source-redshift/src/main/resources/spec.json # docs/integrations/destinations/redshift.md
/test connector=connectors/source-redshift-strict-encrypt
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@VitaliiMaltsev, please follow and check off the Pre-merge Checklist
to properly register this new connector. Not everything is needed to add a strict encrypt connector, but currently some files are missing (e.g. registering the connector in the source_definitions.yaml
file.
Also don't forget to register this new connector in NormalizationRunnerFactory
.
...java/io/airbyte/integrations/source/redshift_strict_encrypt/RedshiftStrictEncryptSource.java
Outdated
Show resolved
Hide resolved
...java/io/airbyte/integrations/source/redshift_strict_encrypt/RedshiftStrictEncryptSource.java
Show resolved
Hide resolved
...rs/source-redshift/src/main/java/io/airbyte/integrations/source/redshift/RedshiftSource.java
Outdated
Show resolved
Hide resolved
...rs/source-redshift/src/main/java/io/airbyte/integrations/source/redshift/RedshiftSource.java
Outdated
Show resolved
Hide resolved
# Conflicts: # airbyte-integrations/connectors/destination-redshift/src/main/java/io/airbyte/integrations/destination/redshift/RedshiftCopyS3Destination.java # airbyte-integrations/connectors/source-redshift/src/main/java/io/airbyte/integrations/source/redshift/RedshiftSource.java
|
hi @tuliren |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do all redshift clusters support encrypted connections? (I don't remember off the top of my head)
In other words, can we safely assume that we can always use ssl=true
in the JDBC URL?
If so, then why don't we always do SSL by default? It eliminates the need for a connector fork and even having this as an option in redshift's config
@sherifnada Yes, the traffic will be encrypted for all redfshift clusters with the included ssl = true option in jdbc url. Do you mean not to create new strict-encrypt connectors, but to enable this option by default in the source-redshift and destination-redshift connectors? |
@VitaliiMaltsev correct. And also removing the option from the redshift connector and always setting it to true |
@sherifnada done in #7234. Please review |
LGTM we can close this one |
What
We want to create secure-only versions of connectors that can be used in the Airbyte cloud. The idea is that these connectors inherently prevent certain insecure connections such as connecting to a database over the public internet without encryption.
How
Modified the connector's spec to hide any options which allow the user to disable TLS. Changed the connector to enable TLS by default if the TLS option is not specified
Recommended reading order
x.java
y.python
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/SUMMARY.md
docs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampledocs/integrations/README.md
airbyte-integrations/builds.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing./publish
command described hereUpdating a connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing./publish
command described hereConnector Generator
-scaffold
in their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates
then checking in your changes