Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[skip ci] Add connectorTestSuitesOptions to metadata #38188

Merged
merged 1 commit into from
May 15, 2024
Merged

[skip ci] Add connectorTestSuitesOptions to metadata #38188

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
9 changes: 9 additions & 0 deletions airbyte-integrations/connectors/destination-amazon-sqs/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,13 @@ data:
sl: 100
ql: 200
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-AMAZON-SQS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
16 changes: 16 additions & 0 deletions airbyte-integrations/connectors/destination-astra/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,20 @@ data:
tags:
- language:python
- cdk:python
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-ASTRA__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- suite: acceptanceTests
testSecrets:
- name: SECRET_DESTINATION-ASTRA__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
9 changes: 9 additions & 0 deletions airbyte-integrations/connectors/destination-aws-datalake/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,13 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-AWS-DATALAKE_CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
19 changes: 19 additions & 0 deletions airbyte-integrations/connectors/destination-azure-blob-storage/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,23 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-AZURE-BLOB-STORAGE-GCS-STAGING__CREDS
fileName: config_gcs_staging.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-AZURE-BLOB-STORAGE-S3-STAGING__CREDS
fileName: config_s3_staging.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-AZURE-BLOB-STORAGE__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
115 changes: 100 additions & 15 deletions airbyte-integrations/connectors/destination-bigquery/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,7 @@ data:
releases:
breakingChanges:
2.0.0:
message:
"**Do not upgrade until you have run a test upgrade as outlined [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#testing-destinations-v2-for-a-single-connection)**.

This version introduces [Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2),
which provides better error handling, incremental delivery of data for large
syncs, and improved final table structures. To review the breaking changes,
and how to upgrade, see [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#quick-start-to-upgrading).
These changes will likely require updates to downstream dbt / SQL models,
which we walk through [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#updating-downstream-transformations).

Selecting `Upgrade` will upgrade **all** connections using this destination
at their next sync. You can manually sync existing connections prior to
the next scheduled sync to start the upgrade early.

"
message: "**Do not upgrade until you have run a test upgrade as outlined [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#testing-destinations-v2-for-a-single-connection)**.\nThis version introduces [Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2), which provides better error handling, incremental delivery of data for large syncs, and improved final table structures. To review the breaking changes, and how to upgrade, see [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#quick-start-to-upgrading). These changes will likely require updates to downstream dbt / SQL models, which we walk through [here](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#updating-downstream-transformations).\nSelecting `Upgrade` will upgrade **all** connections using this destination at their next sync. You can manually sync existing connections prior to the next scheduled sync to start the upgrade early.\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗ were risking that the platform doesnt know how to parse the newlines.

I think thats fine though

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alternatively, define this message field value as a text block, c.f. https://yaml-multiline.info/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was hard to modify yaml and keep ordering, format and comments.
I used ruamel and could not find a way to not make it wrap stuff and add new lines...

upgradeDeadline: "2023-11-07"
resourceRequirements:
jobSpecific:
Expand All @@ -47,4 +33,103 @@ data:
supportsDbt: true
tags:
- language:java
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_1S1T_DISABLETD_GCS_RAW_OVERRIDE
fileName: credentials-1s1t-disabletd-gcs-raw-override.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_1S1T_DISABLETD_STANDARD_OVERRIDE
fileName: credentials-1s1t-disabletd-standard-raw-override.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_1S1T_GCS
fileName: credentials-1s1t-gcs.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_1S1T_GCS_RAW_OVERRIDE
fileName: credentials-1s1t-gcs-raw-override.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_1S1T_STANDARD
fileName: credentials-1s1t-standard.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_1S1T_STANDARD_RAW_OVERRIDE
fileName: credentials-1s1t-standard-raw-override.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_BAD_PROJECT_CREDS
fileName: credentials-badproject.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_GCS_STAGING
fileName: credentials-gcs-staging.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_IMPERSONATE_FAIL_CREDS
fileName: credentials-impersonate-fail.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_IMPERSONATE__CREDS
fileName: credentials-impersonate.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_NO_GCS_CREATE_ROLE
fileName: copy_gcs_no_create_roles_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_NO_GCS_WRITE_ROLE
fileName: copy_gcs_no_write_roles_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_NO_PUBLIC_SCHEMA_EDIT_ROLE
fileName: credentials-no-edit-public-schema-role.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_STANDARD
fileName: credentials-standard.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS_WITH_MISSED_DATASET_CREATION_ROLE__CREDS
fileName: credentials-with-missed-dataset-creation-role.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS__CREDS
fileName: credentials.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_CREDENTIALS__CREDS_OAUTH
fileName: credentials_oauth.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_STANDARD-NO-DATASET-CREATION__CREDS
fileName: credentials-standard-no-dataset-creation.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-BIGQUERY_STANDARD_NON_BILLABLE_PROJECT__CREDS
fileName: credentials-standard-non-billable-project.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-chroma/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,7 @@ data:
tags:
- language:python
- cdk:python
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-clickhouse-strict-encrypt/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,7 @@ data:
supportsDbt: false
tags:
- language:java
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-clickhouse/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,7 @@ data:
sl: 100
ql: 200
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-convex/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,7 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-csv/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,7 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
9 changes: 9 additions & 0 deletions airbyte-integrations/connectors/destination-cumulio/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,13 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-CUMULIO_CREDENTIALS__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
9 changes: 9 additions & 0 deletions airbyte-integrations/connectors/destination-databend/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,13 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION_DATABEND_CLOUD_CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
24 changes: 24 additions & 0 deletions airbyte-integrations/connectors/destination-databricks/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,28 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-DATABRICKS_AZURE__CREDS
fileName: azure_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-DATABRICKS__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION-DATABRICKS__STAGING_CREDS
fileName: staging_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- name: SECRET_DESTINATION_DATABRICKS_OAUTH_CONFIG
fileName: oauth_config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-dev-null/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,7 @@ data:
ql: 100
supportLevel: community
supportsRefreshes: true
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
16 changes: 16 additions & 0 deletions airbyte-integrations/connectors/destination-duckdb/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,20 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION_DUCKDB__MOTHERDUCK__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
- suite: acceptanceTests
testSecrets:
- name: SECRET_DESTINATION_DUCKDB__MOTHERDUCK__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
9 changes: 9 additions & 0 deletions airbyte-integrations/connectors/destination-dynamodb/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,13 @@ data:
sl: 100
ql: 200
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-DYNAMODB__CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-e2e-test/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,7 @@ data:
ql: 100
supportLevel: community
supportsRefreshes: true
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-elasticsearch-strict-encrypt/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,7 @@ data:
documentationUrl: https://docs.airbyte.com/integrations/destinations/elasticsearch
tags:
- language:java
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
3 changes: 3 additions & 0 deletions airbyte-integrations/connectors/destination-elasticsearch/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,7 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
metadataSpecVersion: "1.0"
9 changes: 9 additions & 0 deletions airbyte-integrations/connectors/destination-firebolt/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,13 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-FIREBOLT_CREDS
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
9 changes: 9 additions & 0 deletions airbyte-integrations/connectors/destination-firestore/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,13 @@ data:
sl: 100
ql: 100
supportLevel: community
connectorTestSuitesOptions:
- suite: unitTests
- suite: integrationTests
testSecrets:
- name: SECRET_DESTINATION-FIRESTORE
fileName: config.json
secretStore:
type: GSM
alias: airbyte-connector-testing-secret-store
metadataSpecVersion: "1.0"
Loading
Loading