-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handling configuration exceptions in IntegrationRunner #18989
Conversation
Affected Connector ReportNOTE
|
Connector | Version | Changelog | Publish |
---|---|---|---|
source-alloydb |
1.0.17 |
✅ | ✅ |
source-alloydb-strict-encrypt |
1.0.17 |
✅ | ⚠ (not in seed) |
source-bigquery |
0.2.3 |
✅ | ✅ |
source-clickhouse |
0.1.14 |
✅ | ✅ |
source-clickhouse-strict-encrypt |
0.1.14 |
✅ | ⚠ (not in seed) |
source-cockroachdb |
0.1.18 |
✅ | ✅ |
source-cockroachdb-strict-encrypt |
0.1.18 |
✅ | ⚠ (not in seed) |
source-db2 |
0.1.16 |
✅ | ✅ |
source-db2-strict-encrypt |
0.1.16 |
✅ | ⚠ (not in seed) |
source-e2e-test |
2.1.3 |
✅ | ✅ |
source-e2e-test-cloud |
2.1.1 |
⚠ (doc not found) |
⚠ (not in seed) |
source-elasticsearch |
0.1.0 |
✅ | ✅ |
source-jdbc |
0.3.5 |
⚠ (doc not found) |
⚠ (not in seed) |
source-kafka |
0.2.0 |
✅ | ✅ |
source-mongodb-strict-encrypt |
0.1.19 |
⚠ (doc not found) |
⚠ (not in seed) |
source-mongodb-v2 |
0.1.19 |
✅ | ✅ |
source-mssql |
0.4.25 |
✅ | ✅ |
source-mssql-strict-encrypt |
0.4.25 |
✅ | ⚠ (not in seed) |
source-mysql |
1.0.11 |
✅ | ✅ |
source-mysql-strict-encrypt |
1.0.11 |
✅ | ⚠ (not in seed) |
source-oracle |
0.3.21 |
✅ | ✅ |
source-oracle-strict-encrypt |
0.3.21 |
✅ | ⚠ (not in seed) |
source-postgres |
1.0.22 |
✅ | ✅ |
source-postgres-strict-encrypt |
1.0.22 |
✅ | ⚠ (not in seed) |
source-redshift |
0.3.15 |
✅ | ✅ |
source-relational-db |
0.3.1 |
⚠ (doc not found) |
⚠ (not in seed) |
source-scaffold-java-jdbc |
0.1.0 |
⚠ (doc not found) |
⚠ (not in seed) |
source-sftp |
0.1.2 |
✅ | ✅ |
source-snowflake |
0.1.24 |
✅ | ✅ |
source-tidb |
0.2.1 |
✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
❌ Destinations (42)
Connector | Version | Changelog | Publish |
---|---|---|---|
destination-azure-blob-storage |
0.1.6 |
⚠ (doc not found) |
✅ |
destination-bigquery |
1.2.5 |
✅ | ✅ |
destination-bigquery-denormalized |
1.2.5 |
⚠ (doc not found) |
✅ |
destination-cassandra |
0.1.4 |
❌ (changelog missing) |
✅ |
destination-clickhouse |
0.2.0 |
✅ | ✅ |
destination-clickhouse-strict-encrypt |
0.2.0 |
✅ | ⚠ (not in seed) |
destination-csv |
0.2.10 |
⚠ (doc not found) |
✅ |
destination-databricks |
0.3.1 |
✅ | ✅ |
destination-dev-null |
0.2.7 |
⚠ (doc not found) |
⚠ (not in seed) |
destination-dynamodb |
0.1.7 |
✅ | ✅ |
destination-e2e-test |
0.2.4 |
✅ | ✅ |
destination-elasticsearch |
0.1.6 |
✅ | ✅ |
destination-elasticsearch-strict-encrypt |
0.1.6 |
✅ | ⚠ (not in seed) |
destination-gcs |
0.2.12 |
✅ | ✅ |
destination-jdbc |
0.3.14 |
⚠ (doc not found) |
⚠ (not in seed) |
destination-kafka |
0.1.10 |
❌ (changelog missing) |
✅ |
destination-keen |
0.2.4 |
❌ (changelog missing) |
✅ |
destination-kinesis |
0.1.5 |
❌ (changelog missing) |
✅ |
destination-local-json |
0.2.11 |
✅ | ✅ |
destination-mariadb-columnstore |
0.1.7 |
✅ | ✅ |
destination-mongodb |
0.1.8 |
✅ | ✅ |
destination-mongodb-strict-encrypt |
0.1.8 |
✅ | ⚠ (not in seed) |
destination-mqtt |
0.1.3 |
✅ | ✅ |
destination-mssql |
0.1.22 |
✅ | ✅ |
destination-mssql-strict-encrypt |
0.1.22 |
✅ | ⚠ (not in seed) |
destination-mysql |
0.1.20 |
✅ | ✅ |
destination-mysql-strict-encrypt |
❌ 0.1.21 (mismatch: 0.1.20 ) |
✅ | ⚠ (not in seed) |
destination-oracle |
0.1.19 |
✅ | ✅ |
destination-oracle-strict-encrypt |
0.1.19 |
✅ | ⚠ (not in seed) |
destination-postgres |
0.3.26 |
✅ | ✅ |
destination-postgres-strict-encrypt |
0.3.26 |
✅ | ⚠ (not in seed) |
destination-pubsub |
0.1.6 |
❌ (changelog missing) |
✅ |
destination-pulsar |
0.1.3 |
❌ (changelog missing) |
✅ |
destination-r2 |
0.1.0 |
✅ | ✅ |
destination-redis |
0.1.4 |
✅ | ✅ |
destination-redshift |
0.3.51 |
✅ | ✅ |
destination-rockset |
0.1.4 |
❌ (changelog missing) |
✅ |
destination-s3 |
0.3.17 |
✅ | ✅ |
destination-scylla |
0.1.3 |
✅ | ✅ |
destination-snowflake |
0.4.38 |
✅ | ✅ |
destination-tidb |
0.1.0 |
✅ | ✅ |
destination-yugabytedb |
0.1.0 |
✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
Actionable Items
(click to expand)
Category | Status | Actionable Item |
---|---|---|
Version | ❌ mismatch |
The version of the connector is different from its normal variant. Please bump the version of the connector. |
⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
|
Changelog | ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
❌ changelog missing |
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog. | |
Publish | ⚠ not in seed |
The connector is not in the seed file (e.g. source_definitions.yaml ), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug. |
❌ diff seed version |
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version. |
/test connector=connectors/source-postgres
Build PassedTest summary info:
|
/test connector=connectors/source-postgres-strict-encrypt
Build PassedTest summary info:
|
...tegrations/bases/base-java/src/main/java/io/airbyte/integrations/base/IntegrationRunner.java
Outdated
Show resolved
Hide resolved
...tegrations/bases/base-java/src/main/java/io/airbyte/integrations/base/IntegrationRunner.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nothing blocking, just minor comments on keeping things either the same or spacing out for readability
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some of the format changes can be auto fixed by running ./gradlew :spotlessJavaApply
. This command will format all Java code. You can then check in format updates relevant to your changes.
* master: (69 commits) 🪟 🐛 Fix wrong geography dropdown type #19021 SAT: basic read on full catalog when `test_strictness_level == high` (#18937) Unhide DynamoDB destination (#18994) Fixed tests for destination connectors (#19007) 🐛 Source Facebook Marketing: handle FacebookBadObjectError (#18971) Edit multi-cloud docs (#18972) 🪟 🎉 Load credits consumption separate (#18986) Bmoric/extract source api (#18944) Migrating InvalidCursorException -> ConfigErrorException (#18995) 🪟 🎨 Fix banner link color (#18978) Handling configuration exceptions in IntegrationRunner (#18989) Add new workspace api endpoint (#18983) Add normalization to destination definition and actor definition table (#18300) Fix oauth controller (#18981) Fix migration dev center schema dump by run db-specific initialization script (#18984) fix master build failure (#18982) cleanup: delete debezium 1-4-2 module (#18733) Remove unused job persistence methods. (#18952) Hash filenames of extracted CSS (#18976) Fix typo in source code comment DataDaog ==> Datadog (#18911) ...
* master: (73 commits) 🪟 🐛 Fix wrong geography dropdown type #19021 SAT: basic read on full catalog when `test_strictness_level == high` (#18937) Unhide DynamoDB destination (#18994) Fixed tests for destination connectors (#19007) 🐛 Source Facebook Marketing: handle FacebookBadObjectError (#18971) Edit multi-cloud docs (#18972) 🪟 🎉 Load credits consumption separate (#18986) Bmoric/extract source api (#18944) Migrating InvalidCursorException -> ConfigErrorException (#18995) 🪟 🎨 Fix banner link color (#18978) Handling configuration exceptions in IntegrationRunner (#18989) Add new workspace api endpoint (#18983) Add normalization to destination definition and actor definition table (#18300) Fix oauth controller (#18981) Fix migration dev center schema dump by run db-specific initialization script (#18984) fix master build failure (#18982) cleanup: delete debezium 1-4-2 module (#18733) Remove unused job persistence methods. (#18952) Hash filenames of extracted CSS (#18976) Fix typo in source code comment DataDaog ==> Datadog (#18911) ...
Breaking off this PR: #18560
into smaller pieces.
IntegrationRunner.java
: All config errors are handled here, rather than in the individual protocol methods (read(), check() etc). A trace message is emitted for every config error.ConfigErrorException.java
: New exception classConfigErrorException
indicates an issue with the user's configuration.This should unblock other use cases of throwing a
ConfigErrorException
. Remaining work is still required to: