Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Postgres: Fix not being able to configure wal2json plugin #19985

Merged
merged 11 commits into from
Dec 6, 2022
Merged

Source Postgres: Fix not being able to configure wal2json plugin #19985

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
f1f5360
switch const to default
edgao Dec 1, 2022
2109ae5
update expected specs
edgao Dec 1, 2022
0311a5a
try updating system-stubs?
edgao Dec 1, 2022
d957a42
Revert "try updating system-stubs?"
edgao Dec 1, 2022
86644d2
switch to system stubs
edgao Dec 2, 2022
70b3757
Merge branch 'master' into edgao/source_postgres_cdc_plugin
edgao Dec 2, 2022
a577e95
Merge branch 'master' into edgao/source_postgres_cdc_plugin
edgao Dec 2, 2022
464744a
changelog
edgao Dec 2, 2022
18278f6
Merge branch 'master' into edgao/source_postgres_cdc_plugin
edgao Dec 2, 2022
c393bd0
Merge branch 'master' into edgao/source_postgres_cdc_plugin
edgao Dec 5, 2022
014a85c
fix more tests
edgao Dec 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .../integrations/io/airbyte/integration_tests/sources/CdcBinlogsMySqlSourceDatatypeTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ protected List<AirbyteMessage> runRead(final ConfiguredAirbyteCatalog configured
}

@Override
protected void setupEnvironment(final TestDestinationEnv environment) throws Exception {
protected void postSetup() throws Exception {
final Database database = setupDatabase();
initTests();
for (final TestDataHolder test : testDataHolders) {
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres/src/main/resources/spec.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@
"title": "Plugin",
"description": "A logical decoding plugin installed on the PostgreSQL server. The `pgoutput` plugin is used by default. If the replication table contains a lot of big jsonb values it is recommended to use `wal2json` plugin. Read more about <a href=\"https://docs.airbyte.com/integrations/sources/postgres#step-2-select-a-replication-plugin\">selecting replication plugins</a>.",
"enum": ["pgoutput", "wal2json"],
"const": "pgoutput",
"default": "pgoutput",
"order": 2
},
"replication_slot": {
Expand Down
2 changes: 1 addition & 1 deletion ...integrations/connectors/source-postgres/src/test-integration/resources/expected_spec.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@
"title": "Plugin",
"description": "A logical decoding plugin installed on the PostgreSQL server. The `pgoutput` plugin is used by default. If the replication table contains a lot of big jsonb values it is recommended to use `wal2json` plugin. Read more about <a href=\"https://docs.airbyte.com/integrations/sources/postgres#step-2-select-a-replication-plugin\">selecting replication plugins</a>.",
"enum": ["pgoutput", "wal2json"],
"const": "pgoutput",
"default": "pgoutput",
"order": 2
},
"replication_slot": {
Expand Down
2 changes: 1 addition & 1 deletion ...nnectors/source-postgres/src/test-integration/resources/expected_strict_encrypt_spec.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@
"title": "Plugin",
"description": "A logical decoding plugin installed on the PostgreSQL server. The `pgoutput` plugin is used by default. If the replication table contains a lot of big jsonb values it is recommended to use `wal2json` plugin. Read more about <a href=\"https://docs.airbyte.com/integrations/sources/postgres#step-2-select-a-replication-plugin\">selecting replication plugins</a>.",
"enum": ["pgoutput", "wal2json"],
"const": "pgoutput",
"default": "pgoutput",
"order": 2
},
"replication_slot": {
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres/src/test/resources/expected_spec.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@
"title": "Plugin",
"description": "A logical decoding plugin installed on the PostgreSQL server. The `pgoutput` plugin is used by default. If the replication table contains a lot of big jsonb values it is recommended to use `wal2json` plugin. Read more about <a href=\"https://docs.airbyte.com/integrations/sources/postgres#step-2-select-a-replication-plugin\">selecting replication plugins</a>.",
"enum": ["pgoutput", "wal2json"],
"const": "pgoutput",
"default": "pgoutput",
"order": 2
},
"replication_slot": {
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/postgres.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,7 @@ The root causes is that the WALs needed for the incremental sync has been remove

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:-------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| (none) | 2022-12-02 | [19985](https://github.com/airbytehq/airbyte/pull/19985) | Reenable incorrectly-disabled `wal2json` CDC plugin |
| 1.0.30 | 2022-11-29 | [19024](https://github.com/airbytehq/airbyte/pull/19024) | Skip tables from schema where user do not have Usage permission during discovery. |
| 1.0.29 | 2022-11-29 | [19623](https://github.com/airbytehq/airbyte/pull/19623) | Mark PSQLException related to using replica that is configured as a hot standby server as config error. |
| 1.0.28 | 2022-11-28 | [19514](https://github.com/airbytehq/airbyte/pull/19514) | Adjust batch selection memory limits databases. |
Expand Down