Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable SPEC SAT for Java sources #18779

Merged
merged 36 commits into from
Nov 9, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
838618e
enable py spec compatibility tests
DoNotPanicUA Nov 1, 2022
ce22526
add missing properties
DoNotPanicUA Nov 1, 2022
413439e
use expected spec file instead of the source spec.json + use dummy co…
DoNotPanicUA Nov 2, 2022
ab7ee60
add missing files for the clickhouse
DoNotPanicUA Nov 2, 2022
b968723
move test files to the integration-test from the unit test folder
DoNotPanicUA Nov 2, 2022
999d716
add missing files to cockroachdb
DoNotPanicUA Nov 2, 2022
d579289
add missing files to db2 + fix spec format
DoNotPanicUA Nov 2, 2022
1daeddb
add missing files to elasticsearch + fix spec format
DoNotPanicUA Nov 2, 2022
4ecef1c
add missing files to jdbc
DoNotPanicUA Nov 2, 2022
02a6bfa
add missing files to mongodb_v2 + fix spec format
DoNotPanicUA Nov 2, 2022
f747e13
add missing files to mssql + fix spec format
DoNotPanicUA Nov 3, 2022
d09b7f3
add missing files to mysql + fix spec format
DoNotPanicUA Nov 3, 2022
4d71efd
add missing files to postgres + fix spec format
DoNotPanicUA Nov 3, 2022
c46a5d3
add missing files to oracle + fix spec format
DoNotPanicUA Nov 3, 2022
189cdb6
add missing files to redshift
DoNotPanicUA Nov 3, 2022
46f1763
add missing files to sftp
DoNotPanicUA Nov 3, 2022
27f21ab
add missing files to snowflake + fix spec format
DoNotPanicUA Nov 3, 2022
c85f053
add missing files to tidb
DoNotPanicUA Nov 3, 2022
9601782
add missing files to kafka - fix spec format
DoNotPanicUA Nov 3, 2022
3d7ce65
Merge branch 'master' into aleonets/17580-enable-py-spec-test
grubberr Nov 4, 2022
a4813d2
airbyte-source-acceptance-test added
grubberr Nov 4, 2022
89b2b88
add missing import
DoNotPanicUA Nov 4, 2022
5d6036c
Delete acceptance-test-config.yml
DoNotPanicUA Nov 5, 2022
a560e82
Delete acceptance-test-docker.sh
DoNotPanicUA Nov 5, 2022
a02a0b7
Update build.gradle
DoNotPanicUA Nov 5, 2022
1b9664a
Update build.gradle
DoNotPanicUA Nov 5, 2022
564f116
format
DoNotPanicUA Nov 7, 2022
5f6a7af
Merge remote-tracking branch 'origin/aleonets/17580-enable-py-spec-te…
DoNotPanicUA Nov 7, 2022
98ff91e
revert changes
DoNotPanicUA Nov 7, 2022
60f467f
manual .sh files format
DoNotPanicUA Nov 7, 2022
39ca8fe
Merge branch 'master' into aleonets/17580-enable-py-spec-test
DoNotPanicUA Nov 7, 2022
116bf78
upd expected spec
DoNotPanicUA Nov 8, 2022
c67136b
Merge remote-tracking branch 'origin/aleonets/17580-enable-py-spec-te…
DoNotPanicUA Nov 9, 2022
b1ed0da
format
DoNotPanicUA Nov 9, 2022
235269b
fix SAT after master merge
DoNotPanicUA Nov 9, 2022
c5efca9
Merge branch 'master' into aleonets/17580-enable-py-spec-test
DoNotPanicUA Nov 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-bigquery:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
config_path: "secrets/config.json"
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please run ./gradlew format to format your files

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.sh files are out of the formatting scope, so I do it in an old fashion way 😄

Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these /integration_tests files are needed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@evantahler
if want to run the tests only locally as we see here, yes.
If you want to run it as part of the CI testing, you have to add it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The acceptance.py is indeed required to launch the test suite with the SAT plugin. We should find a better way because it's far from ideal. @DoNotPanicUA could you please create an issue about this?

# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#


import pytest

pytest_plugins = ("source_acceptance_test.plugin",)


@pytest.fixture(scope="session", autouse=True)
def connector_setup():
"""This fixture is a placeholder for external resources that acceptance test might require."""
# TODO: setup test dependencies if needed. otherwise remove the TODO comments
yield
# TODO: clean up test dependencies
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-clickhouse:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-cockroachdb:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
1 change: 1 addition & 0 deletions airbyte-integrations/connectors/source-db2/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-jdbc:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
1 change: 1 addition & 0 deletions airbyte-integrations/connectors/source-jdbc/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
id "java-library"
// https://docs.gradle.org/current/userguide/java_testing.html#sec:java_test_fixtures
id "java-test-fixtures"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-mongodb-v2:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ plugins {
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-performance-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ plugins {
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-performance-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ plugins {
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-performance-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-redshift:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-scaffold-java-jdbc:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env sh

# Build latest connector image
docker build . -t $(cat acceptance-test-config.yml | grep "connector_image" | head -n 1 | cut -d: -f2):dev

# Pull latest acctest image
docker pull airbyte/source-acceptance-test:latest

# Run
docker run --rm -it \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
-v $(pwd):/test_input \
airbyte/source-acceptance-test \
--acceptance-test-config /test_input
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ plugins {
id 'application'
id 'airbyte-docker'
id 'airbyte-integration-test-java'
id 'airbyte-source-acceptance-test'
}

application {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-sftp:dev
tests:
spec:
- spec_path: "src/main/resources/spec.json"
Loading