Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[airbyte-ci] change our python connector build process to use our base images #30450

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Sep 14, 2023

What

Closes #30236
Make our Python connector build step use the base image versions declared in the airbyte-connectors-base-images package.

How

  • Change BuildConnectorImage to build a connector from the base image version it defined in the metadata, otherwise fallback to a dockerfile build.
  • Unit test these changes.

TODO

Fix somehting in #30303 to avoid the CI failure.

🚨 User Impact 🚨

This will impact the connector modified in #30449 and other future migrated connector

Next steps:

Implement the pre/post build hook logic: #30237

@alafanechere alafanechere marked this pull request as ready for review September 14, 2023 22:10
@alafanechere alafanechere requested a review from a team September 14, 2023 22:19
@octavia-squidington-iii octavia-squidington-iii removed area/documentation Improvements or additions to documentation area/connectors Connector related issues labels Sep 14, 2023
@alafanechere alafanechere force-pushed the 09-14-add_connectorBuildOptions_to_connector_metadata branch from 662ef9b to 577b1c2 Compare September 14, 2023 22:36
@alafanechere alafanechere force-pushed the 09-14-change_our_python_connector_build_process_to_use_the_base_images branch from d882792 to 6f5dfbe Compare September 14, 2023 22:38
@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Sep 14, 2023
@alafanechere alafanechere force-pushed the 09-14-change_our_python_connector_build_process_to_use_the_base_images branch from 6f5dfbe to 4c403fb Compare September 14, 2023 22:39
@github-actions
Copy link
Contributor

source-zendesk-chat test report (commit 6f5dfbe556) - ❌

⏲️ Total pipeline duration: 22.54s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-zendesk-chat test

@github-actions
Copy link
Contributor

source-faker test report (commit 6f5dfbe556) - ❌

⏲️ Total pipeline duration: 49.92s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-faker test

@github-actions
Copy link
Contributor

source-file-secure test report (commit 6f5dfbe556) - ❌

⏲️ Total pipeline duration: 01mn07s

Step Result

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-file-secure test

@alafanechere alafanechere force-pushed the 09-14-change_our_python_connector_build_process_to_use_the_base_images branch from 4c403fb to 05e2881 Compare September 14, 2023 22:44
@octavia-squidington-iii octavia-squidington-iii removed the area/documentation Improvements or additions to documentation label Sep 14, 2023
@alafanechere alafanechere merged commit 5c6257a into 09-14-add_connectorBuildOptions_to_connector_metadata Sep 14, 2023
11 of 12 checks passed
@alafanechere alafanechere deleted the 09-14-change_our_python_connector_build_process_to_use_the_base_images branch September 14, 2023 22:46
@alafanechere alafanechere restored the 09-14-change_our_python_connector_build_process_to_use_the_base_images branch September 14, 2023 22:47
@github-actions
Copy link
Contributor

source-file test report (commit 6f5dfbe556) - ❌

⏲️ Total pipeline duration: 20mn05s

Step Result
Connector package install
Code format checks
Validate airbyte-integrations/connectors/source-file/metadata.yaml
Connector version semver check
Connector version increment check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-file test

@github-actions
Copy link
Contributor

source-faker test report (commit c7c25e3a64) - ✅

⏲️ Total pipeline duration: 01mn36s

Step Result
Connector package install
Build source-faker docker image for platform linux/x86_64
Unit tests
Acceptance tests
Code format checks
Validate airbyte-integrations/connectors/source-faker/metadata.yaml
Connector version semver check
QA checks

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-faker test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants