feat(airbyte-ci): include components.py in manifest-only build #44879
+76
−7
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
ChristoGrab
changed the title
2 tasks
…/github.com/airbytehq/airbyte into christo/manifest-only-build-with-components
bnchrch
requested changes
Aug 29, 2024
| base_container = self._get_base_container(platform).with_file( | ||
| f"source_declarative_manifest/{MANIFEST_FILE_PATH}", | ||
| (await self.context.get_connector_dir(include=[MANIFEST_FILE_PATH])).file(MANIFEST_FILE_PATH), | ||
| ) | ||
|
|
||
| # Mount components file if it exists | ||
| components_file = self.context.connector.code_directory / COMPONENTS_FILE_PATH |
...tors/pipelines/pipelines/airbyte_ci/connectors/build_image/steps/manifest_only_connectors.py
Show resolved
Hide resolved
…/github.com/airbytehq/airbyte into christo/manifest-only-build-with-components
ChristoGrab
commented
Sep 3, 2024
| @@ -183,8 +183,7 @@ async def test_check_path_in_workdir(dagger_client): | |||
| .with_workdir(str(connector.code_directory)) | |||
| ) | |||
| assert await utils.check_path_in_workdir(container, "metadata.yaml") | |||
| assert await utils.check_path_in_workdir(container, "pyproject.toml") | |||
| assert await utils.check_path_in_workdir(container, "poetry.lock") | |||
| assert await utils.check_path_in_workdir(container, "manifest.yaml") | |||
There was a problem hiding this comment.
Updated this test as this connector is now manifest-only and does not have poetry files
bnchrch
approved these changes
Sep 3, 2024
| @@ -23,7 +23,7 @@ def mock_diffed_branched(mocker): | |||
|
|
|||
| @pytest.fixture | |||
| def pokeapi_metadata_path(): | |||
| return "airbyte-integrations/connectors/source-pokeapi/metadata.yaml" | |||
| return "airbyte-integrations/connectors/source-zoho-crm/metadata.yaml" | |||
There was a problem hiding this comment.
❓ This change doesnt seem to belong?
There was a problem hiding this comment.
Haha sorry, was typing the explanation at the same time, here 'tis:
Updated as source Pokeapi is now manifest-only, which breaks a couple test cases for expected reviewers (ie, marketplace team gets added). Zoho CRM is written in Python, which makes it unlikely to be migrated to manifest-only anytime soon.
| @@ -37,7 +37,7 @@ def not_tracked_change_expected_team(tmp_path, pokeapi_metadata_path): | |||
| backup_path = tmp_path / "non_strategic_acceptance_test_config.backup" | |||
| shutil.copyfile(pokeapi_metadata_path, backup_path) | |||
| with open(pokeapi_metadata_path, "a") as metadata_file: | |||
| metadata_file.write("not_tracked") | |||
| metadata_file.write("\nnot_tracked: true\n") | |||
There was a problem hiding this comment.
❓ This change doesnt seem to belong too?
There was a problem hiding this comment.
This was me as well! This line was throwing a YAML SafeLoader error because it wasn't formatted as a k:v pair
| @@ -55,4 +58,5 @@ def mock_container(): | |||
| container_mock = AsyncMock(MockContainerClass) | |||
| container_mock.with_label.return_value = container_mock | |||
| container_mock.with_exec.return_value = container_mock | |||
| container_mock.with_file.return_value = container_mock | |||
ChristoGrab
commented
Sep 3, 2024
airbyte-ci/connectors/connector_ops/tests/test_required_reviewer_checks.py
Show resolved
Hide resolved
ChristoGrab
commented
Sep 3, 2024
airbyte-ci/connectors/connector_ops/tests/test_required_reviewer_checks.py
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Mounts
components.pywhen building a manifest-only connector, as a prerequisite to enabling support for custom components in manifest-only connectors, and the connector builder.Resolves #9527
How
Checks for an existing
components.pyfile at the root level of the connector's directory. If it exists, the file is mounted alongside the manifest.For existing low-code connectors,
components.pyresides in the python source code directory. It will be relocated to the root level of the folder alongisde the manifest when these connectors are migrated to manifest-only (Link to migration PR)Review guide
Link to loom
User Impact
None. This is a step to enabling support for custom components in manifest-only conenctors.
Can this PR be safely reverted and rolled back?