Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable record schema validation using v1 type system; CI uses MSG to start EC2 runners #20439

Merged
merged 10 commits into from
Dec 15, 2022

Conversation

edgao
Copy link
Contributor

@edgao edgao commented Dec 13, 2022

Unreverts #19625, and hardcodes build instead of using ${buildDir}. Relevant changes are in the two build.gradle files, plus the Dockerfile.

#19625 was causing the OSS release action to fail (https://github.com/airbytehq/airbyte/actions/runs/3625448595/jobs/6113524965) because the docker build couldn't find WellKnownTypes.json. This is working properly now.

Tested in https://github.com/airbytehq/airbyte/actions/runs/3699371623/jobs/6266680684#step:6:9612 - this action run failed because of an intentional COPY nonexistent_file.txt, rather than not being able to find WellKnownTypes.json. This action was running against https://github.com/airbytehq/airbyte/pull/20439/files/ba107138acde16cce98a659ecd70fa5ce6705c9b; the subsequent commits are just reverting the CI hacks.


Also switches the start ec2 runner action to use MSGv0.0.2 (which supports Multiple Security Groups). This is useful for being able to SSH onto CI runner instances.

@edgao edgao changed the title Add WellKnownTypes.json to worker Enable record schema validation using v1 type system Dec 13, 2022
@octavia-squidington-iv octavia-squidington-iv added area/platform issues related to the platform area/worker Related to worker labels Dec 13, 2022
@supertopher supertopher requested a review from a team December 13, 2022 17:41
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:07 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:07 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:07 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:10 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:10 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:48 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:49 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 22:49 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 23:25 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 14, 2022 23:25 — with GitHub Actions Inactive
@edgao edgao marked this pull request as ready for review December 14, 2022 23:28
@edgao edgao requested a review from a team as a code owner December 14, 2022 23:29
@edgao edgao requested review from gosusnp and git-phu December 14, 2022 23:29
Copy link
Contributor

@git-phu git-phu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still unclear why the original release workflow run failed, but that specific error no longer happens

@edgao edgao changed the title Enable record schema validation using v1 type system Enable record schema validation using v1 type system; CI uses MSG to start EC2 runners Dec 14, 2022
@edgao edgao merged commit 2392acb into master Dec 15, 2022
@edgao edgao deleted the edgao/wellknowntypes_in_worker branch December 15, 2022 00:06
edgao added a commit that referenced this pull request Dec 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/platform issues related to the platform area/worker Related to worker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants