Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

airbyte-ci: fix format cli usage in airbyte-enterprise #43386

Conversation

postamar
Copy link
Contributor

@postamar postamar commented Aug 8, 2024

What

Fixes bugs which prevented airbyte-ci format fix all from working in airbyte-enterprise when run straight from the CLI.
Also includes a QoL fix which makes maven run in batch mode instead of interactive mode, this makes the execution logs nicer.

How

Inclusion and exclusion rules have been corrected to handle symlinks propertly.

Review guide

One commit at a time.

User Impact

None.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Copy link

vercel bot commented Aug 8, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Aug 8, 2024 2:32pm

@postamar postamar force-pushed the postamar/airbyte-ci-support-format-cli-usage-in-airbyte-enterprise branch from f05a477 to 7c508ed Compare August 8, 2024 14:31
@postamar postamar force-pushed the postamar/airbyte-ci-support-format-cli-usage-in-airbyte-enterprise branch from 7c508ed to 84e6a98 Compare August 8, 2024 14:32
@postamar postamar marked this pull request as ready for review August 8, 2024 14:32
@postamar postamar requested a review from a team as a code owner August 8, 2024 14:32
@@ -62,7 +67,7 @@ class Formatter(Enum):
WARM_UP_INCLUSIONS = {
Formatter.JAVA: [
"spotless-maven-pom.xml",
"tools/gradle/codestyle/java-google-style.xml",
"tools/", # Whole directory instead of just 'java-google-style.xml' to support airbyte-enterprise.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why this change is required. And I think it could impact performance as we mount more things during the "warm up" phase.

Copy link
Contributor Author

@postamar postamar Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In airbyte-enterprise we have a symlink to tools and including a file inside of that only works if the symlink works.

I understand why you'd be worried but tools only contains 1.2 MB of files which basically never change. I think it's OK.

include = WARM_UP_INCLUSIONS.get(formatter)
if not include:
return None
include = include + [f"{AIRBYTE_SUBMODULE_DIR_NAME}/{i}" for i in include]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we check thaht AIRBYTE_SUBMODULE_DIR_NAME exists first? I'm not sure if dagger fails if you tell it to include not existing path (as the submodule does not exists in the main repo)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, conveniently enough, it's fine if the path doesn't exist!

Copy link
Contributor

@alafanechere alafanechere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your replies! format ran successfully on this branch so it means you did not break it 😄

@postamar postamar merged commit 64c3870 into master Aug 8, 2024
39 checks passed
@postamar postamar deleted the postamar/airbyte-ci-support-format-cli-usage-in-airbyte-enterprise branch August 8, 2024 15:24
LouisAuneau pushed a commit to LouisAuneau/airbyte that referenced this pull request Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants