-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 airbyte-ci: skip archived connectors pre-emptively when using --modified flag #44786
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I love this. I think @alafanechere had a PR that did this already? Didn't spot it now. Anyway, LGTM
active_connectors = {conn for conn in all_connectors if conn.support_level != "archived"} | ||
modified_connectors = set() | ||
|
||
for connector in all_connectors: | ||
if connector.support_level == "archived": | ||
main_logger.info(f"Skipping connector '{connector}' due to 'archived' support level.") | ||
continue | ||
for connector in active_connectors: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that the per connector logging message is quite annoying. Would you mind keeping a single log line like f"Skipping {len(all_connectors) - len(active_connectors)} archived connectors"
. I'd love to keep it explicit somewhere that airbyte-ci filters out archived connectors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense, I added the log as you suggested!
I noticed it was being printed multiple times because I had added the mapping of active connectors to the _find_modified_connectors
function, which runs on a loop for each modified file. Fixed it by moving the mapping to the outer get_modified_connectors
function, so now we check for and remove archived
connectors once before entering the loop.
What
Adds logic to pre-emptively skip
archived
connectors when using the--modified
flag in airbyte-ci. This should clean up the test logs in CI and make it easier to reach the test reports:Resolves #9176
How
archived
) to iterate through when checking for modified connectors.User Impact
This change is not geared toward improving performance; it is intended to reduce clutter in logs when checking PR test reports.
Can this PR be safely reverted and rolled back?