🐛 airbyte-ci: skip archived connectors pre-emptively when using --modified flag #44786
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
There was a problem hiding this comment.
I love this. I think @alafanechere had a PR that did this already? Didn't spot it now. Anyway, LGTM
| active_connectors = {conn for conn in all_connectors if conn.support_level != "archived"} | ||
| modified_connectors = set() | ||
|
|
||
| for connector in all_connectors: | ||
| if connector.support_level == "archived": | ||
| main_logger.info(f"Skipping connector '{connector}' due to 'archived' support level.") | ||
| continue | ||
| for connector in active_connectors: |
There was a problem hiding this comment.
I agree that the per connector logging message is quite annoying. Would you mind keeping a single log line like f"Skipping {len(all_connectors) - len(active_connectors)} archived connectors". I'd love to keep it explicit somewhere that airbyte-ci filters out archived connectors.
There was a problem hiding this comment.
Makes sense, I added the log as you suggested!
I noticed it was being printed multiple times because I had added the mapping of active connectors to the _find_modified_connectors function, which runs on a loop for each modified file. Fixed it by moving the mapping to the outer get_modified_connectors function, so now we check for and remove archived connectors once before entering the loop.
What
Adds logic to pre-emptively skip
archivedconnectors when using the--modifiedflag in airbyte-ci. This should clean up the test logs in CI and make it easier to reach the test reports:Resolves #9176
How
archived) to iterate through when checking for modified connectors.User Impact
This change is not geared toward improving performance; it is intended to reduce clutter in logs when checking PR test reports.
Can this PR be safely reverted and rolled back?