Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SnowflakeOperatorAsync Extractor Support #869

Merged
merged 2 commits into from
Jun 23, 2022

Conversation

denimalpaca
Copy link
Contributor

@denimalpaca denimalpaca commented Jun 22, 2022

Problem

Snowflake Extractor does not support Astronomer-provided Async Operator.

Closes: #860

Solution

Adds SnowflakeOperatorAsync to the SnowflakeExtractor operator classname list.

Checklist

  • You've signed-off your work
  • Your pull request title follows our guidelines
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
  • You've versioned the core OpenLineage model or facets according to SchemaVer (if relevant)

Signed-off-by: Benji Lampel <benjamin@astronomer.io>
@denimalpaca
Copy link
Contributor Author

@collado-mike @rossturk @mobuchowski this would be a really good, easy one for us to add so we can start getting more lineage on Astro provider operators!

@collado-mike
Copy link
Contributor

Looks good - just a minor tweak to the unit tests is needed to merge this.

@denimalpaca
Copy link
Contributor Author

@collado-mike I don't follow what needs to be changed in the unit tests. Like just adding a test checking that the get operators returns both items in the list?

@rossturk confirmed that the extractor does work on the async operator, it's showing up in the datakin UI, but the queries from the async operator aren't getting parsed it seems, because the queries aren't showing up and the connections between the operators aren't showing up.

@collado-mike
Copy link
Contributor

Signed-off-by: Benji Lampel <benjamin@astronomer.io>
@rossturk rossturk merged commit fdb3914 into OpenLineage:main Jun 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add SnowflakeAsyncOperator to Operator List in Snowflake Extractor
3 participants