Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use show objects in schema on Snowflake #2174

Closed
drewbanin opened this issue Mar 1, 2020 · 0 comments · Fixed by #2322
Closed

Use show objects in schema on Snowflake #2174

drewbanin opened this issue Mar 1, 2020 · 0 comments · Fixed by #2322
Labels
enhancement New feature or request help_wanted Trickier changes, with a clear starting point, good for previous/experienced contributors snowflake

Comments

@drewbanin
Copy link
Contributor

Describe the feature

dbt should leverage the show terse objects in schema query to list the tables and views in a database. Crucially, show .... queries do not require a running warehouse, so they will not queue in the virtual warehouse (leading to long perceived startup times).

Caveats:

  • Snowflake's docs indicate that only 10k records can be returned from one of these queries. I'd like to talk to someone from Snowflake to get an understanding if this limit is still accurate, or if they have near-term plans to lift this restriction.
  • The existing info schema query gets away with a case-insensitive lookup using `ilike, but we'll need to quote and case the schema name exactly for this query to succeed.
  • Per the docs, materialized views are returned with type VIEW, which is kind of unpleasant.
  • We should post-process the results in Python instead of using result_scan, because selecting from result_scan can queue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help_wanted Trickier changes, with a clear starting point, good for previous/experienced contributors snowflake
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant