-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add validate_sql method to base adapter with implementation for SQLAdapters #8001
Merged
Commits on Jun 30, 2023
-
Add dry_run method to base adapter with implementation for SQLAdapters
resolves #7839 In the CLI integration, MetricFlow will issue dry run queries as part of its warehouse-level validation of the semantic manifest, including all semantic model and metric definitions. In most cases, issuing an `explain` query is adequate, however, BigQuery does not support the `explain` keyword and so we cannot simply pre-pend `explain` to our input queries and expect the correct behavior across all contexts. This commit adds a dry_run() method to the BaseAdapter which mirrors the execute() method in that it simply delegates to the ConnectionManager. It also adds a working implementation to the SQLConnectionManager and includes a few test cases for adapter maintainers to try out on their own. The current implementation should work out of the box with most of our adapters. BigQuery will require us to implement the dry_run method on the BigQueryConnectionManager, and community-maintained adapters can opt in by enabling the test and ensuring their own implementations work as expected. Note - we decided to make these concrete methods that throw runtime exceptions for direct descendants of BaseAdapter in order to avoid forcing community adapter maintainers to implement a method that does not currently have any use cases in dbt proper.
Configuration menu - View commit details
-
Copy full SHA for 3a99352 - Browse repository at this point
Copy the full SHA 3a99352View commit details
Commits on Jul 1, 2023
-
Switch dry_run implementation to be macro-based
The common pattern for engine-specific SQL statement construction in dbt is to provide a default macro which can then be overridden on a per-adapter basis by either adapter maintainers or end users. The advantage of this is users can take advantage of alternative SQL syntax for performance or other reasons, or even to enable local usage if an engine relies on a non-standard expression and the adapter maintainer has not updated the package. Although there are some risks here they are minimal, and the benefit of added expressiveness and consistency with other similar constructs is clear, so we adopt this approach here.
Configuration menu - View commit details
-
Copy full SHA for d71a162 - Browse repository at this point
Copy the full SHA d71a162View commit details
Commits on Jul 5, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 3b74488 - Browse repository at this point
Copy the full SHA 3b74488View commit details -
Configuration menu - View commit details
-
Copy full SHA for 32bf27a - Browse repository at this point
Copy the full SHA 32bf27aView commit details
Commits on Jul 6, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 2b8592b - Browse repository at this point
Copy the full SHA 2b8592bView commit details -
Rename dry_run to validate_sql
The validate_sql name has less chance of colliding with dbt's command nomenclature, both now and in some future where we have dry-run operations.
Configuration menu - View commit details
-
Copy full SHA for 83950d7 - Browse repository at this point
Copy the full SHA 83950d7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7d2a6f6 - Browse repository at this point
Copy the full SHA 7d2a6f6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 58b82d6 - Browse repository at this point
Copy the full SHA 58b82d6View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.