Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validate_sql method to base adapter with implementation for SQLAdapters #8001

Merged
merged 8 commits into from
Jul 11, 2023

Commits on Jun 30, 2023

  1. Add dry_run method to base adapter with implementation for SQLAdapters

    resolves #7839
    
    In the CLI integration, MetricFlow will issue dry run queries as
    part of its warehouse-level validation of the semantic manifest,
    including all semantic model and metric definitions.
    
    In most cases, issuing an `explain` query is adequate, however,
    BigQuery does not support the `explain` keyword and so we cannot
    simply pre-pend `explain` to our input queries and expect the
    correct behavior across all contexts.
    
    This commit adds a dry_run() method to the BaseAdapter which mirrors
    the execute() method in that it simply delegates to the ConnectionManager.
    It also adds a working implementation to the SQLConnectionManager and
    includes a few test cases for adapter maintainers to try out on their own.
    
    The current implementation should work out of the box with most
    of our adapters. BigQuery will require us to implement the dry_run
    method on the BigQueryConnectionManager, and community-maintained
    adapters can opt in by enabling the test and ensuring their own
    implementations work as expected.
    
    Note - we decided to make these concrete methods that throw runtime
    exceptions for direct descendants of BaseAdapter in order to avoid
    forcing community adapter maintainers to implement a method that does
    not currently have any use cases in dbt proper.
    tlento committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    3a99352 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2023

  1. Switch dry_run implementation to be macro-based

    The common pattern for engine-specific SQL statement construction
    in dbt is to provide a default macro which can then be overridden
    on a per-adapter basis by either adapter maintainers or end users.
    The advantage of this is users can take advantage of alternative
    SQL syntax for performance or other reasons, or even to enable
    local usage if an engine relies on a non-standard expression and
    the adapter maintainer has not updated the package.
    
    Although there are some risks here they are minimal, and the benefit
    of added expressiveness and consistency with other similar constructs
    is clear, so we adopt this approach here.
    tlento committed Jul 1, 2023
    Configuration menu
    Copy the full SHA
    d71a162 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2023

  1. Configuration menu
    Copy the full SHA
    3b74488 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    32bf27a View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. Configuration menu
    Copy the full SHA
    2b8592b View commit details
    Browse the repository at this point in the history
  2. Rename dry_run to validate_sql

    The validate_sql name has less chance of colliding with dbt's
    command nomenclature, both now and in some future where we have
    dry-run operations.
    tlento committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    83950d7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7d2a6f6 View commit details
    Browse the repository at this point in the history
  4. Fix changelog entry

    tlento committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    58b82d6 View commit details
    Browse the repository at this point in the history