Snapshot config doesn't work in schema.yml

[gshank]

Validation is performed on SnapshotConfig objects when the snapshot node is parsed/created. This validation means that config must be set in the sql file and can't be set in the schema file.

To allow setting in schema files we would need to defer validation until after schema file parsing.

[jtcohen6]

@gshank Ah! I now remember noticing this at one point, too. Thanks for keeping track of it and opening the issue.

The validation currently occurs within the validate method of SnapshotConfig, and I'm guessing that occurs before patching. Do you have a sense of where we could move the validation so that it happens post-patching?

[gshank]

It would have to happen as a post-schema-file parsing step, at about the same place in the manifest parser where we handle updating docs and refs.

[gshank]

Error message for this:
Encountered an error:
Snapshots must be configured with a 'strategy', 'unique_key', and 'target_schema'.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest; add a comment to notify the maintainers.

[jtcohen6]

Reopening this one as it's still relevant. Possibly related to the work Emily was just doing around models set to enabled: false in config .yml files (#5869)

[jtcohen6]

In the meantime, a functional workaround that I don't feel great about:

# dbt_project.yml
snapshots:
  # set defaults here to avoid validation errors during first parse
  strategy: check_cols
  unique_key: id
  target_schema: "{{ target.schema }}"

Then all snapshots will have the default values set when first parsed, allowing me to set snapshot-specific properties in other .yml files instead.

-- snapshots/my_snap.sql
{% snapshot my_snapshot() %}

    select
        1 as specific_id,
        current_timestamp as updated_at
        
{% endsnapshot %}

# snapshots/config.yml
version: 2
snapshots:
  - name: my_snapshot
    config:
      target_schema: specific_schema
      unique_key: specific_id
      strategy: timestamp
      updated_at: updated_at

Downside: If you forget to set those properties individually, for a real snapshot, you'll only find out at runtime.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

[FishtownBuildBot]

Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#5627