Expanded docs for deprecation_date

[dbeatty10]

What are you changing in this pull request and why?

Resolves #3745

• Updated docs for model versions including link back to the reference docs on deprecation_date
• Add examples with different accepted formats, including an aware datetime, a naive datetime, and a plain date
  • 1999-01-01 00:00:00.00+00:00
  • 2023-12-31 23:59:59.99
  • 2024-01-01

Previews

• deprecation_date
• model-versions

🎩

Key questions to answer

Why would I set a deprecation_date for my models?

Declaring a deprecation date is a way of signaling the maturity level of a model, and communicating plans for long-term support and maintenance. It provides a mechanism to communicate plans and timelines for sunsetting models so they don't need to be maintained and supported indefinitely and their build and storage costs can be freed up.

What happens after a model is deprecated?

Does it stop building/being selected? What are the warning/error messages I could expect to see, for myself and for downstream queriers?

Deprecated models can still continue to be built by producers and be selected by consumers until they are disabled or removed.

Models marked for deprecation will raise informative warnings when they are parsed or referenced. (With cross-project references, dbt raises these warnings in downstream projects that are ref'ing a deprecated version of a public model from an upstream project.)

When a project references a model that's slated for deprecation, a warning is generated. If it's a versioned model, with a newer version available, then the warning says so. This added bit of communication, from producers to consumers, is an advantage of using dbt's built-in functionality around model versions to facilitate migrations.

Additionally, WARN_ERROR_OPTIONS gives a mechanism whereby users can promote these warnings to actual runtime errors.

Warning messages:

• DeprecatedModel (warning when parsing a project that defines deprecated model(s)): [WARNING] Model {model_name}[.v{version} if versioned] has passed its deprecation date of {deprecation_date}. This model should be disabled or removed.
• UpcomingDeprecationReference (warning when referencing a model with a future deprecation date): [WARNING] While compiling '{this_model_name}': Found a reference to {ref_model_name}[.v{version} if versioned], which is slated for deprecation on '{ref_model_deprecation_date}'. [if versioned: A new version of '{ref_model_name}' is available. Try it out: {{ ref('ref_model_package', 'ref_model_name', v='{ref_model_latest_version}') }}.
• DeprecatedReference (warning when referencing a model with a past deprecation date): [WARNING] While compiling '{this_model_name}': Found a reference to {ref_model_name}[.v{version} if versioned], which was deprecated on '{ref_model_deprecation_date}'. [if versioned: A new version of '{ref_model_name}' is available. Migrate now: {{ ref('ref_model_package', 'ref_model_name', v='{ref_model_latest_version}') }}.

How does this relate to model versions?

Do I have to be using model contracts/versions/etc to set a deprecation_date?

(no!)

Setting a deprecation_date works well in conjunction with other model governance features like model versions, but can also be used independently from them.

Is there specific selection syntax I can use, to select only deprecated models?

(I don't think we've implemented this)

There is not specific node selection syntax for deprecation_date. Programmatic invocations is one way to identify deprecated models (potentially in conjunction with dbt list). e.g., dbt -q ls --output json --output-keys database schema alias deprecation_date.

How can I clean up / remove tables from the DWH associated with deprecated models?

(nothing built-in/automated, but is there a discourse/pattern we could point people toward?)

Just like it does not automatically drop relations when models are deleted, dbt does not removed tables for deprecated models.

Strategies similar to here or here can be used to drop relations that have been deprecated and are no longer in use.

How does this relate to table expiration on BQ?

(they're different things, but maybe you'd want to use them in synchrony)

dbt-bigquery can set an hours_to_expiration that translates to expiration_timestamp (expired tables in BigQuery will be deleted and their storage reclaimed).

dbt does not automatically synchronize deprecation_date and hours_to_expiration, but users may want to coordinate them in some fashion (such as setting a model to expire 48 hours after its deprecation_date).

Checklist

• Review the Content style guide and About versioning so my content adheres to these guidelines.

[netlify]

✅ Deploy Preview for docs-getdbt-com ready!

Name	Link
🔨 Latest commit	789ecf7
🔍 Latest deploy log	https://app.netlify.com/sites/docs-getdbt-com/deploys/64b6d85e7ddb3b0008acbdef
😎 Deploy Preview	https://deploy-preview-3754--docs-getdbt-com.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[jtcohen6]

Thank you @dbeatty10!! This is a great set of updates :)

Tiny comments, not blockers for merge.

I'm also realizing we implemented deprecated_date as a model property, not config, so it won't be possible for users to configure a deprecation_date for many models at once from dbt_project.yml (like access). Another thing to revisit as part of a larger refactor or consolidation of configs/properties.

[jtcohen6]

Maybe some clarification that:

• DeprecatedModel is for the producer of the deprecated model
• UpcomingDeprecationReference and DeprecatedReference are for the consumers of the deprecated model (could be in a different project!)