Revise v1.6 migration guide (#3728)

resolves #3696
resolves #3723

Revise v1.6 migration guide for readability. Still a handful of TODOs.

### Materialized views
- Add `VersionBlock` components for adapter MVs, missed in
#3593
- We seem to be missing docs on Snowflake dynamic tables. I don't know
why #3494 was closed.

### Other updates
- Add manifest v10 (will need addition to schemas.getdbt.com:
dbt-labs/schemas.getdbt.com#24)
- Add note to upgrade guide for `state:old` and `state:unmodified`
(missed in #3596). I
haven't done a thorough comb-through of other closed v1.6 PRs to see
what needs addition to the migration guide.

website/dbt-versions.js

@@ -1,7 +1,7 @@
 exports.versions = [
   {
     version: "1.6",
-    EOLDate: "2024-07-20", // placeholder - need to confirm the final date
+    EOLDate: "2024-07-31",
     isPrerelease: true
   },
   {

website/docs/docs/collaborate/govern/project-dependencies.md

@@ -5,8 +5,8 @@ sidebar_label: "Project dependencies"
 description: "Reference public models across dbt projects"
 ---
 
-:::info
-"Project" dependencies and cross-project `ref` is currently in closed beta and are features of dbt Cloud Enterprise. To access these features, please contact your account team.
+:::caution Closed Beta - dbt Cloud Enterprise
+"Project" dependencies and cross-project `ref` are features of dbt Cloud Enterprise, currently in Closed Beta. To access these features while they are in beta, please contact your account team at dbt Labs.
 :::
 
 For a long time, dbt has supported code reuse and extension by installing other projects as [packages](/docs/build/packages). When you install another project as a package, you are pulling in its full source code, and adding it to your own. This enables you to call macros and run models defined in that other project.

website/docs/guides/migration/versions/01-upgrading-to-v1.6.md

@@ -1,17 +1,19 @@
 ---
-title: "Upgrading to v1.6 (beta)"
+title: "Upgrading to v1.6 (prerelease)"
 description: New features and changes in dbt Core v1.6
 ---
 
-:::warning Beta Functionality
+:::warning Prerelease
 
-dbt Core v1.6 is in beta, and the features and functionality on this page are subject to change.
+dbt Core v1.6 is available as a release candidate. [Final release is planned for July 31.](https://github.com/dbt-labs/dbt-core/issues/7990)
+
+Test it out, and [let us know](https://github.com/dbt-labs/dbt-core/issues/new/choose) if you run into any issues!
 
 :::
 
 ## Resources
 
-- [Changelog](https://github.com/dbt-labs/dbt-core/blob/main/CHANGELOG.md)
+- [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.6.latest/CHANGELOG.md)
 - [CLI Installation guide](/docs/core/installation)
 - [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud)
 - [Release schedule](https://github.com/dbt-labs/dbt-core/issues/7481)
@@ -22,24 +24,58 @@ dbt Labs is committed to providing backward compatibility for all versions 1.x,
 
 ### Behavior changes
 
-**Coming soon**
+- dbt Core v1.6 does not support Python 3.7, which reached End Of Life on June 23. Support Python versions are 3.8, 3.9, 3.10, and 3.11.
+- As part of the Semantic layer re-launch (in beta), the spec for `metrics` has changed significantly. Migration guide coming soon: https://github.com/dbt-labs/docs.getdbt.com/pull/3705
+- Manifest schema version is now v10, reflecting [TODO] changes
+
+### For consumers of dbt artifacts (metadata)
+
+The [manifest](/reference/artifacts/manifest-json) schema version has updated to `v10`. Specific changes:
+- Addition of `semantic_models` and changes to `metrics` attributes
+- Addition of `deprecation_date` as a model property
+- Addition of `on_configuration_change` as default node configuration (to support materialized views)
+- Small type changes to `contracts` and `constraints`
+- Manifest `metadata` includes `project_name`
+
+### For maintainers of adapter plugins
 
+For more detailed information and to ask questions, please read and comment on the GH discussion: [dbt-labs/dbt-core#7958](https://github.com/dbt-labs/dbt-core/discussions/7958).
 
 ## New and changed documentation
 
-[`dbt retry`](/reference/commands/retry) is a new command that executes the previously run command from the point of failure. This convenient command enables you to continue a failed command without rebuilding all upstream dependencies. 
+### Materialized views
 
-**Materialized view** support (for model and project configs) has been added for three data warehouses:
-    - [Bigquery](/reference/resource-configs/bigquery-configs#materialized-view)
-    - [Postgres](/reference/resource-configs/postgres-configs#materialized-view)
-    - [Redshift](/reference/resource-configs/redshift-configs#materialized-view)
+Supported on:
+- [Postgres](/reference/resource-configs/postgres-configs#materialized-view)
+- [Redshift](/reference/resource-configs/redshift-configs#materialized-view)
+- Snowflake (docs forthcoming)
 
-[**Namespacing:**](/faqs/Models/unique-model-names) Model names can be duplicated across different namespaces (packages/projects), so long as they are unique within each package/project. We strongly encourage using [two-argument `ref`](/reference/dbt-jinja-functions/ref#two-argument-variant) when referencing a model from a different package/project.
+Support for BigQuery and Databricks forthcoming.
 
-[**Project dependencies**](/docs/collaborate/govern/project-dependencies): Introduces `dependencies.yml` and dependent `projects` as a feature of dbt Cloud Enterprise. Allows enforcing model access (public vs. protected/private) across project/package boundaries. Enables cross-project `ref` of public models, without requiring the installation of upstream source code.
+### New commands for mature deployment
 
-### Quick hits
+[`dbt retry`](/reference/commands/retry) executes the previously run command from the point of failure. Rebuild just the nodes that errored or skipped in a previous run/build/test, rather than starting over from scratch.
+
+[`dbt clone`](/reference/commands/clone) leverages each data platform's functionality for creating lightweight copies of dbt models from one environment into another. Useful when quickly spinning up a new development environment, or promoting specific models from a staging environment into production.
+
+### Multi-project collaboration
 
-More consistency and flexibility around packages! Resources defined in a package will respect variable and global macro definitions within the scope of that package.
+[**Deprecation date**](/reference/resource-properties/deprecation_date): Models can declare a deprecation date that will warn model producers and downstream consumers. This enables clear migration windows for versioned models, and provides a mechanism to facilitate removal of immature or little-used models, helping to avoid project bloat.
+
+[Model names](/faqs/Models/unique-model-names) can be duplicated across different namespaces (projects/packages), so long as they are unique within each project/package. We strongly encourage using [two-argument `ref`](/reference/dbt-jinja-functions/ref#two-argument-variant) when referencing a model from a different package/project.
+
+More consistency and flexibility around packages. Resources defined in a package will respect variable and global macro definitions within the scope of that package.
 - `vars` defined in a package's `dbt_project.yml` are now available in the resolution order when compiling nodes in that package, though CLI `--vars` and the root project's `vars` will still take precedence. See ["Variable Precedence"](/docs/build/project-variables#variable-precedence) for details.
 - `generate_x_name` macros (defining custom rules for database, schema, alias naming) follow the same pattern as other "global" macros for package-scoped overrides. See [macro dispatch](/reference/dbt-jinja-functions/dispatch) for an overview of the patterns that are possible.
+
+:::caution Closed Beta - dbt Cloud Enterprise
+[**Project dependencies**](/docs/collaborate/govern/project-dependencies): Introduces `dependencies.yml` and dependent `projects` as a feature of dbt Cloud Enterprise. Allows enforcing model access (public vs. protected/private) across project/package boundaries. Enables cross-project `ref` of public models, without requiring the installation of upstream source code.
+:::
+
+### Quick hits
+
+- [`state:unmodified` and `state:old`](/reference/node-selection/methods#the-state-method) for [MECE](https://en.wikipedia.org/wiki/MECE_principle) stateful selection
+- [`invocation_args_dict`](/reference/dbt-jinja-functions/flags#invocation_args_dict) includes full `invocation_command` as string
+- [`dbt debug --connection`](/reference/commands/debug) to test just the data platform connection specified in a profile
+- [`dbt docs generate --empty-catalog`](/reference/commands/cmd-docs) to skip catalog population while generating docs
+- [`--defer-state`](/reference/node-selection/defer) enables more-granular control 

website/docs/reference/artifacts/dbt-artifacts.md

@@ -39,6 +39,7 @@ All artifacts produced by dbt include a `metadata` dictionary with these propert
 
 In the manifest, the `metadata` may also include:
 - `send_anonymous_usage_stats`: Whether this invocation sent [anonymous usage statistics](/reference/global-configs/usage-stats) while executing.
+- `project_name`: The `name` defined in the root project's `dbt_project.yml`. (Added in manifest v10 / dbt Core v1.6)
 - `project_id`: Project identifier, hashed from `project_name`, sent with anonymous usage stats if enabled.
 - `user_id`: User identifier, stored by default in `~/dbt/.user.yml`, sent with anonymous usage stats if enabled.
 

website/docs/reference/artifacts/manifest-json.md

@@ -2,53 +2,18 @@
 title: "Manifest JSON file"
 sidebar_label: "Manifest"
 ---
-<VersionBlock firstVersion="1.5">
 
-**dbt Core v1.5 produces schema**: [`v9`](https://schemas.getdbt.com/dbt/manifest/v9/index.html)
-
-</VersionBlock>
-
-<VersionBlock firstVersion="1.4" lastVersion="1.4">
-
-**dbt Core v1.4 produces schema**: [`v8`](https://schemas.getdbt.com/dbt/manifest/v8/index.html)
-
-</VersionBlock>
-
-<VersionBlock firstVersion="1.3" lastVersion="1.3">
-
-**dbt Core v1.3 produces schema**: [`v7`](https://schemas.getdbt.com/dbt/manifest/v7/index.html)
-
-</VersionBlock>
-
-<VersionBlock firstVersion="1.2" lastVersion="1.2">
-
-**dbt Core v1.2 produces schema**: [`v6`](https://schemas.getdbt.com/dbt/manifest/v6/index.html)
-
-</VersionBlock>
-
-<VersionBlock firstVersion="1.1" lastVersion="1.1">
-
-**dbt Core v1.1 produces schema**: [`v5`](https://schemas.getdbt.com/dbt/manifest/v5/index.html)
-
-</VersionBlock>
-
-<VersionBlock firstVersion="1.0" lastVersion="1.0">
-
-**dbt Core v1.0 produces schema**: [`v4`](https://schemas.getdbt.com/dbt/manifest/v4/index.html)
-
-</VersionBlock>
-
-<VersionBlock firstVersion="1.5">
-
-**Produced by:** [`build`](/reference/commands/build) [`compile`](/reference/commands/compile) [`docs generate`](/reference/commands/cmd-docs) [`list`](/reference/commands/list) [`seed`](/reference/commands/seed) [`snapshot`](/reference/commands/snapshot) [`source freshness`](/reference/commands/source) [`test`](/reference/commands/test) [`run`](/reference/commands/run) [`run-operation`](/reference/commands/run-operation)
-
-</VersionBlock>
-
-<VersionBlock lastVersion="1.4">
-
-**Produced by:** [`build`](commands/build) [`compile`](commands/compile) [`docs generate`](commands/cmd-docs) [`list`](commands/list) [`parse`](commands/parse) [`run`](commands/run) [`run-operation`](commands/run-operation) [`seed`](commands/seed) [`show`](commands/show) [`snapshot`](commands/snapshot) [`source freshness`](commands/source) [`test`](commands/test) 
-
-</VersionBlock>
+| dbt Core version | Manifest version                                              |
+|------------------|---------------------------------------------------------------|
+| v1.6             | [v10](https://schemas.getdbt.com/dbt/manifest/v10/index.html) |
+| v1.5             | [v9](https://schemas.getdbt.com/dbt/manifest/v9/index.html)   |
+| v1.4             | [v8](https://schemas.getdbt.com/dbt/manifest/v8/index.html)   |
+| v1.3             | [v7](https://schemas.getdbt.com/dbt/manifest/v7/index.html)   |
+| v1.2             | [v6](https://schemas.getdbt.com/dbt/manifest/v6/index.html)   |
+| v1.1             | [v5](https://schemas.getdbt.com/dbt/manifest/v5/index.html)   |
+| v1.0             | [v4](https://schemas.getdbt.com/dbt/manifest/v4/index.html)   |
+
+**Produced by:** Any command that parses your project. This includes all commands **except** [`deps`](/reference/commands/deps), [`clean`](/reference/commands/clean), [`debug`](/reference/commands/debug), [`init`](/reference/commands/init)
 
 This single file contains a full representation of your dbt project's resources (models, tests, macros, etc), including all node configurations and resource properties. Even if you're only running some models or tests, all resources will appear in the manifest (unless they are disabled) with most of their properties. (A few node properties, such as `compiled_sql`, only appear for executed nodes.)
 

website/docs/reference/commands/cmd-docs.md

@@ -10,22 +10,35 @@ id: "cmd-docs"
 
 The command is responsible for generating your project's documentation website by
 
-1. copying the website `index.html` file into the `target/` directory 
-2. compiling the project to `target/manifest.json`
-3. producing the `target/catalog.json` file, which contains metadata about the tables and <Term id="view">views</Term> produced by the models in your project.
+1. Copying the website `index.html` file into the `target/` directory 
+2. Compiling the resources in your project, so that their `compiled_code` will be included in [`manifest.json`](/reference/artifacts/manifest-json)
+3. Running queries against database metadata to produce the [`catalog.json`](/reference/artifacts/catalog-json) file, which contains metadata about the tables and <Term id="view">views</Term> produced by the models in your project.
 
 **Example**:
 ```
 dbt docs generate
 ```
 
-Use the `--no-compile` argument to skip re-compilation. When this flag is provided, `dbt docs generate` will only execute steps (1) and (3), as described above.
+Use the `--no-compile` argument to skip re-compilation. When this flag is provided, `dbt docs generate` will skip step (2) described above.
 
 **Example**:
 ```
 dbt docs generate --no-compile
 ```
 
+<VersionBlock firstVersion="1.6">
+
+Use the `--empty-catalog` argument to skip running the database queries to populate `catalog.json`. When this flag is provided, `dbt docs generate` will skip step (3) described above.
+
+This is not recommended for production environments, as it means that your documentation will be missing information gleaned from database metadata (the full set of columns in each table, and statistics about those tables). It can speed up `docs generate` in development, when you just want to visualize lineage and other information defined within your project.
+
+**Example**:
+```
+dbt docs generate --empty-catalog
+```
+
+</VersionBlock>
+
 ### dbt docs serve
 This command starts a webserver on port 8080 to serve your documentation locally and opens the documentation site in your default browser. The webserver is rooted in your `target/` directory. Be sure to run `dbt docs generate` before `dbt docs serve` because the  `generate` command produces a [catalog metadata artifact](/reference/artifacts/catalog-json) that the `serve` command depends upon. You will see an error message if the catalog is missing.  
 

website/docs/reference/node-selection/defer.md

@@ -9,14 +9,18 @@ title: "Defer"
 
 </Changelog>
 
-Deferral is a powerful, complex feature that enables compelling workflows. As the use cases for `--defer` evolve, dbt Labs might make enhancements to the feature, but commit to providing backward compatibility for supported versions of dbt Core.  For details, see [dbt#5095](https://github.com/dbt-labs/dbt-core/discussions/5095).
-
 Defer is a powerful feature that makes it possible to run a subset of models or tests in a [sandbox environment](/docs/environments-in-dbt) without having to first build their upstream parents. This can save time and computational resources when you want to test a small number of models in a large project.
 
 Defer requires that a manifest from a previous dbt invocation be passed to the `--state` flag or env var. Together with the `state:` selection method, these features enable "Slim CI". Read more about [state](/reference/node-selection/syntax#about-node-selection).
 
 An alternative command that accomplishes similar functionality for different use cases is `dbt clone` - see the docs for [clone](/reference/commands/clone#when-to-use-dbt-clone-instead-of-deferral) for more information.
 
+<VersionBlock firstVersion="1.6">
+
+It is possible to use separate state for `state:modified` and `--defer`, by passing paths to different manifests to each of the `--state`/`DBT_STATE` and `--defer-state`/`DBT_DEFER_STATE`. This enables more granular control in cases where you want to compare against logical state from one environment or past point in time, and defer to applied state from a different environment or point in time. If `--defer-state` is not specified, deferral will use the manifest supplied to `--state`. In most cases, you will want to use the same state for both: compare logical changes against production, and also "fail over" to the production environment for unbuilt upstream resources.
+
+</VersionBlock>
+
 ### Usage
 
 ```shell

website/docs/reference/node-selection/syntax.md

@@ -81,7 +81,7 @@ Note that when you're using `--selector`, most other flags (namely `--select` an
 <Snippet path="discourse-help-feed-header" />
 <DiscourseHelpFeed tags="node-selection"/>
 
-## About node selection 
+## Stateful selection
 
 One of the greatest underlying assumptions about dbt is that its operations should be **stateless** and **<Term id="idempotent" />**. That is, it doesn't matter how many times a model has been run before, or if it has ever been run before. It doesn't matter if you run it once or a thousand times. Given the same raw data, you can expect the same transformed result. A given run of dbt doesn't need to "know" about _any other_ run; it just needs to know about the code in the project and the objects in your database as they exist _right now_.
 
@@ -106,7 +106,7 @@ State and defer can be set by environment variables as well as CLI flags:
 
 </VersionBlock>
 
-<VersionBlock firstVersion="1.5">
+<VersionBlock firstVersion="1.5" lastVersion="1.6">
 
 - `--state` or `DBT_STATE`: file path
 - `--defer` or `DBT_DEFER`: boolean
@@ -119,6 +119,16 @@ In dbt v1.5, we deprecated the original syntax for state (`DBT_ARTIFACT_STATE_PA
 
 </VersionBlock>
 
+<VersionBlock firstVersion="1.6">
+
+- `--state` or `DBT_STATE`: file path
+- `--defer` or `DBT_DEFER`: boolean
+- `--defer-state` or `DBT_DEFER_STATE`: file path to use for deferral only (optional)
+
+If `--defer-state` is not specified, deferral will use the artifacts supplied by `--state`. This enables more granular control in cases where you want to compare against logical state from one environment or past point in time, and defer to applied state from a different environment or point in time.
+
+</VersionBlock>
+
 If both the flag and env var are provided, the flag takes precedence.
 
 #### Notes: