+
+
diff --git a/website/docs/docs/dbt-cloud-apis/discovery-api.md b/website/docs/docs/dbt-cloud-apis/discovery-api.md
index a9d53696890..16c9bc16ec4 100644
--- a/website/docs/docs/dbt-cloud-apis/discovery-api.md
+++ b/website/docs/docs/dbt-cloud-apis/discovery-api.md
@@ -13,7 +13,7 @@ You can access the Discovery API through [ad hoc queries](/docs/dbt-cloud-apis/d
You can query the dbt Cloud metadata:
-- At the [environment](/docs/collaborate/environments/environments-in-dbt) level for both the latest state (use the `environment` endpoint) and historical run results (use `modelByEnvironment`) of a dbt Cloud project in production.
+- At the [environment](/docs/environments-in-dbt) level for both the latest state (use the `environment` endpoint) and historical run results (use `modelByEnvironment`) of a dbt Cloud project in production.
- At the job level for results on a specific dbt Cloud job run for a given resource type, like `models` or `test`.
:::tip Public Preview
diff --git a/website/docs/docs/dbt-cloud-apis/discovery-querying.md b/website/docs/docs/dbt-cloud-apis/discovery-querying.md
index 8d602e73e5f..f75369e92a8 100644
--- a/website/docs/docs/dbt-cloud-apis/discovery-querying.md
+++ b/website/docs/docs/dbt-cloud-apis/discovery-querying.md
@@ -8,7 +8,7 @@ The Discovery API supports ad-hoc queries and integrations.. If you are new to t
Use the Discovery API to evaluate data pipeline health and project state across runs or at a moment in time. dbt Labs provide a [GraphQL explorer](https://metadata.cloud.getdbt.com/graphql) for this API, enabling you to run queries and browse the schema.
-Since GraphQL provides a description of the data in the API, the schema displayed in the GraphQL explorer accurately represents the graph and fields available to query.
+Since GraphQL describes the data in the API, the schema displayed in the GraphQL explorer accurately represents the graph and fields available to query.
@@ -16,11 +16,11 @@ Since GraphQL provides a description of the data in the API, the schema displaye
Currently, authorization of requests takes place [using a service token](/docs/dbt-cloud-apis/service-tokens). dbt Cloud admin users can generate a Metadata Only service token that is authorized to execute a specific query against the Discovery API.
-Once you've created a token, you can use it in the Authorization header of requests to the dbt Cloud Discovery API. Be sure to include the Token prefix in the Authorization header, or the request will fail with a `401 Unauthorized` error. Note that `Bearer` can be used in place of `Token` in the Authorization header. Both syntaxes are equivalent.
+Once you've created a token, you can use it in the Authorization header of requests to the dbt Cloud Discovery API. Be sure to include the Token prefix in the Authorization header, or the request will fail with a `401 Unauthorized` error. Note that `Bearer` can be used instead of `Token` in the Authorization header. Both syntaxes are equivalent.
## Access the Discovery API
-1. Create a [service account token](/docs/dbt-cloud-apis/service-tokens) to authorize requests. dbt Cloud Admin users can generate a _Metadata Only_ service token, which can be used to execute a specific query against the Discovery API for authorization of requests.
+1. Create a [service account token](/docs/dbt-cloud-apis/service-tokens) to authorize requests. dbt Cloud Admin users can generate a _Metadata Only_ service token, which can be used to execute a specific query against the Discovery API to authorize requests.
2. Find your API URL using the endpoint `https://metadata.{YOUR_ACCESS_URL}/graphql`.
@@ -57,14 +57,15 @@ metadata = response.json()['data'][ENDPOINT]
Every query will require an environment ID or job ID. You can get the ID from a dbt Cloud URL or using the Admin API.
-There are several illustrative example queries in this documentation. You can see an examples in the [use case guide](/docs/dbt-cloud-apis/discovery-use-cases-and-examples).
+There are several illustrative example queries on this page. For more examples, refer to [Use cases and examples for the Discovery API](/docs/dbt-cloud-apis/discovery-use-cases-and-examples).
## Reasonable use
-To maintain performance and stability, and prevent abuse, Discovery (GraphQL) API usage is subject to request rate and response size limits.
-- The current request rate limit is 200 requests within a minute for a given IP address. If a user exceeds this limit, they will receive an HTTP 429 response status.
+Discovery (GraphQL) API usage is subject to request rate and response size limits to maintain the performance and stability of the metadata platform and prevent abuse.
+- The current request rate limit is 200 requests for a given IP address within a minute. If you exceed this limit, you will receive an HTTP 429 response status.
- Environment-level endpoints will be subject to response size limits in the future. The depth of the graph should not exceed three levels. A user can paginate up to 500 items per query.
+- Job-level endpoints are subject to query complexity limits. Nested nodes (like parents), code (like rawCode), and catalog columns are considered as most complex. Overly complex queries should be broken up into separate queries with only necessary fields included. dbt Labs recommends using the environment endpoint instead for most use cases to get the latest descriptive and result metadata for a dbt Cloud project.
## Retention limits
You can use the Discovery API to query data from the previous three months. For example, if today was April 1st, you could query data back to January 1st.
diff --git a/website/docs/docs/dbt-cloud-apis/discovery-use-cases-and-examples.md b/website/docs/docs/dbt-cloud-apis/discovery-use-cases-and-examples.md
index 4e00f88d563..030688d9aeb 100644
--- a/website/docs/docs/dbt-cloud-apis/discovery-use-cases-and-examples.md
+++ b/website/docs/docs/dbt-cloud-apis/discovery-use-cases-and-examples.md
@@ -163,52 +163,56 @@ The API returns full identifier information (`database.schema.alias`) and the `e
```graphql
- environment(id: $environmentId) {
- applied {
- models(first: $first) {
- edges {
- node {
- uniqueId
- compiledCode
- database
- schema
- alias
- materializedType
- executionInfo {
- executeCompletedAt
- lastJobDefinitionId
- lastRunGeneratedAt
- lastRunId
- lastRunStatus
- lastRunError
- lastSuccessJobDefinitionId
- runGeneratedAt
- lastSuccessRunId
- }
- }
- }
- }
- }
- }
+ query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ models(first: $first) {
+ edges {
+ node {
+ uniqueId
+ compiledCode
+ database
+ schema
+ alias
+ materializedType
+ executionInfo {
+ executeCompletedAt
+ lastJobDefinitionId
+ lastRunGeneratedAt
+ lastRunId
+ lastRunStatus
+ lastRunError
+ lastSuccessJobDefinitionId
+ runGeneratedAt
+ lastSuccessRunId
+ }
+ }
+ }
+ }
+ }
+ }
+ }
```
### What happened with my job run?
-To review results for specific runs, you can query the metadata at the job level. This is helpful for historical analysis of deployment performance or optimizing particular jobs.
+You can query the metadata at the job level to review results for specific runs. This is helpful for historical analysis of deployment performance or optimizing particular jobs.
Example query
```graphql
-models(jobId: $jobId, runId: $runId) {
- name
- status
- tests {
- name
- status
- }
+query($jobId: Int!, $runId: Int!){
+ models(jobId: $jobId, runId: $runId) {
+ name
+ status
+ tests {
+ name
+ status
+ }
+ }
}
```
@@ -224,39 +228,41 @@ With the API, you can compare the `rawCode` between the definition and applied s
```graphql
-environment(id: $environmentId) {
- applied {
- models(first: $first, filter: {uniqueIds:"MODEL.PROJECT.MODEL_NAME"}) {
- edges {
- node {
- rawCode
- ancestors(types: [Source]){
- ...on SourceAppliedStateNode {
- freshness {
- maxLoadedAt
- }
- }
- }
- executionInfo {
- runGeneratedAt
- executeCompletedAt
- }
- materializedType
- }
- }
- }
- }
- definition {
- models(first: $first, filter: {uniqueIds:"MODEL.PROJECT.MODEL_NAME"}) {
- edges {
- node {
- rawCode
- runGeneratedAt
- materializedType
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ models(first: $first, filter: {uniqueIds:"MODEL.PROJECT.MODEL_NAME"}) {
+ edges {
+ node {
+ rawCode
+ ancestors(types: [Source]){
+ ...on SourceAppliedStateNode {
+ freshness {
+ maxLoadedAt
+ }
+ }
+ }
+ executionInfo {
+ runGeneratedAt
+ executeCompletedAt
+ }
+ materializedType
+ }
+ }
+ }
+ }
+ definition {
+ models(first: $first, filter: {uniqueIds:"MODEL.PROJECT.MODEL_NAME"}) {
+ edges {
+ node {
+ rawCode
+ runGeneratedAt
+ materializedType
+ }
+ }
+ }
+ }
+ }
}
```
@@ -278,27 +284,30 @@ By filtering on the latest status, you can get lists of models that failed to bu
```graphql
-environment(id: $environmentId) {
- applied {
- models(first: $first, filter: {lastRunStatus:error}) {
- edges {
- node {
- name
- executionInfo {
- lastRunId
- }
- }
- }
- }
- tests(first: $first, filter: {status:"fail"}) {
- edges {
- node {
- name
- executionInfo {
- lastRunId
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ models(first: $first, filter: {lastRunStatus:error}) {
+ edges {
+ node {
+ name
+ executionInfo {
+ lastRunId
+ }
+ }
+ }
+ }
+ tests(first: $first, filter: {status:"fail"}) {
+ edges {
+ node {
+ name
+ executionInfo {
+ lastRunId
+ }
+ }
+ }
+ }
+ }
}
}
```
@@ -307,7 +316,7 @@ environment(id: $environmentId) {
```graphql
-query ModelByEnvironment($environmentId: Int!, $uniqueId: String!, $lastRunCount: Int) {
+query($environmentId: Int!, $uniqueId: String!, $lastRunCount: Int) {
modelByEnvironment(environmentId: $environmentId, uniqueId: $uniqueId, lastRunCount: $lastRunCount) {
name
executeStartedAt
@@ -335,48 +344,50 @@ You can get the metadata on the latest execution for a particular model or acros
```graphql
-environment(id: $environmentId) {
- applied {
- models(first: $first,filter:{uniqueIds:"MODEL.PROJECT.MODEL_NAME"}) {
- edges {
- node {
- name
- ancestors(types:[Model, Source, Seed, Snapshot]) {
- ... on ModelAppliedStateNode {
- name
- resourceType
- materializedType
- executionInfo {
- executeCompletedAt
- }
- }
- ... on SourceAppliedStateNode {
- sourceName
- name
- resourceType
- freshness {
- maxLoadedAt
- }
- }
- ... on SnapshotAppliedStateNode {
- name
- resourceType
- executionInfo {
- executeCompletedAt
- }
- }
- ... on SeedAppliedStateNode {
- name
- resourceType
- executionInfo {
- executeCompletedAt
- }
- }
- }
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ models(first: $first,filter:{uniqueIds:"MODEL.PROJECT.MODEL_NAME"}) {
+ edges {
+ node {
+ name
+ ancestors(types:[Model, Source, Seed, Snapshot]) {
+ ... on ModelAppliedStateNode {
+ name
+ resourceType
+ materializedType
+ executionInfo {
+ executeCompletedAt
+ }
+ }
+ ... on SourceAppliedStateNode {
+ sourceName
+ name
+ resourceType
+ freshness {
+ maxLoadedAt
+ }
+ }
+ ... on SnapshotAppliedStateNode {
+ name
+ resourceType
+ executionInfo {
+ executeCompletedAt
+ }
+ }
+ ... on SeedAppliedStateNode {
+ name
+ resourceType
+ executionInfo {
+ executeCompletedAt
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
}
```
@@ -447,39 +458,41 @@ Checking [source freshness](/docs/build/sources#snapshotting-source-data-freshne
Example query
```graphql
-environment(id: $environmentId) {
- applied {
- sources(first: $first, filters:{freshnessChecked:true, database:"production"}) {
- edges {
- node {
- sourceName
- name
- identifier
- loader
- freshness {
- freshnessJobDefinitionId
- freshnessRunId
- freshnessRunGeneratedAt
- freshnessStatus
- freshnessChecked
- maxLoadedAt
- maxLoadedAtTimeAgoInS
- snapshottedAt
- criteria {
- errorAfter {
- count
- period
- }
- warnAfter {
- count
- period
- }
- }
- }
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ sources(first: $first, filters:{freshnessChecked:true, database:"production"}) {
+ edges {
+ node {
+ sourceName
+ name
+ identifier
+ loader
+ freshness {
+ freshnessJobDefinitionId
+ freshnessRunId
+ freshnessRunGeneratedAt
+ freshnessStatus
+ freshnessChecked
+ maxLoadedAt
+ maxLoadedAtTimeAgoInS
+ snapshottedAt
+ criteria {
+ errorAfter {
+ count
+ period
+ }
+ warnAfter {
+ count
+ period
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
}
```
@@ -496,27 +509,29 @@ environment(id: $environmentId) {
For the following example, the `parents` are the nodes (code) that's being tested and `executionInfo` describes the latest test results:
```graphql
-environment(id: $environmentId) {
- applied {
- tests(first: $first) {
- edges {
- node {
- name
- columnName
- parents {
- name
- resourceType
- }
- executionInfo {
- lastRunStatus
- lastRunError
- executeCompletedAt
- executionTime
- }
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ tests(first: $first) {
+ edges {
+ node {
+ name
+ columnName
+ parents {
+ name
+ resourceType
+ }
+ executionInfo {
+ lastRunStatus
+ lastRunError
+ executeCompletedAt
+ executionTime
+ }
+ }
+ }
+ }
+ }
+ }
}
```
@@ -533,34 +548,36 @@ To enforce the shape of a model's definition, you can define contracts on models
```graphql
-environment(id:123) {
- definition {
+query{
+ environment(id:123) {
+ definition {
models(first:100, filter:{access:public}) {
- edges {
- nodes {
- name
- latest_version
- contract_enforced
- constraints{
- name
- type
- expression
- columns
- }
- catalog {
- columns {
- name
- type
- constraints {
- name
- type
- expression
- }
- }
- }
- }
- }
+ edges {
+ nodes {
+ name
+ latest_version
+ contract_enforced
+ constraints{
+ name
+ type
+ expression
+ columns
+ }
+ catalog {
+ columns {
+ name
+ type
+ constraints {
+ name
+ type
+ expression
+ }
+ }
+ }
+ }
+ }
}
+ }
}
}
```
@@ -584,26 +601,28 @@ Query the Discovery API to map a table/view in the data platform to the model in
Example query
```graphql
-environment(id: $environmentId) {
- applied {
- models(first: $first, filter: {database:"analytics", schema:"prod", identifier:"customers"}) {
- edges {
- node {
- name
- description
- tags
- meta
- catalog {
- columns {
- name
- description
- type
- }
- }
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ models(first: $first, filter: {database:"analytics", schema:"prod", identifier:"customers"}) {
+ edges {
+ node {
+ name
+ description
+ tags
+ meta
+ catalog {
+ columns {
+ name
+ description
+ type
+ }
+ }
+ }
+ }
+ }
+ }
+ }
}
```
@@ -727,7 +746,7 @@ query Lineage($environmentId: Int!, $first: Int!) {
}
```
-Then, extract the node definitions and create a lineage graph. You can traverse downstream from sources and seeds (adding an edge from each node with children to its children) or iterate through each node’s parents (if it has them). Keep in mind that models and snapshots can have parents and children, whereas sources and seeds have only children and exposures and metrics only have parents.
+Then, extract the node definitions and create a lineage graph. You can traverse downstream from sources and seeds (adding an edge from each node with children to its children) or iterate through each node’s parents (if it has them). Keep in mind that models, snapshots, and metrics can have parents and children, whereas sources and seeds have only children and exposures only have parents.
2. Extract the node definitions, construct a lineage graph, and plot the graph.
@@ -863,25 +882,27 @@ Metric definitions are coming soon to the Discovery API with dbt v1.6. You’ll
Example query
```graphql
-environment(id: $environmentId) {
- definition {
- metrics(first: $first) {
- edges {
- node {
- name
- description
- type
- formula
- filter
- tags
- parents {
- name
- resourceType
- }
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ definition {
+ metrics(first: $first) {
+ edges {
+ node {
+ name
+ description
+ type
+ formula
+ filter
+ tags
+ parents {
+ name
+ resourceType
+ }
+ }
+ }
+ }
+ }
+ }
}
```
@@ -905,35 +926,37 @@ You can define and surface the groups each model is associated with. Groups cont
Example query
```graphql
-environment(id: $environmentId) {
- applied {
- model(first: $first, filter:{uniqueIds:["MODEL.PROJECT.NAME"]}) {
- edges {
- node {
- name
- description
- resourceType
- access
- group
- }
- }
- }
- }
- definition {
- groups(first: $first) {
- edges {
- node {
- name
- resourceType
- models {
- name
- }
- owner_name
- owner_email
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ model(first: $first, filter:{uniqueIds:["MODEL.PROJECT.NAME"]}) {
+ edges {
+ node {
+ name
+ description
+ resourceType
+ access
+ group
+ }
+ }
+ }
+ }
+ definition {
+ groups(first: $first) {
+ edges {
+ node {
+ name
+ resourceType
+ models {
+ name
+ }
+ owner_name
+ owner_email
+ }
+ }
+ }
+ }
+ }
}
```
@@ -947,31 +970,34 @@ You can enable users the ability to specify the level of access for a given mode
Example query
```graphql
-environment(id: $environmentId) {
- definition {
- models(first: $first) {
- edges {
- node {
- name
- access
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ definition {
+ models(first: $first) {
+ edges {
+ node {
+ name
+ access
+ }
+ }
+ }
+ }
+ }
}
---
-
-environment(id: $environmentId) {
- definition {
- models(first: $first, filters:{access:public}) {
- edges {
- node {
- name
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ definition {
+ models(first: $first, filters:{access:public}) {
+ edges {
+ node {
+ name
+ }
+ }
+ }
+ }
+ }
}
```
@@ -996,35 +1022,37 @@ For development use cases, people typically query the historical or latest defin
This example reviews an exposure and the models used in it, including when they were last executed and their test results:
```graphql
-environment(id: $environmentId) {
- applied {
- exposures(first: $first) {
- edges {
- node {
- name
- description
- owner_name
- url
- parents {
- name
- resourceType
- ... on ModelAppliedStateNode {
- executionInfo {
- executeCompletedAt
- lastRunStatus
- }
- tests {
- executionInfo {
- executeCompletedAt
- lastRunStatus
- }
- }
- }
- }
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ exposures(first: $first) {
+ edges {
+ node {
+ name
+ description
+ owner_name
+ url
+ parents {
+ name
+ resourceType
+ ... on ModelAppliedStateNode {
+ executionInfo {
+ executeCompletedAt
+ lastRunStatus
+ }
+ tests {
+ executionInfo {
+ executeCompletedAt
+ lastRunStatus
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
}
```
@@ -1039,16 +1067,18 @@ The Discovery API provides historical information about any resource in your pro
Review the differences in `compiledCode` or `columns` between runs or plot the “Approximate Size” and “Row Count” `stats` over time:
```graphql
-modelByEnvironment(environmentId: $environmentId, uniqueId: $uniqueId, lastRunCount: $lastRunCount, withCatalog: $withCatalog) {
- name
- compiledCode
- columns {
- name
+query(environmentId: Int!, uniqueId: String!, lastRunCount: Int!, withCatalog: Boolean!){
+ modelByEnvironment(environmentId: $environmentId, uniqueId: $uniqueId, lastRunCount: $lastRunCount, withCatalog: $withCatalog) {
+ name
+ compiledCode
+ columns {
+ name
+ }
+ stats {
+ label
+ value
+ }
}
- stats {
- label
- value
- }
}
```
@@ -1061,28 +1091,30 @@ dbt lineage begins with data sources. For a given source, you can look at which
Example query
```graphql
-environment(id: $environmentId) {
- applied {
- sources(first: $first, filter:{uniqueIds:["SOURCE_NAME.TABLE_NAME"]}) {
- edges {
- node {
- loader
- children {
- uniqueId
- resourceType
- ... on ModelAppliedStateNode {
- database
- schema
- alias
- children {
- uniqueId
- }
- }
- }
- }
- }
- }
- }
+query($environmentId: Int!, $first: Int!){
+ environment(id: $environmentId) {
+ applied {
+ sources(first: $first, filter:{uniqueIds:["SOURCE_NAME.TABLE_NAME"]}) {
+ edges {
+ node {
+ loader
+ children {
+ uniqueId
+ resourceType
+ ... on ModelAppliedStateNode {
+ database
+ schema
+ alias
+ children {
+ uniqueId
+ }
+ }
+ }
+ }
+ }
+ }
+ }
+ }
}
```
diff --git a/website/docs/docs/dbt-cloud-apis/migrating-to-v2.md b/website/docs/docs/dbt-cloud-apis/migrating-to-v2.md
index 1161dd159bd..3e6ac2c3577 100644
--- a/website/docs/docs/dbt-cloud-apis/migrating-to-v2.md
+++ b/website/docs/docs/dbt-cloud-apis/migrating-to-v2.md
@@ -10,7 +10,7 @@ In an attempt to provide an improved dbt Cloud Administrative API experience, th
## Key differences
-When using the [List runs](/dbt-cloud/api-v2#tag/Runs) endpoint, you can include triggered runs and sort by ID. You can use the following request in v2 to get a similar response as v4, replacing the `{accountId}` with your own and `{YOUR_ACCESS_URL}` with the appropriate [Access URL](https://docs.getdbt.com/docs/cloud/about-cloud/regions-ip-addresses) for your region and plan:
+When using the [List runs](/dbt-cloud/api-v2-legacy#tag/Runs) endpoint, you can include triggered runs and sort by ID. You can use the following request in v2 to get a similar response as v4, replacing the `{accountId}` with your own and `{YOUR_ACCESS_URL}` with the appropriate [Access URL](https://docs.getdbt.com/docs/cloud/about-cloud/regions-ip-addresses) for your region and plan:
```shell
GET https://{YOUR_ACCESS_URL}/api/v2/accounts/{accountId}/runs/?include_related=[%22trigger%22]&order_by=-id
diff --git a/website/docs/docs/dbt-cloud-apis/project-state.md b/website/docs/docs/dbt-cloud-apis/project-state.md
index da74a7f60be..a5ee71ebb1b 100644
--- a/website/docs/docs/dbt-cloud-apis/project-state.md
+++ b/website/docs/docs/dbt-cloud-apis/project-state.md
@@ -6,7 +6,7 @@ dbt Cloud provides a stateful way of deploying dbt. Artifacts are accessible pro
With the implementation of the `environment` endpoint in the Discovery API, we've introduced the idea of multiple states. The Discovery API provides a single API endpoint that returns the latest state of models, sources, and other nodes in the DAG.
-A single [deployment environment](/docs/collaborate/environments/environments-in-dbt) should represent the production state of a given dbt Cloud project.
+A single [deployment environment](/docs/environments-in-dbt) should represent the production state of a given dbt Cloud project.
There are two states that can be queried in dbt Cloud:
diff --git a/website/docs/docs/dbt-cloud-apis/schema-discovery-modelByEnv.mdx b/website/docs/docs/dbt-cloud-apis/schema-discovery-modelByEnv.mdx
index 89dc57f643e..400735bdce4 100644
--- a/website/docs/docs/dbt-cloud-apis/schema-discovery-modelByEnv.mdx
+++ b/website/docs/docs/dbt-cloud-apis/schema-discovery-modelByEnv.mdx
@@ -25,14 +25,15 @@ You can use the `environment_id` and `model_unique_id` to return the model and i
```graphql
-modelByEnvironment(environmentId: 834, uniqueId: "model.marketing.customers", lastRunCount: 20) {
- runId, # Get historical results for a particular model
- runGeneratedAt,
- executionTime, # View build time across runs
- status,
- tests { name, status, executeCompletedAt } # View test results across runs
+query{
+ modelByEnvironment(environmentId: 834, uniqueId: "model.marketing.customers", lastRunCount: 20) {
+ runId, # Get historical results for a particular model
+ runGeneratedAt,
+ executionTime, # View build time across runs
+ status,
+ tests { name, status, executeCompletedAt } # View test results across runs
}
-
+}
```
### Fields
diff --git a/website/docs/docs/dbt-cloud-apis/service-tokens.md b/website/docs/docs/dbt-cloud-apis/service-tokens.md
index b2c50f6236d..139eff8fd07 100644
--- a/website/docs/docs/dbt-cloud-apis/service-tokens.md
+++ b/website/docs/docs/dbt-cloud-apis/service-tokens.md
@@ -57,6 +57,9 @@ Account Admin service tokens have full `read + write` access to an account, so p
**Security Admin**
Security Admin service tokens have certain account-level permissions. For more on these permissions, see [Security Admin](/docs/cloud/manage-access/enterprise-permissions#security-admin).
+**Billing Admin**
+Billing Admin service tokens have certain account-level permissions. For more on these permissions, see [Billing Admin](/docs/cloud/manage-access/enterprise-permissions#billing-admin).
+
**Metadata Only**
Metadata-only service tokens authorize requests to the Discovery API.
diff --git a/website/docs/docs/dbt-cloud-environments.md b/website/docs/docs/dbt-cloud-environments.md
new file mode 100644
index 00000000000..5eccf3e7400
--- /dev/null
+++ b/website/docs/docs/dbt-cloud-environments.md
@@ -0,0 +1,47 @@
+---
+title: "dbt Cloud environments"
+id: "dbt-cloud-environments"
+description: "Learn about dbt Cloud's development environment to execute your project in the IDE"
+---
+
+An environment determines how dbt Cloud will execute your project in both the dbt Cloud IDE (for development) and scheduled jobs (for deployment).
+
+Critically, in order to execute dbt, environments define three variables:
+
+1. The version of dbt Core that will be used to run your project
+2. The warehouse connection information (including the target database/schema settings)
+3. The version of your code to execute
+
+Each dbt Cloud project can have only one [development environment](#create-a-development-environment), but there is no limit to the number of [deployment environments](/docs/deploy/deploy-environments), providing you the flexibility and customization to tailor the execution of scheduled jobs.
+
+Use environments to customize settings for different stages of your project and streamline the execution process by using software engineering principles. This page will detail the different types of environments and how to intuitively configure your development environment in dbt Cloud.
+
+
+import CloudEnvInfo from '/snippets/_cloud-environments-info.md';
+
+
+
+
+## Create a development environment
+
+To create a new dbt Cloud development environment:
+
+1. Navigate to **Deploy** -> **Environments**
+2. Click **Create Environment**.
+3. Select **Development** as the environment type.
+4. Fill in the fields under **General Settings** and **Development Credentials**.
+5. Click **Save** to create the environment.
+
+### Set developer credentials
+
+To use the IDE, each developer will need to set up [personal development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide) to your warehouse connection in their **Profile Settings**. This allows you to set separate target information and maintain individual credentials to connect to your warehouse via the dbt Cloud IDE.
+
+
+
+
+
+## Deployment environment
+
+Deployment environments in dbt Cloud are crucial for executing scheduled jobs. A dbt Cloud project can have multiple deployment environments, allowing for flexibility and customization.
+
+To learn more about dbt Cloud deployments and how to configure deployment environments, visit the [Deployment environments](/docs/deploy/deploy-environments) page. For our best practices guide, read [dbt Cloud environment best practices](https://docs.getdbt.com/guides/best-practices/environment-setup/1-env-guide-overview) for more info.
diff --git a/website/docs/docs/dbt-support.md b/website/docs/docs/dbt-support.md
index 23bc3164c7d..a6e9262200c 100644
--- a/website/docs/docs/dbt-support.md
+++ b/website/docs/docs/dbt-support.md
@@ -10,7 +10,7 @@ If you're developing in the command line (CLI) and have questions or need some h
## dbt Cloud support
We want to help you work through implementing and utilizing dbt Cloud at your organization. Have a question you can't find an answer to in [our docs](https://docs.getdbt.com/) or [the Community Forum](https://discourse.getdbt.com/)? Our Support team is here to `dbt help` you!
-Check out our guide on [getting help](/guides/legacy/getting-help) - half of the problem is often knowing where to look... and how to ask good questions!
+Check out our guide on [getting help](/community/resources/getting-help) - half of the problem is often knowing where to look... and how to ask good questions!
Types of dbt Cloud-related questions our Support team can assist you with, regardless of your dbt Cloud plan:
- **How do I...**
diff --git a/website/docs/docs/dbt-versions/core-versions.md b/website/docs/docs/dbt-versions/core-versions.md
index 9e429766d9c..328b6cf4166 100644
--- a/website/docs/docs/dbt-versions/core-versions.md
+++ b/website/docs/docs/dbt-versions/core-versions.md
@@ -4,7 +4,12 @@ id: "core"
description: "Learn about semantic versioning for dbt Core, and how long those versions are supported."
---
-dbt Core releases follow [semantic versioning](https://semver.org/) guidelines. For more on how we use semantic versions, see [How dbt Core uses semantic versioning](#how-dbt-core-uses-semantic-versioning).
+dbt Core releases follow [semantic versioning](https://semver.org/) guidelines. For more on how we use semantic versions, see [How dbt Core uses semantic versioning](#how-dbt-core-uses-semantic-versioning).
+
+dbt Labs provides different support levels for different versions, which may include new features, bug fixes, or security patches:
+
+
+
@@ -20,9 +25,12 @@ All dbt Core versions released prior to 1.0 and their version-specific documenta
## EOL version support
-All dbt Core versions with an end-of-life (EOL) support level will no longer receive bug fixes. To continue receiving bug fixes, dbt Labs recommends upgrading to a newer version.
+All dbt Core minor versions that have reached end-of-life (EOL) will have no new patch releases. This means they will no longer receive any fixes, including for known bugs that have been identified. Fixes for those bugs will instead be made in newer minor versions that are still under active support.
+
+We recommend upgrading to a newer version in [dbt Cloud](/docs/dbt-versions/upgrade-core-in-cloud) or [dbt Core](/docs/core/installation#upgrading-dbt-core) to continue receiving support.
+
+All dbt Core v1.0 and later are available in dbt Cloud until further notice. In the future, we intend to align dbt Cloud availability with dbt Core ongoing support. You will receive plenty of advance notice before any changes take place.
-All dbt Core versions v1.0 and later are available in dbt Cloud until further notice. In the future, we intend to align dbt Cloud availability with dbt Core ongoing support. You will receive plenty of advance notice before any changes take place.
## Current version support
diff --git a/website/docs/docs/dbt-versions/experimental-features.md b/website/docs/docs/dbt-versions/experimental-features.md
new file mode 100644
index 00000000000..35c64146149
--- /dev/null
+++ b/website/docs/docs/dbt-versions/experimental-features.md
@@ -0,0 +1,23 @@
+---
+title: "Preview new and experimental features in dbt Cloud"
+id: "experimental-features"
+sidebar_label: "Preview new dbt Cloud features"
+description: "Gain early access to many new dbt Labs experimental features by enabling this in your profile."
+---
+
+dbt Labs often tests experimental features before deciding to continue on the [Product lifecycle](https://docs.getdbt.com/docs/dbt-versions/product-lifecycles#dbt-cloud).
+
+You can access experimental features to preview beta features that haven’t yet been released to dbt Cloud. You can toggle on or off all experimental features in your Profile settings. Experimental features:
+
+- May not be feature-complete or fully stable as we’re actively developing them.
+- Could be discontinued at any time.
+- May require feedback from you to understand their limitations or impact. Each experimental feature collects feedback directly in dbt Cloud, which may impact dbt Labs' decisions to implement.
+- May have limited technical support and be excluded from our Support SLAs.
+- May not have public documentation available.
+
+To enable or disable experimental features:
+
+1. Navigate to **Profile settings** by clicking the gear icon in the top right.
+2. Find Experimental features at the bottom of Your Profile page.
+3. Click **Beta** to toggle the features on or off as shown in the following image.
+ 
diff --git a/website/docs/docs/dbt-versions/release-notes/06-July-2023/faster-run.md b/website/docs/docs/dbt-versions/release-notes/06-July-2023/faster-run.md
new file mode 100644
index 00000000000..0f88f1d2fa8
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/06-July-2023/faster-run.md
@@ -0,0 +1,34 @@
+---
+title: "Enhancement: Faster run starts and unlimited job concurrency"
+description: "We have enhanced the dbt Cloud Scheduler by reducing prep time for all accounts and provided unlimited job concurrency for Enterprise accounts."
+sidebar_label: "Enhancement: Faster run starts and unlimited job concurrency"
+tags: [07-2023, scheduler]
+date: 2023-07-06
+sidebar_position: 10
+---
+
+We’ve introduced significant improvements to the dbt Cloud Scheduler, offering improved performance, durability, and scalability.
+
+Read more on how you can experience faster run start execution and how enterprise users can now run as many jobs concurrently as they want to.
+
+## Faster run starts
+
+The Scheduler takes care of preparing each dbt Cloud job to run in your cloud data platform. This [prep](/docs/deploy/job-scheduler#scheduler-queue) involves readying a Kubernetes pod with the right version of dbt installed, setting environment variables, loading data platform credentials, and git provider authorization, amongst other environment-setting tasks. Only after the environment is set up, can dbt execution begin. We display this time to the user in dbt Cloud as “prep time”.
+
+
+
+For all its strengths, Kubernetes has challenges, especially with pod management impacting run execution time. We’ve rebuilt our scheduler by ensuring faster job execution with a ready pool of pods to execute customers’ jobs. This means you won't experience long prep times at the top of the hour, and we’re determined to keep runs starting near instantaneously. Don’t just take our word, review the data yourself.
+
+
+
+Jobs scheduled at the top of the hour used to take over 106 seconds to prepare because of the volume of runs the scheduler has to process. Now, even with increased runs, we have reduced prep time to 27 secs (at a maximum) — a 75% speed improvement for runs at peak traffic times!
+
+## Unlimited job concurrency for Enterprise accounts
+
+Our enhanced scheduler offers more durability and empowers users to run jobs effortlessly.
+
+This means Enterprise, multi-tenant accounts can now enjoy the advantages of unlimited job concurrency. Previously limited to a fixed number of run slots, Enterprise accounts now have the freedom to operate without constraints. Single-tenant support will be coming soon. Team plan customers will continue to have only 2 run slots.
+
+Something to note, each running job occupies a run slot for its duration, and if all slots are occupied, jobs will queue accordingly.
+
+For more feature details, refer to the [dbt Cloud pricing page](https://www.getdbt.com/pricing/).
diff --git a/website/docs/docs/dbt-versions/release-notes/08-May-2023/product-docs-may.md b/website/docs/docs/dbt-versions/release-notes/08-May-2023/product-docs-may.md
new file mode 100644
index 00000000000..762a6a723f8
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/08-May-2023/product-docs-may.md
@@ -0,0 +1,43 @@
+---
+title: "May 2023 product docs updates"
+id: "May-product-docs"
+description: "May 2023: Find out what the product docs team has been busy doing in the month of May."
+sidebar_label: "Update: Product docs changes"
+sidebar_position: 1
+tags: [May-2023, product-docs]
+date: 2023-06-01
+---
+
+Hello from the dbt Docs team: @mirnawong1, @matthewshaver, @nghi-ly, and @runleonarun! First, we’d like to thank the 13 new community contributors to docs.getdbt.com!
+
+Here's what's new to [docs.getdbt.com](http://docs.getdbt.com/) in May:
+
+## 🔎 Discoverability
+
+- We made sure everyone knows that Cloud-users don’t need a [profiles.yml file](/docs/core/connect-data-platform/profiles.yml) by adding a callout on several key pages.
+- Fleshed out the [model jinja variable page](/reference/dbt-jinja-functions/model), which originally lacked conceptual info and didn’t link to the schema page.
+- Added a new [Quickstarts landing page](/quickstarts). This new format sets up for future iterations that will include filtering! But for now, we are excited you can step through quickstarts in a focused way.
+
+## ☁ Cloud projects
+
+- We launched [dbt Cloud IDE user interface doc](/docs/cloud/dbt-cloud-ide/ide-user-interface), which provides a thorough walkthrough of the IDE UI elements and their definitions.
+- Launched a sparkling new [dbt Cloud Scheduler page](/docs/deploy/job-scheduler) ✨! We went from previously having little content around the scheduler to a subsection that breaks down the awesome scheduler features and how it works.
+- Updated the [dbt Cloud user license page](/docs/cloud/manage-access/seats-and-users#licenses) to clarify how to add or remove cloud users.
+- Shipped these Discovery API docs to coincide with the launch of the Discovery API:
+ - [About the Discovery API](/docs/dbt-cloud-apis/discovery-api)
+ - [Use cases and examples for the Discovery API](/docs/dbt-cloud-apis/discovery-use-cases-and-examples)
+ - [Query the Discovery API](/docs/dbt-cloud-apis/discovery-querying)
+
+## 🎯 Core projects
+
+- See what’s coming up [in Core v 1.6](https://github.com/dbt-labs/docs.getdbt.com/issues?q=is%3Aissue+label%3A%22dbt-core+v1.6%22)!
+- We turned the `profiles.yml` [page](/docs/core/connect-data-platform/profiles.yml) into a landing page, added more context to profile.yml page, and moved the ‘About CLI’ higher up in the `Set up dbt` section.
+
+## New 📚 Guides, ✏️ blog posts, and FAQs
+
+If you want to contribute to a blog post, we’re focusing on content
+
+- Published a blog post: [Accelerate your documentation workflow: Generate docs for whole folders at once](/blog/generating-dynamic-docs-dbt)
+- Published a blog post: [Data engineers + dbt v1.5: Evolving the craft for scale](/blog/evolving-data-engineer-craft)
+- Added an [FAQ](/faqs/Warehouse/db-connection-dbt-compile) to clarify the common question users have on *Why does dbt compile needs to connect to the database?*
+- Published a [discourse article](https://discourse.getdbt.com/t/how-to-configure-external-user-email-notifications-in-dbt-cloud/8393) about configuring job notifications for non-dbt Cloud users
diff --git a/website/docs/docs/dbt-versions/release-notes/08-May-2023/run-history-endpoint.md b/website/docs/docs/dbt-versions/release-notes/08-May-2023/run-history-endpoint.md
index ddf71872846..050fd8339a2 100644
--- a/website/docs/docs/dbt-versions/release-notes/08-May-2023/run-history-endpoint.md
+++ b/website/docs/docs/dbt-versions/release-notes/08-May-2023/run-history-endpoint.md
@@ -12,7 +12,7 @@ dbt Labs is making a change to the metadata retrieval policy for Run History in
**Beginning June 1, 2023,** developers on the dbt Cloud multi-tenant application will be able to self-serve access to their account’s run history through the dbt Cloud user interface (UI) and API for only 365 days, on a rolling basis. Older run history will be available for download by reaching out to Customer Support. We're seeking to minimize the amount of metadata we store while maximizing application performance.
-Specifically, all `GET` requests to the dbt Cloud [Runs endpoint](https://docs.getdbt.com/dbt-cloud/api-v2#tag/Runs) will return information on runs, artifacts, logs, and run steps only for the past 365 days. Additionally, the run history displayed in the dbt Cloud UI will only show runs for the past 365 days.
+Specifically, all `GET` requests to the dbt Cloud [Runs endpoint](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs) will return information on runs, artifacts, logs, and run steps only for the past 365 days. Additionally, the run history displayed in the dbt Cloud UI will only show runs for the past 365 days.
diff --git a/website/docs/docs/dbt-versions/release-notes/09-April-2023/api-endpoint-restriction.md b/website/docs/docs/dbt-versions/release-notes/09-April-2023/api-endpoint-restriction.md
index 2959cc2f1ed..8507fe3dbbb 100644
--- a/website/docs/docs/dbt-versions/release-notes/09-April-2023/api-endpoint-restriction.md
+++ b/website/docs/docs/dbt-versions/release-notes/09-April-2023/api-endpoint-restriction.md
@@ -20,4 +20,4 @@ dbt Cloud is hosted in multiple regions around the world, and each region has a
:::
-For more info, refer to our [documentation](/dbt-cloud/api-v2#tag/Runs/operation/listRunsForAccount).
+For more info, refer to our [documentation](/dbt-cloud/api-v2-legacy#tag/Runs/operation/listRunsForAccount).
diff --git a/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/apiv2-limit.md b/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/apiv2-limit.md
index fca26d7a535..85c4af48b54 100644
--- a/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/apiv2-limit.md
+++ b/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/apiv2-limit.md
@@ -11,4 +11,4 @@ To make the API more scalable and reliable, we've implemented a maximum limit of
This maximum limit applies to [multi-tenant instances](/docs/cloud/about-cloud/regions-ip-addresses) only, and _does not_ apply to single tenant instances.
-Refer to the [Pagination](https://docs.getdbt.com/dbt-cloud/api-v2#section/Pagination) section for more information on this change.
+Refer to the [Pagination](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#section/Pagination) section for more information on this change.
diff --git a/website/docs/docs/dbt-versions/release-notes/26-Sept-2022/liststeps-endpoint-deprecation.md b/website/docs/docs/dbt-versions/release-notes/26-Sept-2022/liststeps-endpoint-deprecation.md
index 377b4a12d08..545847efd90 100644
--- a/website/docs/docs/dbt-versions/release-notes/26-Sept-2022/liststeps-endpoint-deprecation.md
+++ b/website/docs/docs/dbt-versions/release-notes/26-Sept-2022/liststeps-endpoint-deprecation.md
@@ -6,9 +6,9 @@ sidebar_label: "Deprecation: List Steps API endpoint"
tags: [Sept-2022]
---
-On October 14th, 2022 dbt Labs is deprecating the [List Steps](https://docs.getdbt.com/dbt-cloud/api-v2#tag/Runs/operation/listSteps) API endpoint. From October 14th, any GET requests to this endpoint will fail. Please prepare to stop using the List Steps endpoint as soon as possible.
+On October 14th, 2022 dbt Labs is deprecating the [List Steps](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs/operation/listSteps) API endpoint. From October 14th, any GET requests to this endpoint will fail. Please prepare to stop using the List Steps endpoint as soon as possible.
-dbt Labs will continue to maintain the [Get Run](https://docs.getdbt.com/dbt-cloud/api-v2#tag/Runs/operation/getRunById) endpoint, which is a viable alternative depending on the use case.
+dbt Labs will continue to maintain the [Get Run](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs/operation/getRunById) endpoint, which is a viable alternative depending on the use case.
You can fetch run steps for an individual run with a GET request to the following URL, replacing `YOUR_ACCESS_URL` with the [appropriate Access URL](/docs/cloud/about-cloud/regions-ip-addresses) for your region and plan:
diff --git a/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md b/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md
index 17c0a21fa63..6c9ffe5d60e 100644
--- a/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md
+++ b/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md
@@ -3,8 +3,6 @@ title: "Upgrade Core version in Cloud"
id: "upgrade-core-in-cloud"
---
-## Upgrading to the latest version of dbt in Cloud
-
In dbt Cloud, both jobs and environments are configured to use a specific version of dbt Core. The version can be upgraded at any time.
### Environments
@@ -23,15 +21,17 @@ Each job in dbt Cloud can be configured to inherit parameters from the environme
The example job seen in the screenshot above belongs to the environment "Prod". It inherits the dbt version of its environment as shown by the **Inherited from ENVIRONMENT_NAME (DBT_VERSION)** selection. You may also manually override the dbt version of a specific job to be any of the current Core releases supported by Cloud by selecting another option from the dropdown.
-## Supported Versions
+## Supported versions
-We have always encouraged our customers to upgrade dbt Core versions whenever a new minor version is released. We released our first major version of dbt - `dbt 1.0` - in December 2021. Alongside this release, we updated our policy on which versions of dbt Core we will support in dbt Cloud.
+dbt Labs has always encouraged users to upgrade dbt Core versions whenever a new minor version is released. We released our first major version of dbt - `dbt 1.0` - in December 2021. Alongside this release, we updated our policy on which versions of dbt Core we will support in dbt Cloud.
+> **Starting with v1.0, all subsequent minor versions are available in dbt Cloud. Versions are actively supported, with patches and bug fixes, for 1 year after their initial release. At the end of the 1-year window, we encourage all users to upgrade to a newer version for better ongoing maintenance and support.**
+We provide different support levels for different versions, which may include new features, bug fixes, or security patches:
- > **Starting with v1.0, any subsequent minor versions will be supported in dbt Cloud for 1 year post release. At the end of the 1 year window, accounts must upgrade to a supported version of dbt or risk service disruption.**
+
-We will continue to update this table so that customers know when we plan to stop supporting different versions of Core in dbt Cloud.
+We'll continue to update the following release table so that users know when we plan to stop supporting different versions of Core in dbt Cloud.
diff --git a/website/docs/docs/deploy/dbt-cloud-job.md b/website/docs/docs/deploy/dbt-cloud-job.md
index d756693b17e..fa9eead2d3b 100644
--- a/website/docs/docs/deploy/dbt-cloud-job.md
+++ b/website/docs/docs/deploy/dbt-cloud-job.md
@@ -1,86 +1,26 @@
---
-title: "Deploy with dbt Cloud"
+title: "dbt Cloud jobs"
id: "dbt-cloud-job"
-description: "You can enable continuous integration (CI) to test every single change prior to deploying the code to production just like in a software development workflow."
+description: "Manage, setup, and configure your dbt Cloud job using elegant job commands and triggers."
hide_table_of_contents: true
tags: ["scheduler"]
---
-dbt Cloud offers the easiest way to run your dbt project in production.
Deploying with dbt Cloud lets you:
-- Keep production data fresh on a timely basis
-- Ensure CI and production pipelines are efficient
-- Identify the root cause of failures in deployment environments
-- Maintain high quality code and data in production
-- Gain visibility into the health of deployment jobs, models, and tests
+Manage, set up, and automate your dbt jobs using robust custom job settings. You can use the job scheduler to configure when and how your jobs run, helping you keep production data fresh on a timely basis.
-Learn more about the features you can use in dbt Cloud to help your team ship timely, quality production data more easily.
+This portion of our documentation will go over dbt Cloud's various job settings using:
-
+- [Job settings](/docs/deploy/job-settings) — Intuitively navigate the user interface to create new dbt jobs or edit existing ones.
+- [Job commands](/docs/deploy/job-commands) — Use job commands to configure dbt commands on a schedule.
+- [Job triggers](/docs/deploy/job-triggers) — You can configure when and how dbt should run your job, such as:
+ * Running on scheduled days or cron schedules
+ * Setting up continuous integration (CI) to run when someone opens a new pull request in your dbt repository
+ * Using the API to trigger jobs
-
+
-
+
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
diff --git a/website/docs/docs/deploy/deploy-environments.md b/website/docs/docs/deploy/deploy-environments.md
new file mode 100644
index 00000000000..da54b918436
--- /dev/null
+++ b/website/docs/docs/deploy/deploy-environments.md
@@ -0,0 +1,187 @@
+---
+title: "Deployment environments"
+id: "deploy-environments"
+description: "Learn about dbt Cloud's deployment environment to seamlessly schedule jobs or enable CI."
+---
+
+Deployment environments in dbt Cloud are crucial for deploying dbt jobs. To execute dbt, environments determine the settings used during job runs, including:
+
+- The version of dbt Core that will be used to run your project
+- The warehouse connection information (including the target database/schema settings)
+- The version of your code to execute
+
+A dbt Cloud project can have multiple deployment environments, providing you the flexibility and customization to tailor the execution of dbt jobs. You can use deployment environments to [create and schedule jobs](/docs/deploy/job-settings#create-and-schedule-jobs), [enable continuous integration](/docs/deploy/continuous-integration), or more based on your specific needs or requirements.
+
+:::tip Learn how to manage dbt Cloud environments
+To learn different approaches to managing dbt Cloud environments and recommendations for your organization's unique needs, read [dbt Cloud environment best practices](https://docs.getdbt.com/guides/best-practices/environment-setup/1-env-guide-overview).
+:::
+
+This page will go over the different types of environments and how to intuitively configure your deployment environment in dbt Cloud.
+
+import CloudEnvInfo from '/snippets/_cloud-environments-info.md';
+
+
+
+## Create a deployment environment
+
+To create a new dbt Cloud development environment, navigate to **Deploy** -> **Environments** and then click **Create Environment**. Select **Deployment** as the environment type.
+
+
+
+### Semantic Layer
+
+For Semantic Layer-eligible customers, the next section of environment settings is the Semantic Layer configurations. [The Semantic Layer setup guide](/docs/use-dbt-semantic-layer/setup-dbt-semantic-layer) has the most up-to-date setup instructions!
+
+### Deployment connection
+
+:::info Warehouse Connections
+
+ Warehouse connections are set at the Project level for dbt Cloud accounts, and each Project can have one connection (Snowflake account, Redshift host, Bigquery project, Databricks host, etc.). Some details of that connection (databases/schemas/etc.) can be overridden within this section of the dbt Cloud environment settings.
+
+:::
+
+This section determines the exact location in your warehouse dbt should target when building warehouse objects! This section will look a bit different depending on your warehouse provider.
+
+
+
+
+
+
+This section will not appear if you are using Postgres, as all values are inferred from the project's connection.
+
+
+
+
+
+This section will not appear if you are using Redshift, as all values are inferred from the project's connection.
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Role**: Snowflake role
+- **Database**: Target database
+- **Warehouse**: Snowflake warehouse
+
+
+
+
+
+This section will not appear if you are using Bigquery, as all values are inferred from the project's connection.
+
+
+
+
+
+This section will not appear if you are using Spark, as all values are inferred from the project's connection.
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Catalog** (optional): [Unity Catalog namespace](/docs/core/connect-data-platform/databricks-setup)
+
+
+
+
+
+
+### Deployment credentials
+
+This section allows you to determine the credentials that should be used when connecting to your warehouse. The authentication methods may differ depending on the warehouse and dbt Cloud tier you are on.
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Username**: Postgres username to use (most likely a service account)
+- **Password**: Postgres password for the listed user
+- **Schema**: Target schema
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Username**: Redshift username to use (most likely a service account)
+- **Password**: Redshift password for the listed user
+- **Schema**: Target schema
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Auth Method**: This determines the way dbt connects to your warehouse
+ - One of: [**Username & Password**, **Key Pair**]
+- If **Username & Password**:
+ - **Username**: username to use (most likely a service account)
+ - **Password**: password for the listed user
+- If **Key Pair**:
+ - **Username**: username to use (most likely a service account)
+ - **Private Key**: value of the Private SSH Key (optional)
+ - **Private Key Passphrase**: value of the Private SSH Key Passphrase (optional, only if required)
+- **Schema**: Target Schema for this environment
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Dataset**: Target dataset
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Token**: Access token
+- **Schema**: Target schema
+
+
+
+
+
+
+
+#### Editable fields
+
+- **Token**: Access token
+- **Schema**: Target schema
+
+
+
+
+
+
+## Related docs
+
+- [dbt Cloud environment best practices](https://docs.getdbt.com/guides/best-practices/environment-setup/1-env-guide-overview)
+- [Deploy dbt jobs](/docs/deploy/dbt-cloud-job)
+- [Deploy CI jobs](/docs/deploy/continuous-integration)
+- [Delete a job or environment in dbt Cloud](/faqs/Environments/delete-environment-job)
+
diff --git a/website/docs/docs/deploy/deployment-overview.md b/website/docs/docs/deploy/deployment-overview.md
index 8f8937620f7..dddc252211e 100644
--- a/website/docs/docs/deploy/deployment-overview.md
+++ b/website/docs/docs/deploy/deployment-overview.md
@@ -1,36 +1,129 @@
---
-title: "Deploy dbt jobs"
+title: "Deploy dbt"
id: "deployments"
-sidebar: "About job deployments"
+sidebar: "Use dbt Cloud's capabilities to seamlessly run a dbt job in production."
+hide_table_of_contents: true
+tags: ["scheduler"]
---
-Running dbt in production means setting up a system to run a _dbt job on a schedule_, rather than running dbt commands manually from the command line. Your production dbt jobs should create the tables and
views that your business intelligence tools and end users query. Before continuing, make sure you understand dbt's approach to [managing environments](/docs/collaborate/environments/environments-in-dbt).
+Use dbt Cloud's capabilities to seamlessly run a dbt job in production or staging environments. Rather than run dbt commands manually from the command line, you can leverage the [dbt Cloud's in-app scheduling](/docs/deploy/job-scheduler) to automate how and when you execute dbt.
-In addition to setting up a schedule, there are other considerations when setting up dbt to run in production:
+dbt Cloud offers the easiest and most reliable way to run your dbt project in production. Effortlessly promote high quality code from development to production and build fresh data assets that your business intelligence tools and end users query to make business decisions.
Deploying with dbt Cloud lets you:
+- Keep production data fresh on a timely basis
+- Ensure CI and production pipelines are efficient
+- Identify the root cause of failures in deployment environments
+- Maintain high-quality code and data in production
+- Gain visibility into the health of deployment jobs, models, and tests
-* The complexity involved in creating a new dbt job or editing an existing one.
-* Setting up notifications if a step within your job returns an error code (for example, a model can't be built or a test fails).
-* Accessing logs to help debug any issues.
-* Pulling the latest version of your git repo before running dbt (continuous deployment).
-* Running and testing your dbt project before merging code into master (continuous integration).
-* Allowing access for team members that need to collaborate on your dbt project.
+Before continuing, make sure you understand dbt's approach to [deployment environments](/docs/deploy/deploy-environments).
-
+
-
+
+
+
+## dbt Cloud jobs
+
+
+
+ title="Job settings"
+ body="Create and schedule jobs for the dbt Cloud scheduler to run."
+ link="/docs/deploy/job-settings"
+ icon="dbt-bit"/>
+
+
+
+
+
+
+
+## Monitor jobs and alerts
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+## Related docs
-
\ No newline at end of file
+- [Integrate with other orchestration tools](/docs/deploy/deployment-tools)
diff --git a/website/docs/docs/deploy/deployment-tools.md b/website/docs/docs/deploy/deployment-tools.md
index e7a0d3c43c3..26e9e4ea317 100644
--- a/website/docs/docs/deploy/deployment-tools.md
+++ b/website/docs/docs/deploy/deployment-tools.md
@@ -1,13 +1,12 @@
---
-title: "Deploy with other tools"
+title: "Integrate with other orchestration tools"
id: "deployment-tools"
-sidebar: "Deploy with other tools"
+sidebar_label: "Integrate with other tools"
---
-Discover additional ways to schedule and run your dbt jobs with the help of robust tools such as Airflow, Prefect, Dagster, automation server, Cron, and Azure Data Factory (ADF), alongside [dbt Cloud](/docs/deploy/dbt-cloud-job).
-
-Use these tools to automate your data workflows, trigger dbt jobs (including those hosted on dbt Cloud), and enjoy a hassle-free experience, saving time and increasing efficiency.
+Alongside [dbt Cloud](/docs/deploy/dbt-cloud-job), discover other ways to schedule and run your dbt jobs with the help of tools such as Airflow, Prefect, Dagster, automation server, Cron, and Azure Data Factory (ADF),
+Build and install these tools to automate your data workflows, trigger dbt jobs (including those hosted on dbt Cloud), and enjoy a hassle-free experience, saving time and increasing efficiency.
## Airflow
diff --git a/website/docs/docs/deploy/job-commands.md b/website/docs/docs/deploy/job-commands.md
index 04d63ecac62..acdc3a00228 100644
--- a/website/docs/docs/deploy/job-commands.md
+++ b/website/docs/docs/deploy/job-commands.md
@@ -19,7 +19,7 @@ Job commands are specific tasks executed by the job, and you can configure them
During a job run, the commands are "chained" together and executed as run steps. When you add a dbt command in the **Commands** section, you can expect different outcomes compared to the checkbox option.
-
+
### Built-in commands
@@ -29,7 +29,7 @@ Every job invocation automatically includes the [`dbt deps`](/reference/commands
**Job outcome** — During a job run, the built-in commands are "chained" together. This means if one of the run steps in the chain fails, then the next commands aren't executed, and the entire job fails with an "Error" job status.
-
+
### Checkbox commands
@@ -56,7 +56,7 @@ Use [selectors](/reference/node-selection/syntax) as a powerful way to select an
In the following example image, the first four run steps are successful. However, if the fifth run step (`dbt run --select state:modified+ --full-refresh --fail-fast`) fails, then the next run steps aren't executed, and the entire job fails. The failed job returns a non-zero [exit code](/reference/exit-codes) and "Error" job status:
-
+
## Job command failures
diff --git a/website/docs/docs/deploy/job-settings.md b/website/docs/docs/deploy/job-settings.md
index e8808397b6a..3b53880bddf 100644
--- a/website/docs/docs/deploy/job-settings.md
+++ b/website/docs/docs/deploy/job-settings.md
@@ -21,7 +21,7 @@ You can create a job and configure it to run on [scheduled days and times](/docs
- You must have a dbt project connected to a [data platform](/docs/cloud/connect-data-platform/about-connections).
- You must [create and schedule a dbt Cloud job](#create-and-schedule-jobs).
- You must have [access permission](/docs/cloud/manage-access/about-user-access) to view, create, modify, or run jobs.
-- You must set up a [deployment environment](/docs/collaborate/environments/dbt-cloud-environments).
+- You must set up a [deployment environment](/docs/deploy/deploy-environments).
## Create and schedule jobs {#create-and-schedule-jobs}
@@ -55,4 +55,4 @@ You can create a job and configure it to run on [scheduled days and times](/docs
-7. Select **Save**, then click **Run Now** to run your job. Click the run and watch its progress under **Run history**.
\ No newline at end of file
+7. Select **Save**, then click **Run Now** to run your job. Click the run and watch its progress under **Run history**.
diff --git a/website/docs/docs/deploy/monitor-jobs.md b/website/docs/docs/deploy/monitor-jobs.md
new file mode 100644
index 00000000000..c4c5fcb73a5
--- /dev/null
+++ b/website/docs/docs/deploy/monitor-jobs.md
@@ -0,0 +1,28 @@
+---
+title: "Monitor jobs and alerts"
+id: "monitor-jobs"
+description: "Monitor your dbt Cloud job and set up alerts to ensure seamless orchestration and optimize your data transformations"
+tags: ["scheduler"]
+---
+
+Monitor your dbt Cloud jobs to help identify improvement and set up alerts to proactively alert the right people or team.
+
+This portion of our documentation will go over dbt Cloud's various capabilities that help you monitor your jobs and set up alerts to ensure seamless orchestration, including:
+
+- [Run visibility](/docs/deploy/run-visibility) — View your run history to help identify where improvements can be made to scheduled jobs.
+- [Job notifications](/docs/deploy/job-notifications) — Receive email or slack notifications when a job run succeeds, fails, or is canceled.
+- [Webhooks](/docs/deploy/webhooks) — Use webhooks to send events about your dbt jobs' statuses to other systems.
+- [Leverage artifacts](/docs/deploy/artifacts) — dbt Cloud generates and saves artifacts for your project, which it uses to power features like creating docs for your project and reporting freshness of your sources.
+- [Source freshness](/docs/deploy/source-freshness) — Monitor data governance by enabling snapshots to capture the freshness of your data sources.
+- [Dashboard status tiles](/docs/deploy/dashboard-status-tiles) — Set up and add status tiles to view data freshness and quality checks
+
+
+
+
+
+
+
+
+
+
+
diff --git a/website/docs/docs/deploy/slim-ci-jobs.md b/website/docs/docs/deploy/slim-ci-jobs.md
index 8f8f359c1cf..ee00d0b15ba 100644
--- a/website/docs/docs/deploy/slim-ci-jobs.md
+++ b/website/docs/docs/deploy/slim-ci-jobs.md
@@ -15,7 +15,7 @@ You can set up Slim [continuous integration](/docs/deploy/continuous-integration
## Set up Slim CI jobs
-dbt Labs recommends that you create your Slim CI job in a dedicated dbt Cloud [deployment environment](/docs/collaborate/environments/dbt-cloud-environments#create-a-deployment-environment) that's connected to a staging database. Having a separate environment dedicated for CI will provide better isolation between your temporary CI schemas builds and your production data builds. Additionally, sometimes teams need their Slim CI jobs to be triggered when a PR is made to a branch other than main. If your team maintains a staging branch in your release process, having a separate environment will allow you to set a [custom branch](/faqs/environments/custom-branch-settings), and accordingly the CI job in that dedicated environment will be triggered only when PRs are made to the specified, custom branch.
+dbt Labs recommends that you create your Slim CI job in a dedicated dbt Cloud [deployment environment](/docs/deploy/deploy-environments#create-a-deployment-environment) that's connected to a staging database. Having a separate environment dedicated for CI will provide better isolation between your temporary CI schemas builds and your production data builds. Additionally, sometimes teams need their Slim CI jobs to be triggered when a PR is made to a branch other than main. If your team maintains a staging branch in your release process, having a separate environment will allow you to set a [custom branch](/faqs/environments/custom-branch-settings), and accordingly the CI job in that dedicated environment will be triggered only when PRs are made to the specified, custom branch.
1. On your deployment environment page, click **Create One** to create a new CI job.
2. In the **Execution Settings** section:
@@ -137,4 +137,4 @@ If your temporary pull request schemas aren't dropping after a merge or close of
• ❌ Database has changed from the default connection (like
dev).