Skip to content

Commit

Permalink
Remove and correct outdated docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
jordivandooren committed Sep 17, 2024
1 parent 77aeb3e commit 1fa97e1
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 20 deletions.
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The main subdirectories of core/dbt:
- [`deps`](core/dbt/deps/README.md): Package installation and dependency resolution
- [`events`](core/dbt/events/README.md): Logging events
- [`graph`](core/dbt/graph/README.md): Produce a `networkx` DAG of project resources, and selecting those resources given user-supplied criteria
- [`include`](core/dbt/include/README.md): The dbt "global project," which defines default implementations of Jinja2 macros
- [`include`](core/dbt/include/README.md): Set up the starter project scaffold.
- [`parser`](core/dbt/parser/README.md): Read project files, validate, construct python objects
- [`task`](core/dbt/task/README.md): Set forth the actions that dbt can perform when invoked

Expand Down
2 changes: 1 addition & 1 deletion core/dbt/clients/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

#### How are materializations defined

Model materializations are kept in `core/dbt/include/global_project/macros/materializations/models/`. Materializations are defined using syntax that isn't part of the Jinja standard library. These tags are referenced internally, and materializations can be overridden in user projects when users have specific needs.
Model materializations are defined by adapters. See the [dbt-adapters](https://github.com/dbt-labs/dbt-adapters) project for (base implementations)[https://github.com/dbt-labs/dbt-adapters/tree/main/dbt/include/global_project/macros/materializations/models]. Materializations are defined using syntax that isn't part of the Jinja standard library. These tags are referenced internally, and materializations can be overridden in user projects when users have specific needs.

```
-- Pseudocode for arguments
Expand Down
22 changes: 4 additions & 18 deletions core/dbt/include/README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,19 @@
# Include Module

The Include module is responsible for housing default macro definitions, starter project scaffold, and the html file used to generate the docs page.
The Include module is responsible for the starter project scaffold.

# Directories

## `global_project`
Defines the default implementations of jinja2 macros for `dbt-core` which can be overwritten in each adapter repo to work more in line with those adapter plugins. To view adapter specific jinja2 changes please check the relevant adapter repo [`adapter.sql` ](https://github.com/dbt-labs/dbt-bigquery/blob/main/dbt/include/bigquery/macros/adapters.sql) file in the `include` directory or in the [`impl.py`](https://github.com/dbt-labs/dbt-bigquery/blob/main/dbt/adapters/bigquery/impl.py) file for some ex. BigQuery (truncate_relation).

These directories contain the macros which wrap model code in DDL/DML. That code, in turn, “materializes” as a fully fledged database object in a warehouse pointed to by the current `target`. Here’re the major steps of this process:
1. the SQL `select` query is compiled. In its current state, it could be run directly through a query editor to retrieve a dataset that will match 1:1 the database object materialized by `dbt`.
2. `dbt` embeds this SQL query into a materialization macro.
3. `dbt` executes this materialization macro which unravels a table into a DML/DDL statement that creates/truncate-and-replenishes/replaces a relation (e.g. view, table) in the `target`.

Note: `dbt compile` will not progress past step 1. It places the code produced in `target/compiled`.
## `starter_project`
Produces the default project after running the `dbt init` command for the CLI. `dbt-cloud` initializes the project by using [dbt-starter-project](https://github.com/dbt-labs/dbt-starter-project).

### adapter.dispatch
# adapter.dispatch
Packages (e.g. `include` directories of adapters, any [hub](https://hub.getdbt.com/)-hosted package) can be interpreted as namespaces of functions a.k.a macros. In `dbt`'s macrospace, we take advantage of the multiple dispatch programming language concept. In short, multiple dispatch supports dynamic searching for a function across several namespaces—usually in a manually specified manner/order.

Adapters can have their own implementation of the same macro X. For example, a macro executed by `dbt-redshift` may need a specific implementation different from `dbt-snowflake`'s macro. We use multiple dispatch via `adapter.dispatch`, a Jinja function, which enables polymorphic macro invocations. The chosen implementation is selected according to what the `adapter` object is set to at runtime (it could be for redshift, postgres, and so on).

For more on this object, check out the dbt docs [here](https://docs.getdbt.com/reference/dbt-jinja-functions/adapter).

## `starter_project`
Produces the default project after running the `dbt init` command for the CLI. `dbt-cloud` initializes the project by using [dbt-starter-project](https://github.com/dbt-labs/dbt-starter-project).


# Files
- `index.html` a file generated from [dbt-docs](https://github.com/dbt-labs/dbt-docs) prior to new releases and replaced in the `dbt-core` directory. It is used to generate the docs page after using the `generate docs` command in dbt.

# dbt and database adapter python package interop

Let’s say we have a fictional python app named `dbt-core` with this structure
Expand Down

0 comments on commit 1fa97e1

Please sign in to comment.