You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When dbt starts, it should combine the profile and project configurations, along with defaults, into a single unified runtime configuration.
Feature description
Dbt effectively has two configuration files that are read in at runtime, the profile (and target) from the DBT_PROFILES directory, and the project configuration at dbt_project.yml. Various parts of the system only need one of these, but in general to do anything meaningful with dbt you need both. For each of profiles and projects, there is also another sort of "hidden" configuration, the defaults, and there are command-line arguments that impact the configuration: --target, --profile, --vars, and --threads.
Currently, defaults are set mostly at runtime after examining the profile, through things like (for example) profile.get('quoting', {}).get('identifier', True). It makes it difficult to reason about what the "default value" in any particular case is, and requires a lot of digging around in the code. Sometimes it's surprising what the defaults even are! Instead, we should have a single point in the code that accepts the project and profile configurations, has all the defaults built-in, and uses them to present an object interface (so, not just nested dicts). We're a significant portion of the way there with the current profile state, but projects need some work and they should get merged together into a single configuration at runtime, instead of passing them around as individual parts.
This will also involve minor adapter changes, in particular to how the DatabaseWrapper works and a wide assortment of methods and functions in dbt that care about either the profile or the project.
Who will this benefit?
This feature will lay the groundwork for replacing the DatabaseWrapper entirely with instances (instead of classes) of adapters that "own" their configs and ultimately make caching easier to implement. It'll also help developers adding features in the future, as we'll have a single entry point for all configuration data after parsing projects/profiles.
The text was updated successfully, but these errors were encountered:
Feature
When dbt starts, it should combine the profile and project configurations, along with defaults, into a single unified runtime configuration.
Feature description
Dbt effectively has two configuration files that are read in at runtime, the profile (and target) from the
DBT_PROFILES
directory, and the project configuration atdbt_project.yml
. Various parts of the system only need one of these, but in general to do anything meaningful with dbt you need both. For each of profiles and projects, there is also another sort of "hidden" configuration, the defaults, and there are command-line arguments that impact the configuration:--target
,--profile
,--vars
, and--threads
.Currently, defaults are set mostly at runtime after examining the profile, through things like (for example)
profile.get('quoting', {}).get('identifier', True)
. It makes it difficult to reason about what the "default value" in any particular case is, and requires a lot of digging around in the code. Sometimes it's surprising what the defaults even are! Instead, we should have a single point in the code that accepts the project and profile configurations, has all the defaults built-in, and uses them to present an object interface (so, not just nested dicts). We're a significant portion of the way there with the current profile state, but projects need some work and they should get merged together into a single configuration at runtime, instead of passing them around as individual parts.This will also involve minor adapter changes, in particular to how the DatabaseWrapper works and a wide assortment of methods and functions in dbt that care about either the profile or the project.
Who will this benefit?
This feature will lay the groundwork for replacing the DatabaseWrapper entirely with instances (instead of classes) of adapters that "own" their configs and ultimately make caching easier to implement. It'll also help developers adding features in the future, as we'll have a single entry point for all configuration data after parsing projects/profiles.
The text was updated successfully, but these errors were encountered: