Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(internal) Combine profile and project configurations with defaults #953

Closed
beckjake opened this issue Aug 27, 2018 · 1 comment
Closed

Comments

@beckjake
Copy link
Contributor

Feature

When dbt starts, it should combine the profile and project configurations, along with defaults, into a single unified runtime configuration.

Feature description

Dbt effectively has two configuration files that are read in at runtime, the profile (and target) from the DBT_PROFILES directory, and the project configuration at dbt_project.yml. Various parts of the system only need one of these, but in general to do anything meaningful with dbt you need both. For each of profiles and projects, there is also another sort of "hidden" configuration, the defaults, and there are command-line arguments that impact the configuration: --target, --profile, --vars, and --threads.

Currently, defaults are set mostly at runtime after examining the profile, through things like (for example) profile.get('quoting', {}).get('identifier', True). It makes it difficult to reason about what the "default value" in any particular case is, and requires a lot of digging around in the code. Sometimes it's surprising what the defaults even are! Instead, we should have a single point in the code that accepts the project and profile configurations, has all the defaults built-in, and uses them to present an object interface (so, not just nested dicts). We're a significant portion of the way there with the current profile state, but projects need some work and they should get merged together into a single configuration at runtime, instead of passing them around as individual parts.

This will also involve minor adapter changes, in particular to how the DatabaseWrapper works and a wide assortment of methods and functions in dbt that care about either the profile or the project.

Who will this benefit?

This feature will lay the groundwork for replacing the DatabaseWrapper entirely with instances (instead of classes) of adapters that "own" their configs and ultimately make caching easier to implement. It'll also help developers adding features in the future, as we'll have a single entry point for all configuration data after parsing projects/profiles.

@beckjake
Copy link
Contributor Author

Fixed in #973

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant