Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dbt init Interactive profile creation #3625

Merged
merged 22 commits into from
Oct 20, 2021
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- Turns on the static parser by default and adds the flag `--no-static-parser` to disable it. ([#3377](https://github.com/dbt-labs/dbt/issues/3377), [#3939](https://github.com/dbt-labs/dbt/pull/3939))
- Generic test FQNs have changed to include the relative path, resource, and column (if applicable) where they are defined. This makes it easier to configure them from the `tests` block in `dbt_project.yml` ([#3259](https://github.com/dbt-labs/dbt/pull/3259), [#3880](https://github.com/dbt-labs/dbt/pull/3880)
- Turn on partial parsing by default ([#3867](https://github.com/dbt-labs/dbt/issues/3867), [#3989](https://github.com/dbt-labs/dbt/issues/3989))
- `dbt init` is now interactive, generating profiles.yml when run inside existing project ([#3625](https://github.com/dbt-labs/dbt/pull/3625))

### Fixes
- Add generic tests defined on sources to the manifest once, not twice ([#3347](https://github.com/dbt-labs/dbt/issues/3347), [#3880](https://github.com/dbt-labs/dbt/pull/3880))
Expand All @@ -29,6 +30,7 @@ Contributors:

- [@dave-connors-3](https://github.com/dave-connors-3) ([#3920](https://github.com/dbt-labs/dbt/issues/3920))
- [@kadero](https://github.com/kadero) ([#3952](https://github.com/dbt-labs/dbt/issues/3952))
- [@NiallRees](https://github.com/NiallRees) ([#3625](https://github.com/dbt-labs/dbt/pull/3625))

## dbt 0.21.0 (Release TBD)

Expand Down
7 changes: 3 additions & 4 deletions core/dbt/clients/yaml_helper.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
import dbt.exceptions
from typing import Any, Dict, Optional
import yaml
import yaml.scanner
import oyaml as yaml
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced yaml with oyaml in order to retain ordering when prompting for user input. Rather than keep both I just replaced every reference with oyaml. It may well be preferred that we leave all other imports as-is, let me know.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kwigley do you have any thoughts here?

Copy link
Contributor

@jtcohen6 jtcohen6 Oct 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the switch from yaml to oyaml has broken one highly specific integration test, which checks the sorting behavior of the toyaml Jinja context method. The sort_keys argument is not being respected by oyaml.safe_dump. Glad we have a test for it!

https://github.com/dbt-labs/dbt/blob/3789acc5a7b3f71b4e333ac6e235c62ee0c957f5/test/integration/013_context_var_tests/tests/to_yaml.sql#L5

https://github.com/dbt-labs/dbt/blob/3789acc5a7b3f71b4e333ac6e235c62ee0c957f5/core/dbt/context/base.py#L416

I think my preference would probably be to avoid switching from yaml to oyaml wherever possible. I'm also wondering if there's another way we can preserve prompt order for target_options.yml, even if it requires an extra attribute

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would seem my reasoning for using oyaml was misplaced - I've removed it and apart from some changes to the order of dumped yaml in profiles.yml the rest of the behaviour is identical. So the order of the questions to the user is still the order of the keys in the target_options.yml. Hooray!


# the C version is faster, but it doesn't always exist
try:
from yaml import (
from oyaml import (
CLoader as Loader,
CSafeLoader as SafeLoader,
CDumper as Dumper
)
except ImportError:
from yaml import ( # type: ignore # noqa: F401
from oyaml import ( # type: ignore # noqa: F401
Loader, SafeLoader, Dumper
)

Expand Down
8 changes: 4 additions & 4 deletions core/dbt/include/starter_project/dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'my_new_project'
name: '{project_name}'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'default'
profile: '{profile_name}'

# These configurations specify where dbt should look for different types of files.
# The `source-paths` config, for example, states that models in this project can be
Expand All @@ -30,9 +30,9 @@ clean-targets: # directories to be removed by `dbt clean`

# In this example config, we tell dbt to build all models in the example/ directory
# as tables. These settings can be overridden in the individual model files
# using the `{{ config(...) }}` macro.
# using the `{{{{ config(...) }}}}` macro.
models:
my_new_project:
{project_name}:
# Config indicated by + and applies to all files under models/example/
example:
+materialized: view
15 changes: 0 additions & 15 deletions core/dbt/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,6 @@ def run_from_args(parsed):

with track_run(task):
results = task.run()

return task, results


Expand Down Expand Up @@ -348,20 +347,6 @@ def _build_init_subparser(subparsers, base_subparser):
Initialize a new DBT project.
'''
)
sub.add_argument(
'project_name',
type=str,
help='''
Name of the new project
''',
)
sub.add_argument(
'--adapter',
type=str,
help='''
Write sample profiles.yml for which adapter
''',
)
sub.set_defaults(cls=init_task.InitTask, which='init', rpc_method=None)
return sub

Expand Down
Loading