Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support quoted parameter list for MultiOption cli options #8665

Merged
merged 16 commits into from
Sep 22, 2023

Conversation

emmyoop
Copy link
Member

@emmyoop emmyoop commented Sep 18, 2023

resolves #8598
resolves #8575

Problem

  1. When the list of parameters sent into are quoted, it cannot be processed and produces an error:
dbt -q ls --resource-type "model semantic_model"
Usage: dbt ls [OPTIONS]
Try 'dbt ls -h' for help.

Error: Invalid value for '--resource-types' / '--resource-type': 'model semantic_model' is not one of 'metric', 'semantic_model', 'source', 'analysis', 'model', 'test', 'exposure', 'snapshot', 'seed', 'default', 'all'.
  1. resource-type does not work on retry.
dbt build -s some_model --resource-type all
dbt retry

The above fails with

click.exceptions.BadParameter: "['all']" is not one of 'metric', 'source', 'analysis', 'model', 'test', 'exposure', 'snapshot', 'seed', 'default', 'all'.

Solution

  1. Support quoted parameter lists while continuing to support unquoted lists, separated args and single parameters.
  2. Stop treating the parameters in retry as a quoted list

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@cla-bot cla-bot bot added the cla:yes label Sep 18, 2023
@codecov
Copy link

codecov bot commented Sep 18, 2023

Codecov Report

Patch coverage: 93.33% and project coverage change: -0.01% ⚠️

Comparison is base (d597b80) 86.60% compared to head (e1fb937) 86.60%.
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8665      +/-   ##
==========================================
- Coverage   86.60%   86.60%   -0.01%     
==========================================
  Files         175      176       +1     
  Lines       25638    25674      +36     
==========================================
+ Hits        22205    22235      +30     
- Misses       3433     3439       +6     
Flag Coverage Δ
integration 83.36% <93.33%> (-0.11%) ⬇️
unit 65.15% <80.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
core/dbt/cli/options.py 96.42% <92.30%> (+0.06%) ⬆️
core/dbt/cli/flags.py 92.18% <100.00%> (+0.08%) ⬆️

... and 17 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@emmyoop emmyoop marked this pull request as ready for review September 19, 2023 18:04
@emmyoop emmyoop requested a review from a team as a code owner September 19, 2023 18:04
@emmyoop emmyoop requested a review from martynydbt September 19, 2023 18:04
tests/unit/test_cli_flags.py Outdated Show resolved Hide resolved
@emmyoop emmyoop requested review from gshank and aranke and removed request for martynydbt September 19, 2023 18:04
@dbeatty10
Copy link
Contributor

@emmyoop does this PR solve this issue as well?

This is the problem described in that issue:

model/some_model.sql

select null as id

model/_some_model.yml

version: 2

models:
  - name: some_model
    columns:
      - name: id
        tests:
          - not_null
dbt build -s some_model --resource-type all
dbt retry

Copy link
Member

@aranke aranke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, couple nits

core/dbt/cli/options.py Outdated Show resolved Hide resolved
tests/functional/cli/test_multioption.py Outdated Show resolved Hide resolved
@emmyoop emmyoop merged commit 417fc2a into main Sep 22, 2023
50 checks passed
@emmyoop emmyoop deleted the er/8598-multioption branch September 22, 2023 19:34
@github-actions
Copy link
Contributor

The backport to 1.5.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.5.latest 1.5.latest
# Navigate to the new working tree
cd .worktrees/backport-1.5.latest
# Create a new branch
git switch --create backport-8665-to-1.5.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 417fc2a73587df8532a241c083f34fb832f73426
# Push it to GitHub
git push --set-upstream origin backport-8665-to-1.5.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.5.latest

Then, create a pull request where the base branch is 1.5.latest and the compare/head branch is backport-8665-to-1.5.latest.

github-actions bot pushed a commit that referenced this pull request Sep 22, 2023
* allow multioption to be quoted

* changelog

* fix test

* remove list format

* fix tests

* fix list object

* review arg change

* fix quotes

* Update .changes/unreleased/Features-20230918-150855.yaml

* add types

* convert list to set in test

* make mypy happy

* mroe mypy happiness

* more mypy happiness

* last mypy change

* add node to test

(cherry picked from commit 417fc2a)
@github-actions
Copy link
Contributor

The backport to 1.5.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.5.latest 1.5.latest
# Navigate to the new working tree
cd .worktrees/backport-1.5.latest
# Create a new branch
git switch --create backport-8665-to-1.5.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 417fc2a73587df8532a241c083f34fb832f73426
# Push it to GitHub
git push --set-upstream origin backport-8665-to-1.5.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.5.latest

Then, create a pull request where the base branch is 1.5.latest and the compare/head branch is backport-8665-to-1.5.latest.

emmyoop added a commit that referenced this pull request Sep 22, 2023
* allow multioption to be quoted

* changelog

* fix test

* remove list format

* fix tests

* fix list object

* review arg change

* fix quotes

* Update .changes/unreleased/Features-20230918-150855.yaml

* add types

* convert list to set in test

* make mypy happy

* mroe mypy happiness

* more mypy happiness

* last mypy change

* add node to test
emmyoop added a commit that referenced this pull request Sep 25, 2023
* allow multioption to be quoted

* changelog

* fix test

* remove list format

* fix tests

* fix list object

* review arg change

* fix quotes

* Update .changes/unreleased/Features-20230918-150855.yaml

* add types

* convert list to set in test

* make mypy happy

* mroe mypy happiness

* more mypy happiness

* last mypy change

* add node to test
emmyoop added a commit that referenced this pull request Sep 26, 2023
)

* allow multioption to be quoted

* changelog

* fix test

* remove list format

* fix tests

* fix list object

* review arg change

* fix quotes

* Update .changes/unreleased/Features-20230918-150855.yaml

* add types

* convert list to set in test

* make mypy happy

* mroe mypy happiness

* more mypy happiness

* last mypy change

* add node to test

(cherry picked from commit 417fc2a)

Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
emmyoop added a commit that referenced this pull request Sep 26, 2023
)

* allow multioption to be quoted

* changelog

* fix test

* remove list format

* fix tests

* fix list object

* review arg change

* fix quotes

* Update .changes/unreleased/Features-20230918-150855.yaml

* add types

* convert list to set in test

* make mypy happy

* mroe mypy happiness

* more mypy happiness

* last mypy change

* add node to test
QMalcolm pushed a commit that referenced this pull request Oct 9, 2023
* allow multioption to be quoted

* changelog

* fix test

* remove list format

* fix tests

* fix list object

* review arg change

* fix quotes

* Update .changes/unreleased/Features-20230918-150855.yaml

* add types

* convert list to set in test

* make mypy happy

* mroe mypy happiness

* more mypy happiness

* last mypy change

* add node to test
@aranke aranke mentioned this pull request Jul 12, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants