Only output failing models and violated rules per default in HumanReadableFormatter #77

thomend · 2024-10-02T06:17:40Z

Fixes #71

It changes the default behavior of the HumanReadableFormatter:
When running dbt-score lint it will now only show failing models and/or rules which were violated.
To have all output printed (exactly the way it was before this PR) the user has now to pass the --show_all flag to dbt-score lint.
It also considers the fail_any_model_under option.

Furthermore, I had to adjust some of tests since the default behavior of the HumanReadableFormatter is now changed - let me know if I would need to expand this.

The docs also reflect this change by slightly tweaking the first example in index.md

In this current form, both rule violations and failing models are filtered through the same argument. This is based on @matthieucan. 's comment on #71 (comment)

…ch fail

src/dbt_score/cli.py

jochemvandooren

Thank you for your contribution! 🙌 Looks in good shape already, just some minor comments!

docs/index.md

src/dbt_score/cli.py

jochemvandooren · 2024-10-04T14:43:28Z

src/dbt_score/formatters/human_readable_formatter.py

+            score.value < self._config.fail_any_model_under
+            or any(isinstance(result, RuleViolation) for result in results.values())
+            or self._config.show_all


I think this is not 100% correct 🤔 If I understand correctly we want to print the following output if:

score.value < self._config.fail_any_model_under. Then, only show the failing rules. Now it will also show the failing rules of models that did not fail

That's a good point! My line of thought here was, that in case that a project as whole fails and only very few model scores are too low , I probably would be interested as a user to also see failing rules of all models: Imagine you have 100 models, only ~5 fail but ~20 have lowish scores while 75 are perfect. In that case it could be of interest to also see the failing rules of all models.

I also referred to this in the issue discussion: #71 (comment). But I guess in that case one could just increase fail_any_model_under. Probably this is a bit too implicit and I could remove this (e.g only test for score.value < self._config.fail_any_model_under).

Just let me know which way you prefer and then I adjust it accordingly.

Ok I got it! 👍 I think for the scenario you describe it is indeed useful to be able to show all the failing rules and we can definitely leave that as the default. I do think that we should also have the option to show only failing models, with their failing rules.

Maybe we should have two flags: --show-all-rules and --show-all-models so the user is able to further specify the output. @matthieucan curious to hear your opinion as well!

So then the user is able to:

show failing models, with all rules

show failing models, with failing rules

show all models, with all rules

show all models, with failing rules

Good to align on those expectations indeed.
Maybe the --show parameter could take an argument, e.g.

--show all - show all models, all rules

--show failing-models - show failing rules of failing models

--show failing-rules - show failing rules of all models (the default?)

I'm not sure if the first scenario mentioned (show failing models, with all rules) is useful, considering the option to use --select in combination. For example --select my_model --show all might be more actionable. But let me know what you think :)

You are right that option 1 (failing models with all rules) is probably not very useful. I think the direction of --show something would be a nice one. It's indeed simpler than providing two flags. And agreed that --show failing-rules should be the default!

I think it's simpler to remember for users if the same term is used for the CLI options, as it's an abstraction over the code base

Hey @thomend! Will you be able to continue on this PR sooner or later? We can assist if needed

Hi @jochemvandooren sorry not picking it up earlier! I initially planned to work on it much sooner again. I am picking it up today evening again and will come back to you asap then - hope that works for you.

I just added the different options as discussed above. I also took care of the merge conflicts. Tests run through + precommit hooks as well.
The show parameter takes the following options now:

--show all
--show failing-models
--show failig-rules (default)

No problem @thomend ! Was afraid you had forgotten 😢 I will review tomorrow!

jochemvandooren · 2024-10-04T14:48:22Z

tests/formatters/test_human_readable_formatter.py

@@ -44,7 +75,7 @@ def test_human_readable_formatter_project(capsys, default_config, manifest_loade
    )
    formatter.project_evaluated(Score(10.0, "🥇"))
    stdout = capsys.readouterr().out
-    assert stdout == "Project score: \x1B[1m10.0\x1B[0m 🥇\n"
+    assert stdout == "Project score: \x1b[1m10.0\x1b[0m 🥇\n"


Why did all the B's turn into b? 😁

Hexadecimal is not case sensitive, but indeed strange to see those changed 🤔

Sorry for that! I am using the ruff vscode extension and the autoformat on save did that:
Hex codes and Unicode sequences. Although I don't know why the linter during pre-commit would not pick it up to revert it? The ruff version of the vscode extension is 0.6.6, I believe (which is newer than the one running in the pre-commit hook.
Let me know if you want it reverted.

Co-authored-by: Jochem van Dooren <jochemmathijs.vandooren@teampicnic.com>

src/dbt_score/cli.py

jochemvandooren

Looks good, some minor comment on the output and some naming! 🚀

src/dbt_score/cli.py

src/dbt_score/formatters/human_readable_formatter.py

jochemvandooren · 2024-12-10T08:24:11Z

tests/formatters/test_human_readable_formatter.py

+    assert stdout == dedent(expected)
+
+
+def test_human_readable_formatter_model_show_all(


Can we have a test for failing-models as well?

I added a parametrized test to test for the different options in the show parameter - hope this works :)

src/dbt_score/cli.py

jochemvandooren

Code looks good! 🙌

Happy to approve once you a test for the --show failing-items configuration, also please add a CHANGELOG.md entry! 🚀

docs/index.md

Co-authored-by: Jochem van Dooren <jochemmathijs.vandooren@teampicnic.com>

thomend · 2024-12-12T07:16:15Z

Thanks for the review ! Hope the PR is now as you imagined.
Just let me know if there is anything else outstanding on the PR :)

jochemvandooren

Awesome, thanks a lot! 🚀 Just need to add a CHANGELOG.md entry, but approving already!

thomend · 2024-12-12T07:19:42Z

Sorry! The changelog commit was not pushed! (pushed to remote right now)

matthieucan

Thanks for your work @thomend! 😍
Only formatting issues to fix with prettier, and it can be merged

CHANGELOG.md

thomend · 2024-12-13T13:10:21Z

Thanks for your work @thomend! 😍 Only formatting issues to fix with prettier, and it can be merged

Thank you as well! Yeah I saw it (it's odd though since I thought it runs also during pre-commit hooks). Will check it out.

Co-authored-by: Matthieu Caneill <matthieucan@users.noreply.github.com>

jochemvandooren · 2024-12-18T12:44:43Z

Awesome @thomend ! Thanks a lot for the contribution, major feature 🤩 We will try to release it soon in 0.9.0

jochemvandooren · 2024-12-19T10:03:22Z

Released in 0.9.0! @thomend

thomend · 2024-12-19T20:36:14Z

Awesome @thomend ! Thanks a lot for the contribution, major feature 🤩 We will try to release it soon in 0.9.0

Was very fun to work on and thank you for the opportunity!

thomend added 5 commits September 29, 2024 15:14

only_show_failing argument

ce39962

added flag and changed default behavior to only show rules/models whi…

16f8079

…ch fail

changed HR formater to print Violations of all failing rules

321a846

removed redudant print statement

1cd404d

changed doc example to showcase new default behavior

7fd5b74

thomend mentioned this pull request Oct 2, 2024

Add lint option to only output models with score under some threshold #71

Closed

matthieucan reviewed Oct 3, 2024

View reviewed changes

src/dbt_score/cli.py Outdated Show resolved Hide resolved

matthieucan requested a review from jochemvandooren October 3, 2024 14:24

thomend marked this pull request as ready for review October 4, 2024 09:14

renamed CLI argument from show_all to show-all

94a1c69

jochemvandooren reviewed Oct 4, 2024

View reviewed changes

thomend and others added 2 commits October 7, 2024 21:04

Update docs/index.md

b752a41

Co-authored-by: Jochem van Dooren <jochemmathijs.vandooren@teampicnic.com>

Update src/dbt_score/cli.py

48abfc1

Co-authored-by: Jochem van Dooren <jochemmathijs.vandooren@teampicnic.com>

matthieucan reviewed Oct 22, 2024

View reviewed changes

src/dbt_score/cli.py Outdated Show resolved Hide resolved

thomend added 2 commits December 3, 2024 16:21

changed --show from flag to option

512b04f

resolved merge conflicts

f2b02a1

jochemvandooren reviewed Dec 10, 2024

View reviewed changes

matthieucan reviewed Dec 10, 2024

View reviewed changes

src/dbt_score/cli.py Outdated Show resolved Hide resolved

thomend added 4 commits December 11, 2024 06:44

replaced instancers of model with item

0f8bfd8

improved docstring

0b1dc0b

fixed if condition for printing output from evaluable

3928d56

fixed output for --failing-items mode

667908f

jochemvandooren reviewed Dec 11, 2024

View reviewed changes

docs/index.md Outdated Show resolved Hide resolved

thomend and others added 2 commits December 11, 2024 18:35

Update docs/index.md

583cc6b

Co-authored-by: Jochem van Dooren <jochemmathijs.vandooren@teampicnic.com>

added tests for show parameter

ff241de

jochemvandooren approved these changes Dec 12, 2024

View reviewed changes

added entry in changelog

3c9df1d

matthieucan approved these changes Dec 12, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

thomend and others added 2 commits December 17, 2024 16:52

Update CHANGELOG.md

ce49628

Co-authored-by: Matthieu Caneill <matthieucan@users.noreply.github.com>

run prettier to fix issue in docs

825c48b

jochemvandooren merged commit c8a7d27 into PicnicSupermarket:master Dec 18, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Only output failing models and violated rules per default in HumanReadableFormatter #77

Only output failing models and violated rules per default in HumanReadableFormatter #77

thomend commented Oct 2, 2024 •

edited by matthieucan

Loading

jochemvandooren left a comment

jochemvandooren Oct 4, 2024

thomend Oct 7, 2024 •

edited

Loading

jochemvandooren Oct 8, 2024

matthieucan Oct 9, 2024

jochemvandooren Oct 9, 2024

matthieucan Oct 22, 2024

jochemvandooren Nov 18, 2024 •

edited

Loading

thomend Dec 3, 2024

thomend Dec 3, 2024 •

edited

Loading

jochemvandooren Dec 9, 2024

jochemvandooren Oct 4, 2024

matthieucan Oct 4, 2024

thomend Oct 7, 2024 •

edited

Loading

jochemvandooren left a comment

jochemvandooren Dec 10, 2024

thomend Dec 12, 2024

jochemvandooren left a comment

thomend commented Dec 12, 2024

jochemvandooren left a comment

thomend commented Dec 12, 2024

matthieucan left a comment

thomend commented Dec 13, 2024

jochemvandooren commented Dec 18, 2024

jochemvandooren commented Dec 19, 2024

thomend commented Dec 19, 2024

		assert stdout == dedent(expected)


		def test_human_readable_formatter_model_show_all(

Only output failing models and violated rules per default in HumanReadableFormatter #77

Only output failing models and violated rules per default in HumanReadableFormatter #77

Conversation

thomend commented Oct 2, 2024 • edited by matthieucan Loading

jochemvandooren left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomend Oct 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jochemvandooren Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomend Dec 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomend Oct 7, 2024 • edited Loading

Choose a reason for hiding this comment

jochemvandooren left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jochemvandooren left a comment

Choose a reason for hiding this comment

thomend commented Dec 12, 2024

jochemvandooren left a comment

Choose a reason for hiding this comment

thomend commented Dec 12, 2024

matthieucan left a comment

Choose a reason for hiding this comment

thomend commented Dec 13, 2024

jochemvandooren commented Dec 18, 2024

jochemvandooren commented Dec 19, 2024

thomend commented Dec 19, 2024

thomend commented Oct 2, 2024 •

edited by matthieucan

Loading

thomend Oct 7, 2024 •

edited

Loading

jochemvandooren Nov 18, 2024 •

edited

Loading

thomend Dec 3, 2024 •

edited

Loading

thomend Oct 7, 2024 •

edited

Loading