Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ruff] Detect unnecessary dict comprehensions for iterables (RUF025) #9613

Merged
merged 13 commits into from
Jan 24, 2024

Conversation

mikeleppane
Copy link
Contributor

Summary

Checks for unnecessary dict comprehension when creating a new dictionary from iterable. Suggest to replace with dict.fromkeys(iterable)

See: #9592

Test Plan

cargo test

@mikeleppane
Copy link
Contributor Author

Remark

I'm getting the following error when running tests locally with cargo test:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot file: crates/ruff_linter/src/rules/flake8_executable/snapshots/ruff_linter__rules__flake8_executable__tests__EXE001_1.py.snap
Snapshot: EXE001_1.py
Source: crates/ruff_linter/src/rules/flake8_executable/mod.rs:43
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
    0       │-EXE001_1.py:1:1: EXE001 Shebang is present but file is not executable␊
    1       │-  |␊
    2       │-1 | #!/usr/bin/python␊
    3       │-  | ^^^^^^^^^^^^^^^^^ EXE001␊
    4       │-2 | ␊
    5       │-3 | if __name__ == '__main__':␊
    6       │-  |␊
────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
To update snapshots run `cargo insta review`
Stopped on the first failure. Run `cargo insta test` to run all snapshots.
thread 'rules::flake8_executable::tests::rules::path_new_exe001_1_py_expects' panicked at /home/mikko/.cargo/registry/src/index.crates.io-6f17d22bba15001f/insta-1.34.0/src/runtime.rs:563:9:
snapshot assertion for 'EXE001_1.py' failed in line 43
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    rules::flake8_executable::tests::rules::path_new_exe001_1_py_expects
    rules::flake8_executable::tests::rules::path_new_exe002_1_py_expects

Is this a known issue or should I do something differently?

Suggestion

Have you considered using cargo-make? It could be used to execute e.g. fmt, clippy and tests in one task.

Copy link
Contributor

github-actions bot commented Jan 22, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+28 -0 violations, +0 -0 fixes in 11 projects; 32 projects unchanged)

DisnakeDev/disnake (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ disnake/ext/commands/flag_converter.py:314:28: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

RasaHQ/rasa (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ rasa/core/evaluation/marker_stats.py:114:51: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rasa/core/featurizers/single_state_featurizer.py:119:20: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ tests/core/featurizers/test_precomputation.py:110:17: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

apache/airflow (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview --select ALL

+ helm_tests/airflow_aux/test_annotations.py:409:63: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

bokeh/bokeh (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview --select ALL

+ src/bokeh/command/subcommands/file_output.py:145:17: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ src/bokeh/command/subcommands/serve.py:828:17: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

ibis-project/ibis (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ ibis/backends/dask/tests/execution/conftest.py:71:20: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ ibis/common/egraph.py:768:17: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

pypa/build (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ src/build/__main__.py:37:14: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

pypa/cibuildwheel (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ cibuildwheel/util.py:558:24: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

rotki/rotki (+12 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ rotkehlchen/api/rest.py:4472:28: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/ethereum/modules/convex/decoder.py:290:17: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/ethereum/modules/convex/decoder.py:291:27: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/ethereum/modules/curve/decoder.py:768:62: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/ethereum/modules/curve/decoder.py:772:24: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/ethereum/modules/uniswap/v3/decoder.py:551:16: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/evm/decoding/gitcoin/decoder.py:266:65: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/evm/decoding/gitcoin/decoder.py:269:21: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/evm/decoding/gitcoin/decoder.py:272:21: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/evm/decoding/interfaces.py:249:37: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/optimism/modules/velodrome/decoder.py:247:62: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ rotkehlchen/chain/optimism/modules/velodrome/decoder.py:251:24: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

scikit-build/scikit-build (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ skbuild/setuptools_wrap.py:521:22: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ skbuild/setuptools_wrap.py:524:19: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

scikit-build/scikit-build-core (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ src/scikit_build_core/_logging.py:102:14: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

zulip/zulip (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview --select ALL

+ zerver/tests/test_message_fetch.py:3373:23: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead
+ zproject/backends.py:168:31: RUF025 [*] Unnecessary dict comprehension for iterable; use `dict.fromkeys` instead

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
RUF025 28 28 0 0 0

@charliermarsh charliermarsh self-requested a review January 22, 2024 15:56
@charliermarsh charliermarsh added rule Implementing or modifying a lint rule preview Related to preview mode features labels Jan 22, 2024
…or key-value pair in comprehension is binded to a same name
Copy link
Member

@dhruvmanila dhruvmanila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty solid, thanks for working on this!

I've left some minor nits and one question about the fix generation but otherwise it looks good to go.

/// ```
#[violation]
pub struct UnnecessaryDictComprehensionForIterable {
is_value_none_literal: bool,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we could use an enum instead of a boolean. A boolean value is hard to read until it is linked with some context like a variable name. An enum can prove to be readable in such cases.

enum Value {
	// A `None` literal
	NoneLiteral,
	// Any value other than `None`
	Other
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I thought about using an Enum but it kind of felt slightly over board for such a simple thing.


impl AlwaysFixableViolation for UnnecessaryDictComprehensionForIterable {
#[derive_message_formats]
#[allow(clippy::match_bool)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this must be a leftover and can be removed now?

}
}

#[allow(clippy::match_bool)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this must be a leftover and can be removed now?

Comment on lines 78 to 80
// Don't suggest `dict.fromkeys` for async generator expressions, because `dict.fromkeys` is not async.
// Don't suggest `dict.fromkeys` for nested generator expressions, because `dict.fromkeys` might be error-prone option at least for fixing.
// Don't suggest `dict.fromkeys` for generator expressions with `if` clauses, because `dict.fromkeys` might not be valid option.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we combine these comments and use bullet points instead?

// Don't suggest `dict.fromkeys` for
// - ..., because ...

Comment on lines 84 to 86
let Expr::Name(_) = &generator.target else {
return;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we can use the is_ method on the node: generator.target.is_name_expr

/// Generate a [`Fix`] to replace `dict` comprehension with `dict.fromkeys`.
/// (RUF025) Convert `{n: None for n in [1,2,3]}` to `dict.fromkeys([1,2,3])` or
/// `{n: 1 for n in [1,2,3]}` to `dict.fromkeys([1,2,3], 1)`.
fn fix_unnecessary_dict_comprehension(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason you choose to use libcst nodes instead of the AST nodes to construct the fixed node? I might be wrong but I think it might be easier to construct the node using the AST nodes with the Generator to generate the equivalent source code. I'm fine with either but just trying to understand this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, specific reason. I was not quite sure what would be the correct way to construct the fixed node. Seems that using AST node with Generator would be more reasonable.

@mikeleppane
Copy link
Contributor Author

@dhruvmanila thanks for the good comments! 👍

Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

return;
}

if !is_constant(dict_comp.value.as_ref()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mikeleppane - I ended up using is_constant here with doesn't include Expr::Name, so that changed some of the test case results.

For example, we no longer flag this:

def func():
    values = ["a", "b", "c"]
    [{n: values for n in [1,2,3]}] # RUF025

But I think we're correct not to flag, since in that case, values is actually mutable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, hmm, but the user is already sharing it between elements. What you had is probably correct then.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refining back.

@charliermarsh charliermarsh changed the title [RUF] - Add unnecessary dict comprehension for iterable rule (RUF025) [ruff] Detect unnecessary dict comprehensions for iterables (RUF025) Jan 24, 2024
@charliermarsh charliermarsh enabled auto-merge (squash) January 24, 2024 02:05
@charliermarsh charliermarsh merged commit eab1a68 into astral-sh:main Jan 24, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants