Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pylint] Implement redefined-argument-from-local (R1704) #8159

Merged
merged 37 commits into from
Nov 10, 2023

Conversation

danbi2990
Copy link
Contributor

Summary

It implements Pylint rule R1704: redefined-argument-from-local

Problematic code:

def show(host_id=10.11):
    # +1: [redefined-argument-from-local]
    for host_id, host in [[12.13, "Venus"], [14.15, "Mars"]]:
        print(host_id, host)

Correct code:

def show(host_id=10.11):
    for inner_host_id, host in [[12.13, "Venus"], [14.15, "Mars"]]:
        print(host_id, inner_host_id, host)

References:
Pylint documentation
Related Issue

Test Plan

cargo test

@github-actions
Copy link
Contributor

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

scope = &checker.semantic().scopes[scope_id];
} else {
break;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if there's a slightly different way to structure this rule. Bear with me...

Instead of analyzing the AST, I think we can achieve the same effect by looking at bindings.

When we have something like:

def f(x):
    for x in y:
        ...

We create a binding for the x in def f(x), and then later, a binding for x in for x in y, and we track that this second binding shadows the first.

If you look in deferred_scopes.rs, you'll see we have some rules based on looking at these shadows:

if checker.enabled(Rule::ImportShadowedByLoopVar) {
    for (name, binding_id) in scope.bindings() {
        for shadow in checker.semantic.shadowed_bindings(scope_id, binding_id) {
            // If the shadowing binding isn't a loop variable, abort.
            let binding = &checker.semantic.bindings[shadow.binding_id()];
            if !binding.kind.is_loop_var() {
                continue;
            }

            // If the shadowed binding isn't an import, abort.
            let shadowed = &checker.semantic.bindings[shadow.shadowed_id()];
            if !matches!(
                shadowed.kind,
                BindingKind::Import(..)
                    | BindingKind::FromImport(..)
                    | BindingKind::SubmoduleImport(..)
                    | BindingKind::FutureImport
            ) {
                continue;
            }

            // If the bindings are in different forks, abort.
            if shadowed.source.map_or(true, |left| {
                binding.source.map_or(true, |right| {
                    checker.semantic.different_branches(left, right)
                })
            }) {
                continue;
            }

            #[allow(deprecated)]
            let line = checker.locator.compute_line_index(shadowed.start());

            checker.diagnostics.push(Diagnostic::new(
                pyflakes::rules::ImportShadowedByLoopVar {
                    name: name.to_string(),
                    line,
                },
                binding.range(),
            ));
        }
    }
}

I think this rule could be structured similarly: iterate over the pairs of shadowed bindings, and look for pairs in which the original binding is BindingKind::Argument, and the new binding is BindingKind::LoopVar.

Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contributing! I'd like us to explore one other approach and see if it produces the same result -- are you up for that?

@danbi2990
Copy link
Contributor Author

Appreciate the comment.
I'll check it out and let you know the result. (after daily work 😅)

@danbi2990
Copy link
Contributor Author

danbi2990 commented Nov 10, 2023

Applied the comment
but there's an issue about with stmt.

In the current code, it looks there's no BindingKind for withitem variable.
I tried adding WithItemVar to the BindingKind but failed.

Could you give an advice for the task?
You can check my trial code in the two reverted commits.

Revert "add WithItemVar"

Revert "add binding with stmt"

let shadowed = &checker.semantic.bindings[shadow.shadowed_id()];
if !shadowed.kind.is_argument() {
continue;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to check if the argument is unused? Or does it fire in either case?

Copy link
Contributor Author

@danbi2990 danbi2990 Nov 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not checking the usage.

It looks sensible to check binding.is_used() as below

if !shadowed.kind.is_argument() || !binding.is_used() {
    continue;
}

In this case, results are following

def foo(i):
    for i in range(10):  # emit error
        print(i)
        ...

def foo(i):
    for i in range(10):  # no error because it's unused
        ...

Is it good to go?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was referring to the inverse -- should this depend on the argument being unused? I'll check Pylint.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I erred on the side of removing the !binding.is_used() piece here. I think it still makes sense to raise, even if the new binding is unused, since it will cause the same problems?

@charliermarsh
Copy link
Member

Thanks @danbi2990! Overall looks good. Let me take a look at adding a binding kind for WithItem, it might be more involved.

Copy link
Contributor

github-actions bot commented Nov 10, 2023

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+24 -0 violations, +0 -0 fixes in 41 projects)

apache/airflow (+4 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --select ALL --preview

+ dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:659:51: PLR1704 Redefining argument with the local name `python`
+ dev/breeze/src/airflow_breeze/commands/sbom_commands.py:379:9: PLR1704 Redefining argument with the local name `provider_id`
+ dev/breeze/src/airflow_breeze/commands/sbom_commands.py:450:26: PLR1704 Redefining argument with the local name `provider_version`
+ scripts/in_container/verify_providers.py:188:15: PLR1704 Redefining argument with the local name `prefix`

bokeh/bokeh (+7 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --select ALL --preview

+ src/bokeh/document/callbacks.py:361:13: PLR1704 Redefining argument with the local name `callback_obj`
+ src/bokeh/plotting/_figure.py:384:13: PLR1704 Redefining argument with the local name `kw`
+ src/bokeh/plotting/_figure.py:425:13: PLR1704 Redefining argument with the local name `kw`
+ src/bokeh/plotting/_figure.py:475:17: PLR1704 Redefining argument with the local name `kw`
+ src/bokeh/plotting/_figure.py:564:13: PLR1704 Redefining argument with the local name `kw`
+ src/bokeh/plotting/_figure.py:606:13: PLR1704 Redefining argument with the local name `kw`
+ tests/support/util/examples.py:161:9: PLR1704 Redefining argument with the local name `path`

ibis-project/ibis (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ ibis/__init__.py:125:9: PLR1704 Redefining argument with the local name `name`

pandas-dev/pandas (+7 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ pandas/core/computation/eval.py:323:9: PLR1704 Redefining argument with the local name `expr`
+ pandas/core/generic.py:4747:13: PLR1704 Redefining argument with the local name `axis`
+ pandas/core/generic.py:4747:19: PLR1704 Redefining argument with the local name `labels`
+ pandas/io/common.py:591:24: PLR1704 Redefining argument with the local name `compression`
+ pandas/plotting/_matplotlib/boxplot.py:540:27: PLR1704 Redefining argument with the local name `ax`
+ pandas/tests/io/test_parquet.py:218:35: PLR1704 Redefining argument with the local name `path`
+ pandas/tests/reshape/merge/test_merge.py:566:17: PLR1704 Redefining argument with the local name `kwarg`

sphinx-doc/sphinx (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --preview

+ sphinx/ext/inheritance_diagram.py:316:13: PLR1704 Redefining argument with the local name `name`

zulip/zulip (+4 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --select ALL --preview

+ zerver/lib/scim.py:232:13: PLR1704 Redefining argument with the local name `path`
+ zerver/webhooks/clubhouse/view.py:178:9: PLR1704 Redefining argument with the local name `action`
+ zerver/webhooks/clubhouse/view.py:465:13: PLR1704 Redefining argument with the local name `action`
+ zerver/webhooks/clubhouse/view.py:669:13: PLR1704 Redefining argument with the local name `action`

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
PLR1704 24 24 0 0 0

@charliermarsh
Copy link
Member

@danbi2990 - I added a with-item kind in #8594. Can you try updating against main?

@danbi2990
Copy link
Contributor Author

It's working with updated main.
Please check updated code 🙏

Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I just need to review the ecosystem checks once they finish.

@charliermarsh charliermarsh merged commit c8edac9 into astral-sh:main Nov 10, 2023
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants