F401 - Distinguish between imports we wish to remove and those we wish to make explicit-exports #11168
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
plredmond
force-pushed
the
ruff.F401
branch
2 times, most recently
from
d9d863e
to
0baf16e
Compare
plredmond
commented
Apr 29, 2024
plredmond
commented
Apr 29, 2024
plredmond
commented
Apr 29, 2024
|
| code | total | + violation | - violation | + fix | - fix |
|---|---|---|---|---|---|
| F401 | 104 | 0 | 0 | 104 | 0 |
zanieb
reviewed
Apr 29, 2024
crates/ruff_linter/resources/test/fixtures/pyflakes/F401_24/__init__.py
Outdated
Show resolved
Hide resolved
crates/ruff_linter/resources/test/fixtures/pyflakes/F401_25__all/__init__.py
Outdated
Show resolved
Hide resolved
...rules/pyflakes/snapshots/ruff_linter__rules__pyflakes__tests__F401_F401_24____init__.py.snap
Outdated
Show resolved
Hide resolved
...rules/pyflakes/snapshots/ruff_linter__rules__pyflakes__tests__F401_F401_24____init__.py.snap
Outdated
Show resolved
Hide resolved
|
The ecosystem checks look concerning. We're losing fixes in preview? I think there should be no changes in stable? |
|
Yes, yes those do look concerning. I'll look into it. |
…ability; remove unnecessary pattern match
…urn a fix if imports were actually given
… either moves to __all__ or makes imports explicit
… two groups of import statement bindings; then iterate over those bindings and the fix which applies to them
…ework after discussion w/zanie
|
Not sure what's up with the Latch changes. I ran it locally and they are indeed classified as first-party (at least by the isort rule). |
MichaReiser
reviewed
May 1, 2024
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Micha Reiser <micha@reiser.io>
| Some(UnusedImportContext::Init { | ||
| first_party: is_first_party( | ||
| checker, | ||
| &binding.import.qualified_name().to_string(), |
There was a problem hiding this comment.
Just trying to understand what's happening in Latch, at least one thing is that I think isort::categorize expects .module_name() rather than .qualified_name(), so this should probably be binding.import.module_name() (although I wouldn't expect it to change the outcome -- the module name is generally a prefix of the qualified name, and the import categorization just operates on the first segment).
There was a problem hiding this comment.
Oh interesting, but crates/ruff_linter/src/rules/flake8_type_checking/rules/typing_only_runtime_import.rs uses the .qualified_name with level=0. Interesting... I mean, on further review, it's possible that level=0 should work here, because the binding name is meant to be the fully-resolved name.
Like, if the user does this from a module named foo:
from . import barThen the qualified name on the binding should be ["foo", "bar"] -- and so we should be able to resolve it without taking into account the dots. I'm surprised it's not working for those projects. It's probably worth checking them out, running them locally, and looking at what binding.import.qualified_name is here.
|
Done with this round of review. I think the outstanding issues (in order of importance) are: (1) Incorrectly categorizing many imports as not first-party in ecosystem results.
(2) Dangling `__all__` code
(3) Some functions still take `impl Iterator<...>` but might be better to take `&'a [...]`
|
|
|
|
I think there might just be a bug in my logic somewhere, since it seems that the only things being miscategorized are bindings in the syntax So I'll write a test for that syntax and see what happens with it. |
|
Cool, I can help take a look at the remaining Latch categorizations tomorrow if it’s helpful! |
|
On second glance, aren't the ecosystem changes correct? We're now offering fixes for first-party imports in |
This comment was marked as duplicate.
This comment was marked as duplicate.
Sorry, something went wrong.
charliermarsh
approved these changes
May 2, 2024
|
Congrats on the merge :) |
plredmond
added a commit
that referenced
this pull request
May 15, 2024
Followup on #11168 and resolve #10391 # User facing changes * F401 now recommends a fix to add unused import bindings to to `__all__` if a single `__all__` list or tuple is found in `__init__.py`. * If there are no `__all__` found in the file, fall back to recommending redundant-aliases. * If there are multiple `__all__` or only one but of the wrong type (non list or tuple) then diagnostics are generated without fixes. * `fix_title` is updated to reflect what the fix/recommendation is. Subtlety: For a renamed import such as `import foo as bees`, we can generate a fix to add `bees` to `__all__` but cannot generate a fix to produce a redundant import (because that would break uses of the binding `bees`). # Implementation changes * Add `name` field to `ImportBinding` to contain the name of the _binding_ we want to add to `__all__` (important for the `import foo as bees` case). It previously only contained the `AnyImport` which can give us information about the import but not the binding. * Add `binding` field to `UnusedImport` to contain the same. (Naming note: the field `name` field already existed on `UnusedImport` and contains the qualified name of the imported symbol/module) * Change `fix_by_reexporting` to branch on the size of `dunder_all: Vec<&Expr>` * For length 0 call the edit-producing function `make_redundant_alias`. * For length 1 call edit-producing function `add_to_dunder_all`. * Otherwise, produce no fix. * Implement the edit-producing function `add_to_dunder_all` and add unit tests. * Implement several fixture tests: empty `__all__ = []`, nonempty `__all__ = ["foo"]`, mis-typed `__all__ = None`, plus-eq `__all__ += ["foo"]` * `UnusedImportContext::Init` variant now has two fields: whether the fix is in `__init__.py` and how many `__all__` were found. # Other changes * Remove a spurious pattern match and instead use field lookups b/c the addition of a field would have required changing the unrelated pattern. * Tweak input type of `make_redundant_alias` --------- Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
plredmond
added a commit
that referenced
this pull request
May 15, 2024
plredmond
added a commit
that referenced
this pull request
May 15, 2024
plredmond
added a commit
that referenced
this pull request
May 21, 2024
#11436) ## Summary * Update documentation for F401 following recent PRs * #11168 * #11314 * Deprecate `ignore_init_module_imports` * Add a deprecation pragma to the option and a "warn user once" message when the option is used. * Restore the old behavior for stable (non-preview) mode: * When `ignore_init_module_imports` is set to `true` (default) there are no `__init_.py` fixes (but we get nice fix titles!). * When `ignore_init_module_imports` is set to `false` there are unsafe `__init__.py` fixes to remove unused imports. * When preview mode is enabled, it overrides `ignore_init_module_imports`. * Fixed a bug in fix titles where `import foo as bar` would recommend reexporting `bar as bar`. It now says to reexport `foo as foo`. (In this case we don't issue a fix, fwiw; it was just a fix title bug.) ## Test plan Added new fixture tests that reuse the existing fixtures for `__init__.py` files. Each of the three situations listed above has fixture tests. The F401 "stable" tests cover: > * When `ignore_init_module_imports` is set to `true` (default) there are no `__init_.py` fixes (but we get nice fix titles!). The F401 "deprecated option" tests cover: > * When `ignore_init_module_imports` is set to `false` there are unsafe `__init__.py` fixes to remove unused imports. These complement existing "preview" tests that show the new behavior which recommends fixes in `__init__.py` according to whether the import is 1st party and other circumstances (for more on that behavior see: #11314).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #10390 and starts to address #10391
Changes to behavior
__init__.pywe now offer some fixes for unused imports.__init__.py.checker.settings.ignore_init_module_importssetting is deprecated/ignored. There is probably a documentation change to make that complete which I haven't done.Old description of implementation changes
Changes to the implementation
to_reexportandto_remove(according to how we want to resolve the fact they're unused) with the following predicate:to_removefixes are unsafe whenin_init.to_explicitfixes are safe. Currently, until a future PR, we make them redundant aliases (e.g.import awould becomeimport a as a).Other changes
checker.settings.ignore_init_module_importsis deprecated/ignored. Instead, all fixes are gated onchecker.settings.preview.is_enabled().// FIXME: rename "imports" to "bindings"if reviewer agrees (see code)// FIXME: rename "node_id" to "import_statement"if reviewer agrees (see code)Scope cut until a future PR
to_explicitfixes will be added to__all__unless it doesn't exist. When__all__doesn't exist they're resolved by converting to redundant aliases (e.g.import awould becomeimport a as a).Test plan
crates/ruff_linter/resources/test/fixtures/pyflakes/F401_24contains an__init__.pywithout__all__that exercises the features in this PR, but it doesn't pass.crates/ruff_linter/resources/test/fixtures/pyflakes/F401_25_dunder_allcontains an__init__.pywith__all__that exercises the features in this PR, but it doesn't pass.fix::edits::make_redundant_alias.