Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pylint crash when it imports module dkim "KeyError: <class '_ast.Sub'>" #3540

Closed
jnguiot opened this issue Apr 28, 2020 · 2 comments
Closed

Pylint crash when it imports module dkim "KeyError: <class '_ast.Sub'>" #3540

jnguiot opened this issue Apr 28, 2020 · 2 comments
Assignees
@PCManticore
Labels
Bug 🪲 Regression

Comments

@jnguiot
Copy link

jnguiot commented Apr 28, 2020

Hello,

I have recently encounter this strange behavior on our tests. I have boiled it down to an issue with a module named dkimpy. However, I don't understand what is the issue and why an import would crash pylint. It seems to be linked to the new version of pylint

Even if I am experiencing some issues with pylint today, I would like to thank you for this wonderful and useful piece of software.

Steps to reproduce

  1. Install latest pylint astroid : pip install pylint astroid --pre -U
  2. Install dkimpy: pip install pylint
  3. Create test.py :
import dkim
  1. Pylint it: pylint test.py

Notes

  • test.py can be executed without issue (of course, it outputs nothing)
  • It was working in 2.4.4

Current behavior

It crashes with this error:

************* Module test
test.py:1:0: C0114: Missing module docstring (missing-module-docstring)
Traceback (most recent call last):
  File "/home/jng/test/aaa/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/run.py", line 338, in __init__
    linter.check(args)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 871, in check
    self.get_ast, self._iterate_file_descrs(files_or_modules)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 904, in _check_files
    self._check_file(get_ast, check_astroid_module, name, filepath, modname)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 930, in _check_file
    check_astroid_module(ast_node)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 1063, in check_astroid_module
    ast_node, walker, rawcheckers, tokencheckers
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/lint/pylinter.py", line 1107, in _check_astroid_module
    walker.walk(ast_node)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/utils/ast_walker.py", line 72, in walk
    callback(astroid)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/pylint/checkers/variables.py", line 1177, in visit_import
    module = next(_infer_name_module(node, parts[0]))
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/util.py", line 160, in limit_inference
    yield from islice(iterator, size)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/context.py", line 113, in cache_generator
    for result in generator:
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/decorators.py", line 132, in raise_if_nothing_inferred
    yield next(generator)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/decorators.py", line 96, in wrapped
    res = next(generator)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/inference.py", line 254, in infer_import
    yield self.do_import_module(name)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/mixins.py", line 100, in do_import_module
    modname, level=level, relative_only=level and level >= 1
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/scoped_nodes.py", line 642, in import_module
    return MANAGER.ast_from_module_name(absmodname)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/manager.py", line 189, in ast_from_module_name
    return self.ast_from_file(found_spec.location, modname, fallback=False)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/manager.py", line 98, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/builder.py", line 137, in file_build
    module = self._data_build(data, modname, path)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/builder.py", line 190, in _data_build
    module = builder.visit_module(node, modname, node_file, package)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 165, in visit_module
    newnode.postinit([self.visit(child, newnode) for child in node.body])
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 165, in <listcomp>
    newnode.postinit([self.visit(child, newnode) for child in node.body])
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 177, in visit
    return visit_method(node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 676, in visit_functiondef
    return self._visit_functiondef(nodes.FunctionDef, node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 666, in _visit_functiondef
    body=[self.visit(child, newnode) for child in node.body],
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 666, in <listcomp>
    body=[self.visit(child, newnode) for child in node.body],
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 177, in visit
    return visit_method(node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 911, in visit_return
    newnode.postinit(self.visit(node.value, newnode))
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 177, in visit
    return visit_method(node, parent)
  File "/home/jng/test/aaa/lib/python3.7/site-packages/astroid/rebuilder.py", line 384, in visit_binop
    self._bin_op_classes[type(node.op)], node.lineno, node.col_offset, parent
KeyError: <class '_ast.Sub'>

Expected behavior

It should not crash. If there is a problem with import, it should deliver an understandable message.

pylint --version output

pylint 2.5.0
astroid 2.4.0
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0]
@jnguiot jnguiot mentioned this issue Apr 28, 2020
Closed
@PCManticore PCManticore self-assigned this Apr 29, 2020
@PCManticore PCManticore mentioned this issue Apr 29, 2020
Closed
PCManticore added a commit to pylint-dev/astroid that referenced this issue Apr 29, 2020
@PCManticore
Restructure the AST parsing heuristic to always pick the same module
b71df5b 
When a file contained a misplaced type annotation, we were retrying the parsing
without type comments support. That second parsing was using the builtin ast module,
but the rest of the tree utilities (the builder and rebuilder) were not aware of the
new parsing module that was used to build the AST nodes a second time.

This commit moves the logic of picking the parsing module and the corresponding
AST node mapping in a single place, which can be used by both the builder
and the rebuilder.

Close pylint-dev/pylint#3540
Close #773
@PCManticore PCManticore mentioned this issue Apr 29, 2020
Merged
PCManticore added a commit to pylint-dev/astroid that referenced this issue Apr 29, 2020
@PCManticore
Restructure the AST parsing heuristic to always pick the same module
efb5155 
When a file contained a misplaced type annotation, we were retrying the parsing
without type comments support. That second parsing was using the builtin ast module,
but the rest of the tree utilities (the builder and rebuilder) were not aware of the
new parsing module that was used to build the AST nodes a second time.

This commit moves the logic of picking the parsing module and the corresponding
AST node mapping in a single place, which can be used by both the builder
and the rebuilder.

Close pylint-dev/pylint#3540
Close #773
@PCManticore
Copy link
Contributor

Thanks for the report! This should already be fixed in the 2.4 and master branches on astroid. Please give it a go and let us know if you encounter any issues. We'll release the fix as astroid 2.4.1 in a couple of days.

@jnguiot
Copy link
Author

jnguiot commented Apr 29, 2020

Indeed I can confirm it works with the latest 2.4 version of astroid. Thank you for the quick fix!

@ray-grointel ray-grointel mentioned this issue May 1, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@PCManticore PCManticore

Labels
Bug 🪲 Regression
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@PCManticore @jnguiot