Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RecursionError on numpy 2.0.x when completing numpy.finfo and reading type #2020

Open
niloc132 opened this issue Aug 1, 2024 · 4 comments
Open

RecursionError on numpy 2.0.x when completing numpy.finfo and reading type #2020

niloc132 opened this issue Aug 1, 2024 · 4 comments
Labels
bug

Comments

@niloc132
Copy link

niloc132 commented Aug 1, 2024

Steps to reproduce:

Install latest jedi (currently 0.19.1) and numpy (verified with 2.0.0 and 2.0.1)

$ python -m jed
$ python -m venv jedi-numpy
$ source jedi-numpy/bin/activate
$ pip install jedi numpy
$ python

Verify versions

>>> import jedi
>>> jedi.__version__
'0.19.1'
>>> import numpy
>>> numpy.__version__
'2.0.1'

Try to read the type of numpy.finfo:

>>> from jedi import Interpreter
>>> items = Interpreter('import numpy\nnumpy.finfo', [locals()]).complete()
>>> items
[<Completion: finfo>]
>>> items[0].type
Traceback (most recent call last):
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/api/completion_cache.py", line 21, in _get_from_cache
    return _cache[module_name][name][number]
KeyError: 'numpy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 651, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 238, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 356, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
... trimmed 1876 lines here, the same 14 as above, still repeated a few times below...
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 651, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 238, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 356, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 651, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 238, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 356, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 651, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 238, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 356, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 651, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 238, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 356, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 651, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 238, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 356, in py__doc__
    return _merge_name_docs(names)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 24, in _merge_name_docs
    doc += name.py__doc__()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/names.py", line 645, in py__doc__
    names = convert_names(names, prefer_stub_to_compiled=False)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/gradual/conversion.py", line 154, in convert_names
    return _try_stub_to_python_names(
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/utils.py", line 16, in wrapper
    return list(func(*args, **kwargs))
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/gradual/conversion.py", line 73, in _try_stub_to_python_names
    values = convert_values(name.infer(), ignore_compiled=prefer_stub_to_compiled)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/gradual/conversion.py", line 168, in convert_values
    return ValueSet.from_sets(
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/base_value.py", line 430, in from_sets
    for set_ in sets:
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/gradual/conversion.py", line 169, in <genexpr>
    _stub_to_python_value_set(stub_value, ignore_compiled=ignore_compiled)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/gradual/conversion.py", line 36, in _stub_to_python_value_set
    values = _infer_from_stub(stub_module_context, qualified_names, ignore_compiled)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/gradual/conversion.py", line 60, in _infer_from_stub
    non_stubs = non_stubs.py__getattribute__(name)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/base_value.py", line 496, in py__getattribute__
    return ValueSet.from_sets(c.py__getattribute__(*args, **kwargs) for c in self._set)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/base_value.py", line 430, in from_sets
    for set_ in sets:
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/base_value.py", line 496, in <genexpr>
    return ValueSet.from_sets(c.py__getattribute__(*args, **kwargs) for c in self._set)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/base_value.py", line 83, in py__getattribute__
    names = self.goto(name_or_str, name_context, analysis_errors)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/base_value.py", line 72, in goto
    names = finder.filter_name(filters, name_or_str)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/finder.py", line 35, in filter_name
    for filter in filters:
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/base_value.py", line 62, in _get_value_filters
    yield from self.get_filters(origin_scope=origin_scope)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/value/module.py", line 63, in get_filters
    ParserTreeFilter(
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/filters.py", line 138, in __init__
    super().__init__(parent_context, node_context)
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/filters.py", line 85, in __init__
    module_context = node_context.get_root_context()
  File "/home/colin/workspace/illumon/core.new/jedi-numpy/lib/python3.10/site-packages/jedi/inference/context.py", line 111, in get_root_context
    parent_context = self.parent_context
RecursionError: maximum recursion depth exceeded

Workaround

If you have already imported the numpy module, this will not happen, as long as you don't hit cached results (i.e. import numpy above before the Interpreter is created). This suggests to me at least that this is an edge case in reading types from not-yet-imported modules, rather than a general issue.

On the other hand, a call to jedi.preload_module("numpy") does not resolve this by itself resolve this.
@niloc132 niloc132 mentioned this issue Aug 1, 2024
Closed
@davidhalter
Copy link
Owner

It feels like this is definitely a bug, _merge_name_docs should not be called like this I guess. There are other cases where RecursionErrors could be caused by legitimate cases, but this is not inference.

If you want to help, it would really be awesome to have a smaller reproducer for this. This is probably reproducible with a few lines of Python code.

@davidhalter davidhalter added the bug label Aug 3, 2024
yut23 added a commit to yut23/jedi that referenced this issue Aug 22, 2024
@yut23
Add test case/reproducer for issue davidhalter#2020
99e811a 
Trimmed down from `numpy.matrix`.
@yut23
Copy link

yut23 commented Aug 22, 2024

I got annoyed by my editor lagging out whenever I typed np., so I trimmed it down a simple reproducer: yut23@99e811a

The important part appears to be that the inner stub imports matrix from the outer module with an alias. Changing that line to from repro2020 import matrix makes the RecursionError go away.

@niloc132
Copy link
Author

Thank you for working on this - I managed to break it down so that I could add the bug to numpy 1.26, or remove it from 2.0.x, but was not able to reproduce it in a small, simple project. Your example has the highlights: either replace the matrix as matrix with just matrix in inner/__init__.pyi or change the import in __init__.py to use a wildcard - either change and the bug will go away.

Also, and I'm not sure if this is a bug, but omitting the inner/__init__.pyi file will result in repro2020.matrix being described by jedi as having a "module" type rather than a "class" type.

@davidhalter
Copy link
Owner

That sounds really strange. But I guess if that reproduces it, I'm very happy you made it work @yut23.

Does the traceback still look the same? With a lot of _merge_name_docs in there?

I'm not surprised that there is a RecursionError, but I'm surprised that that case comes from _merge_name_docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@niloc132 @yut23 @davidhalter