multiprocessing classes SimpleQueue and Queue don't support typing in 3.11.0

[rafrafek]

Bug report

SimpleQueue and Queue classes from multiprocessing module in Python 3.11.0 do not support type [str] annotation.

Minimal, reproducible example:

from multiprocessing import Queue
multiprocessing_queue: Queue[str] = Queue()

or

from multiprocessing import SimpleQueue
multiprocessing_queue: SimpleQueue[str] = SimpleQueue()

Result - error:

    multiprocessing_queue: SimpleQueue[str] = SimpleQueue()
                           ~~~~~~~~~~~^^^^^
TypeError: 'method' object is not subscriptable

How it should work:

It should work like Queue from the queue module:

from queue import Queue
standard_queue: Queue[str] = Queue()

Result - no error.

Why do I need this?

I want my IDE to know that queue.get() returns str object.

Your environment

Python 3.11.0 arm64
Python 3.11.0 (main, Nov 4 2022, 17:22:54) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
MacBook M1 Pro macOS Ventura 13.0.1.

Linked PRs

• gh-99509: Add __class_getitem__ to multiprocessing.Queue #99511

[linguini1]

I am using Python 3.11.4 and still get this error:

from multiprocessing import Queue
q: Queue[str] = Queue()

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'method' object is not subscriptable

And using

import multiprocessing
from multiprocessing.queues import Queue
q: Queue[str] = multiprocessing.Queue()

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: type 'Queue' is not subscriptable

SimpleQueue subscripting only works when imported using from multiprocessing.queues import SimpleQueue.

[Starbuck5]

I believe this should be re-opened.

As reviewers noted in the linked PR, the linked PR only solves it for an undocumented way to import Queue. Also, even with the new import it's not a drop in replacement. I tested with 3.12beta 3.

from multiprocessing.queues import Queue

queue: Queue[str] = Queue()
print(queue)

The new import yields an error.

Traceback (most recent call last):
  File "myfilelocation\test.py", line 3, in <module>
    queue: Queue[str] = Queue()
                        ^^^^^^^
TypeError: Queue.__init__() missing 1 required keyword-only argument: 'ctx'

To work around subscripting after this PR, you need to use the undocumented queue import and then provide this unknown context argument directly.

from multiprocessing.queues import Queue
import multiprocessing

queue: Queue[str] = Queue(ctx=multiprocessing.get_context())
print(queue)

And of course the documented way of making a Queue still fails.

from multiprocessing import Queue

queue: Queue[str] = Queue()
print(queue)

Traceback (most recent call last):
  File "myfilelocation\test2.py", line 3, in <module>
    queue: Queue[str] = Queue()
           ~~~~~^^^^^
TypeError: 'method' object is not subscriptable

[hongwen000]

@Starbuck5 Same issue

[AlexWaygood]

To make both mypy and CPython happy, you can do something like this:

import multiprocessing
import multiprocessing.queues

queue: multiprocessing.queues.Queue[str] = multiprocessing.Queue()

This accurately reflects the facts that:

1. multiprocessing.Queue() is the documented, public-API way of creating the class; however,
2. multiprocessing.Queue itself is not a type (it's a method), so can't be used in a type annotation

Could we special-case multiprocessing.Queue, perhaps by pretending in typeshed that it's a class rather than a method? Perhaps we could, but we like to avoid doing that kind of thing as much as possible. It's generally best for type checkers to be told the truth; lying to type checkers in typeshed about whether things are methods or classes can have unfortunate consequences. In any event, that conversation would probably need to take place on the https://github.com/python/typeshed/issues issue tracker rather than at CPython.

Could multiprocessing.Queue itself be changed so that it's an actual class rather than a method? Again, perhaps. I'm not a multiprocessing expert, though (just a typing expert), so I can't really comment on that one.