bytes is not being expanded to Union[bytes|bytearray|memoryview]

[paxswill]

Environment data

• Language Server version: v2020.12.0
• OS and version: WSL2 Ubuntu 20.04.1
• Python version (& distribution if applicable, e.g. Anaconda): System Python 3.8.5

Expected behaviour

bytes is able to be used as a shorthand for typing.Union[bytes|bytearray|memoryview] in argument types, so this code should not have a typing error:

#!/usr/bin/env python3
import os

fd = os.open("./example", os.O_WRONLY | os.O_CREAT)
data = bytearray(b"hello world\n")
os.write(fd, data)
os.close(fd)

Actual behaviour

The second argument to os.write() is marked with this message:

Argument of type "bytearray" cannot be assigned to parameter "__data" of type "bytes" in function "write"
  "bytearray" is incompatible with "bytes"

Logs

Not sure I enabled this correctly, there was only one mention of Pylance.

User belongs to experiment group 'ShowPlayIcon - start'
User belongs to experiment group 'DebugAdapterFactory - experiment'
User belongs to experiment group 'PtvsdWheels37 - experiment'
User belongs to experiment group 'UseTerminalToGetActivatedEnvVars - control'
User belongs to experiment group 'LocalZMQKernel - experiment'
User belongs to experiment group 'CollectLSRequestTiming - control'
User belongs to experiment group 'CollectNodeLSRequestTiming - experiment'
User belongs to experiment group 'EnableIPyWidgets - experiment'
User belongs to experiment group 'DeprecatePythonPath - control'
User belongs to experiment group 'RunByLine - experiment'
User belongs to experiment group 'CustomEditorSupport - control'
User belongs to experiment group 'pythonaa'
> conda --version
> pyenv root
> python3.7 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3.6 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python2 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> /usr/bin/python3 ~/.vscode-server/extensions/ms-python.python-2020.11.371526539/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> conda info --json
Error 2020-12-05 13:30:00: Failed to check if file needs to be fixed [EntryNotFound (FileSystemError): Unable to read file 'vscode-remote://wsl+ubuntu-20.04/home/paxswill/.config/Code/User/settings.json' (EntryNotFound (FileSystemError): Error: ENOENT: no such file or directory, open '/home/paxswill/.config/Code/User/settings.json')
	at _handleError (/home/paxswill/.vscode-server/bin/e5a624b788d92b8d34d1392e4c4d9789406efe8f/out/vs/server/remoteExtensionHostProcess.js:754:838)
	at processTicksAndRejections (internal/process/task_queues.js:94:5)
	at async y.readText (/home/paxswill/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:9:152841)
	at async p.doesFileNeedToBeFixed (/home/paxswill/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:469606)
	at async /home/paxswill/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:468733
	at async Promise.all (index 0)
	at async p.getFilesToBeFixed (/home/paxswill/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:468679)
	at async p.updateTestSettings (/home/paxswill/.vscode-server/extensions/ms-python.python-2020.11.371526539/out/client/extension.js:39:468306)] {
  code: 'FileNotFound',
  name: 'EntryNotFound (FileSystemError)'
}
Starting Pylance language server.
Python interpreter path: /usr/bin/python3
> conda --version

Code Snippet / Additional information

The shorthand is documented. I came across this when looking at the history of python/typeshed#3109, where one of the review comments said that expanding the typing stub for os.write() to include types other than bytes wasn't necessary.

[erictraut]

Thanks for the bug report. I hadn't ever heard of this either. I've added the support, and it will be in the next release.

[Kein]

Is this something I'm struggling with myself?
Here is an example:

def parse_str(i: int, data: ???):
    # do some
    if (data[i+1] <= 0): # taking element as an int here
        # dome some
    result = codecs.decode(data[x:y], 'encoding')

If I use type Union[bytes, bytearray, memoryview] for the data type hint then I have no issues on call-site (), but there will be warning for codecs.decode(data, literal):
No overloads for "codecs.decode(data[x: y], "latin")" match parameters: Argument types: (bytes | memoryview, Literal['latin'])
If i specify bytes as a type for data then codecs is satisfied but at call-site when I pass memoryview as an argument to parse_str I get a similar warning.

I'm completely confused here - is this a Pylance-strict issue or I dont understand something about python type-checking?

[jakebailey]

Unless I'm mistaken, the effect of the fix will be that any of those three types will be compatible with bytes, so your code should be able to type check either of the two ways you've described it.

[Kein]

So in other words it is/was pylance-specific issue? Since I dont have much experience working in python env I wasnt sure if vanilla mypy is under-reporting (since it has zero issues whatsoever with anything I put in there) or pylance being wrong.

[erictraut]

Even though there is no actual class relationship between bytes, bytearray and memoryview (i.e. none of them are subclasses of each other), bytes is documented to accept any of the three types. There are a few anomalous special cases like this in Python typing. The other documented example is that complex accepts float or int and float accepts int. These are sometimes referred to as "type promotions".

The version of Pylance that was published last week doesn't properly handle the type promotions for bytes. It will be addressed in this week's release.

[Kein]

Hmm, okay, sure. But I dont think that explains why explicitly stated type Union[bytes, bytearray, memoryview] is being reported as a type error when handled to codecs.decode ( as taken from y example above):

def parse_str(i: int, data: Union[bytes, bytearray, memoryview]) -> str:
    # do some
    if (data[i+1] <= 0): # taking element as an int here
        # dome some
    result = codecs.decode(data[x:y], 'encoding')
                              ^--- type error here

Why Pylance reports a type error here in this case?

[erictraut]

Union[bytes|bytearray|memoryview] isn't a valid type annotation. It should be Union[bytes, bytearray, memoryview]. Or if you're using Python 3.10a, you can use bytes | bytearray | memoryview.

[Kein]

Union[bytes|bytearray|memoryview] isn't a valid type annotation. It should be Union[bytes, bytearray, memoryview]. Or if you're using Python 3.10a, you can use bytes | bytearray | memoryview.

Sorry, that was just a mock up, yes, of course it is typed Union[bytes, bytearray, memoryview] that is not the issue that needed the answer, though. The issue is why i'm getting type warning from Pylance here when type is explicit.
Language server version is the same as in OP

[erictraut]

This is definitely not the same problem as the bug that was reported in this issue.

If you think there's a separate bug, please open another issue and provide a minimal code sample that exhibits the problem.

[jakebailey]

This issue has been fixed in version 2020.12.1, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/master/CHANGELOG.md#2020121-9-december-2020