pip fails to log an existing distribution if its location attribute is None

[daniil-konovalenko]

Description

I've encountered an issue with how pip handles custom Distributions that have their location atrribute set to None.
As far as I understand, it's allowed, because the location type is here Optional:

pip/src/pip/_internal/metadata/importlib/_compat.py

Lines 34 to 43 in a057c9b

	def get_info_location(d: importlib.metadata.Distribution) -> Optional[BasePath]:
	"""Find the path to the distribution's metadata directory.
	HACK: This relies on importlib.metadata's private ``_path`` attribute. Not
	all distributions exist on disk, so importlib.metadata is correct to not
	expose the attribute as public. But pip's code base is old and not as clean,
	so we do this to avoid having to rewrite too many things. Hopefully we can
	eliminate this some day.
	"""
	return getattr(d, "_path", None)

And

Not all distributions exist on disk

This is exactly my case — I have a custom meta_path finder that loads Distributions on the fly.
When I'm trying to install an already installed package, pip finds an existing distribution and tries to log a message. It fails here when it tries to convert None to an absolute path (see below for the traceback).

pip/src/pip/_internal/req/req_install.py

Line 198 in a057c9b

s += " in {}".format(display_path(self.satisfied_by.location))

I can fix this internally by overriding the Distribution._path attribute, but I feel that it's better to resolve the underlying issue in pip itself. Perhaps we could fall back to a placeholder like <memory> in case the location is None.

I realise that this is a pretty rare case, and I'll be happy to submit a PR, if the proposed solution is acceptable.

Expected behavior

I think it's better to display a placeholder in case the location is None. Something like

$ pip install example
Requirement already satisfied: example in <memory> (1.0.0)

pip version

22.3.1

Python version

3.11.1

OS

macOS 12.6.2

How to Reproduce

• create a fresh virtualenv: python -m venv venv
• create a sitecustomize.py file in the virtualenv's site-packages directory with with the following contents:

import sys
from importlib.metadata import Distribution, DistributionFinder


EXAMPLE_METADATA = '''Metadata-Version: 2.1
Name: example
Version: 1.0.0

'''


class ExampleDistribution(Distribution):
    def locate_file(self, path):
        return path

    def read_text(self, filename):
        if filename == 'METADATA':
            return EXAMPLE_METADATA


class CustomFinder(DistributionFinder):
    def find_distributions(self, context=None):
        return [ExampleDistribution()]


sys.meta_path.append(CustomFinder())

• run pip install example

Output

$ pip install example
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 158, in _make_candidate_from_dist
    base = self._installed_candidate_cache[dist.canonical_name]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
KeyError: 'example'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 400, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 147, in _add_to_criteria
    matches = self._p.find_matches(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 224, in find_matches
    return self._factory.find_candidates(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 424, in find_candidates
    return self._iter_found_candidates(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 320, in _iter_found_candidates
    _get_installed_candidate(),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 268, in _get_installed_candidate
    candidate = self._make_candidate_from_dist(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 160, in _make_candidate_from_dist
    base = AlreadyInstalledCandidate(dist, template, factory=self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 354, in __init__
    factory.preparer.prepare_installed_requirement(self._ireq, skip_reason)
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 656, in prepare_installed_requirement
    logger.info(
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 1489, in info
    self._log(INFO, msg, args, **kwargs)
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 1634, in _log
    self.handle(record)
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 1644, in handle
    self.callHandlers(record)
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 1706, in callHandlers
    hdlr.handle(record)
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 978, in handle
    self.emit(record)
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/utils/logging.py", line 168, in emit
    message = self.format(record)
              ^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 953, in format
    return fmt.format(record)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/utils/logging.py", line 112, in format
    formatted = super().format(record)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 687, in format
    record.message = record.getMessage()
                     ^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/.pyenv/versions/3.11.1/lib/python3.11/logging/__init__.py", line 377, in getMessage
    msg = msg % self.args
          ~~~~^~~~~~~~~~~
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 198, in __str__
    s += " in {}".format(display_path(self.satisfied_by.location))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/danielkono/personal/python/pip-none-path-example/venv/lib/python3.11/site-packages/pip/_internal/utils/misc.py", line 154, in display_path
    path = os.path.normcase(os.path.abspath(path))
                            ^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 399, in abspath
TypeError: expected str, bytes or os.PathLike object, not NoneType

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

Definitely a bug. Pull request would be most welcomed. I am curious why Mypy does not already catch this.

[daniil-konovalenko]

I am curious why Mypy does not already catch this.

That's a good question, I'll try to look into it while working on the PR

[daniil-konovalenko]

mypy seems to think it's an str🤔

        if self.satisfied_by is not None:
            reveal_type(self.satisfied_by.location)
            ...

Mypy output:

src/pip/_internal/req/req_install.py:198: note: Revealed type is "builtins.str"

[daniil-konovalenko]

Okay, found why mypy didn't catch it:

pip/src/pip/_internal/req/req_install.py

Line 2 in a057c9b

# mypy: strict-optional=False

[daniil-konovalenko]

Okay, found why mypy didn't catch it:

pip/src/pip/_internal/req/req_install.py

Line 2 in a057c9b

# mypy: strict-optional=False

A relevant PR that enables this check: #11379