Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provenance url #3

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions AUTHORS.txt
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ Bogdan Opanchuk
BorisZZZ
Brad Erickson
Bradley Ayers
Branch Vincent
Brandon L. Reiss
Brandt Bucher
Brannon Dorsey
Expand All @@ -131,11 +132,13 @@ Carol Willing
Carter Thayer
Cass
Chandrasekhar Atina
Charlie Marsh
Chih-Hsuan Yen
Chris Brinker
Chris Hunt
Chris Jerdonek
Chris Kuehl
Chris Markiewicz
Chris McDonough
Chris Pawley
Chris Pryer
Expand Down Expand Up @@ -234,6 +237,7 @@ Dos Moonen
Douglas Thor
DrFeathers
Dustin Ingram
Dustin Rodrigues
Dwayne Bailey
Ed Morley
Edgar Ramírez
Expand Down Expand Up @@ -365,12 +369,14 @@ Jeff Dairiki
Jeff Widman
Jelmer Vernooij
jenix21
Jeremy Fleischman
Jeremy Stanley
Jeremy Zafran
Jesse Rittner
Jiashuo Li
Jim Fisher
Jim Garrison
Jinzhe Zeng
Jiun Bae
Jivan Amara
Joe Bylund
Expand All @@ -391,6 +397,7 @@ Jorge Niedbalski
Joseph Bylund
Joseph Long
Josh Bronson
Josh Cannon
Josh Hansen
Josh Schneier
Joshua
Expand Down Expand Up @@ -425,6 +432,7 @@ konstin
kpinc
Krishna Oza
Kumar McMillan
Kuntal Majumder
Kurt McKee
Kyle Persohn
lakshmanaram
Expand Down Expand Up @@ -513,6 +521,7 @@ Miro Hrončok
Monica Baluna
montefra
Monty Taylor
morotti
mrKazzila
Muha Ajjan
Nadav Wexler
Expand Down Expand Up @@ -625,6 +634,7 @@ Richard Jones
Richard Si
Ricky Ng-Adam
Rishi
rmorotti
RobberPhex
Robert Collins
Robert McGibbon
Expand Down Expand Up @@ -700,6 +710,7 @@ Stéphane Klein
Sumana Harihareswara
Surbhi Sharma
Sviatoslav Sydorenko
Sviatoslav Sydorenko (Святослав Сидоренко)
Swat009
Sylvain
Takayuki SHIMIZUKAWA
Expand Down
73 changes: 73 additions & 0 deletions NEWS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,79 @@

.. towncrier release notes start

24.2 (2024-07-28)
=================

Deprecations and Removals
-------------------------

- Deprecate ``pip install --editable`` falling back to ``setup.py develop``
when using a setuptools version that does not support :pep:`660`
(setuptools v63 and older). (`#11457 <https://github.com/pypa/pip/issues/11457>`_)

Features
--------

- Check unsupported packages for the current platform. (`#11054 <https://github.com/pypa/pip/issues/11054>`_)
- Use system certificates *and* certifi certificates to verify HTTPS connections on Python 3.10+.
Python 3.9 and earlier only use certifi.

To revert to previous behaviour, pass the flag ``--use-deprecated=legacy-certs``. (`#11647 <https://github.com/pypa/pip/issues/11647>`_)
- Improve discovery performance of installed packages when the ``importlib.metadata``
backend is used to load distribution metadata (used by default under Python 3.11+). (`#12656 <https://github.com/pypa/pip/issues/12656>`_)
- Improve performance when the same requirement string appears many times during
resolution, by consistently caching the parsed requirement string. (`#12663 <https://github.com/pypa/pip/issues/12663>`_)
- Minor performance improvement of finding applicable package candidates by not
repeatedly calculating their versions (`#12664 <https://github.com/pypa/pip/issues/12664>`_)
- Disable pip's self version check when invoking a pip subprocess to install
PEP 517 build requirements. (`#12683 <https://github.com/pypa/pip/issues/12683>`_)
- Improve dependency resolution performance by caching platform compatibility
tags during wheel cache lookup. (`#12712 <https://github.com/pypa/pip/issues/12712>`_)
- ``wheel`` is no longer explicitly listed as a build dependency of ``pip``.
``setuptools`` injects this dependency in the ``get_requires_for_build_wheel()``
hook and no longer needs it on newer versions. (`#12728 <https://github.com/pypa/pip/issues/12728>`_)
- Ignore ``--require-virtualenv`` for ``pip check`` and ``pip freeze`` (`#12842 <https://github.com/pypa/pip/issues/12842>`_)
- Improve package download and install performance.

Increase chunk sizes when downloading (256 kB, up from 10 kB) and reading files (1 MB, up from 8 kB).
This reduces the frequency of updates to pip's progress bar. (`#12810 <https://github.com/pypa/pip/issues/12810>`_)
- Improve pip install performance.

Files are now extracted in 1MB blocks, or in one block matching the file size for
smaller files. A decompressor is no longer instantiated when extracting 0 bytes files,
it is not necessary because there is no data to decompress. (`#12803 <https://github.com/pypa/pip/issues/12803>`_)

Bug Fixes
---------

- Set ``no_color`` to global ``rich.Console`` instance. (`#11045 <https://github.com/pypa/pip/issues/11045>`_)
- Fix resolution to respect ``--python-version`` when checking ``Requires-Python``. (`#12216 <https://github.com/pypa/pip/issues/12216>`_)
- Perform hash comparisons in a case-insensitive manner. (`#12680 <https://github.com/pypa/pip/issues/12680>`_)
- Avoid ``dlopen`` failure for glibc detection in musl builds (`#12716 <https://github.com/pypa/pip/issues/12716>`_)
- Avoid keyring logging crashes when pip is run in verbose mode. (`#12751 <https://github.com/pypa/pip/issues/12751>`_)
- Fix finding hardlink targets in tar files with an ignored top-level directory. (`#12781 <https://github.com/pypa/pip/issues/12781>`_)
- Improve pip install performance by only creating required parent
directories once, instead of before extracting every file in the wheel. (`#12782 <https://github.com/pypa/pip/issues/12782>`_)
- Improve pip install performance by calculating installed packages printout
in linear time instead of quadratic time. (`#12791 <https://github.com/pypa/pip/issues/12791>`_)

Vendored Libraries
------------------

- Remove vendored tenacity.
- Update the preload list for the ``DEBUNDLED`` case, to replace ``pep517`` that has been renamed to ``pyproject_hooks``.
- Use tomllib from the stdlib if available, rather than tomli
- Upgrade certifi to 2024.7.4
- Upgrade platformdirs to 4.2.2
- Upgrade pygments to 2.18.0
- Upgrade setuptools to 70.3.0
- Upgrade typing_extensions to 4.12.2

Improved Documentation
----------------------

- Correct ``—-ignore-conflicts`` (including an em dash) to ``--ignore-conflicts``. (`#12851 <https://github.com/pypa/pip/issues/12851>`_)

24.1.2 (2024-07-07)
===================

Expand Down
1 change: 0 additions & 1 deletion news/10822.vendor.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/11045.bugfix.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/11054.feature.rst

This file was deleted.

3 changes: 0 additions & 3 deletions news/11457.removal.rst

This file was deleted.

4 changes: 0 additions & 4 deletions news/11647.feature.rst

This file was deleted.

1 change: 1 addition & 0 deletions news/11865.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Implement PEP-710 for storing provenance_url.json file.
1 change: 0 additions & 1 deletion news/12216.bugfix.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12572.trivial.rst

This file was deleted.

3 changes: 0 additions & 3 deletions news/12656.feature.rst

This file was deleted.

2 changes: 0 additions & 2 deletions news/12660.trivial.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12663.feature.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12664.feature.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12680.bugfix.rst

This file was deleted.

2 changes: 0 additions & 2 deletions news/12683.feature.rst

This file was deleted.

2 changes: 0 additions & 2 deletions news/12712.feature.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12716.bugfix.rst

This file was deleted.

5 changes: 0 additions & 5 deletions news/12728.feature.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12751.bugfix.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12776.trivial.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12781.bugfix.rst

This file was deleted.

2 changes: 0 additions & 2 deletions news/12782.bugfix.rst

This file was deleted.

2 changes: 0 additions & 2 deletions news/12791.bugfix.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12796.vendor.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12797.vendor.rst

This file was deleted.

4 changes: 0 additions & 4 deletions news/12803.bugfix.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12805.trivial.rst

This file was deleted.

5 changes: 0 additions & 5 deletions news/12810.feature.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12842.feature.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/12851.doc.rst

This file was deleted.

Empty file.
Empty file.
Empty file.
Empty file.
1 change: 0 additions & 1 deletion news/certifi.vendor.rst

This file was deleted.

Empty file.
1 change: 0 additions & 1 deletion news/platformdirs.vendor.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/pygments.vendor.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/setuptools.vendor.rst

This file was deleted.

1 change: 0 additions & 1 deletion news/typing_extensions.vendor.rst

This file was deleted.

2 changes: 1 addition & 1 deletion src/pip/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from typing import List, Optional

__version__ = "24.2.dev0"
__version__ = "24.3.dev0"


def main(args: Optional[List[str]] = None) -> int:
Expand Down
16 changes: 12 additions & 4 deletions src/pip/_internal/models/direct_url.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
T = TypeVar("T")

DIRECT_URL_METADATA_NAME = "direct_url.json"
PROVENANCE_URL_METADATA_NAME = "provenance_url.json"
ENV_VAR_RE = re.compile(r"^\$\{[A-Za-z0-9-_]+\}(:\$\{[A-Za-z0-9-_]+\})?$")


Expand Down Expand Up @@ -205,20 +206,27 @@ def from_dict(cls, d: Dict[str, Any]) -> "DirectUrl":
),
)

def to_dict(self) -> Dict[str, Any]:
def to_dict(self, *, keep_legacy_hash_key: bool = True) -> Dict[str, Any]:
res = _filter_none(
url=self.redacted_url,
subdirectory=self.subdirectory,
)
res[self.info.name] = self.info._to_dict()

info_dict = self.info._to_dict()
if not keep_legacy_hash_key:
info_dict.pop("hash", None)

res[self.info.name] = info_dict
return res

@classmethod
def from_json(cls, s: str) -> "DirectUrl":
return cls.from_dict(json.loads(s))

def to_json(self) -> str:
return json.dumps(self.to_dict(), sort_keys=True)
def to_json(self, *, keep_legacy_hash_key: bool = True) -> str:
return json.dumps(
self.to_dict(keep_legacy_hash_key=keep_legacy_hash_key), sort_keys=True
)

def is_local_editable(self) -> bool:
return isinstance(self.info, DirInfo) and self.info.editable
35 changes: 28 additions & 7 deletions src/pip/_internal/operations/install/wheel.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,12 @@
FilesystemWheel,
get_wheel_distribution,
)
from pip._internal.models.direct_url import DIRECT_URL_METADATA_NAME, DirectUrl
from pip._internal.models.direct_url import (
DIRECT_URL_METADATA_NAME,
PROVENANCE_URL_METADATA_NAME,
ArchiveInfo,
DirectUrl,
)
from pip._internal.models.scheme import SCHEME_KEYS, Scheme
from pip._internal.utils.filesystem import adjacent_tmp_file, replace
from pip._internal.utils.misc import StreamWrapper, ensure_dir, hash_file, partition
Expand Down Expand Up @@ -424,9 +429,10 @@ def _install_wheel( # noqa: C901, PLR0915 function is too long
wheel_zip: ZipFile,
wheel_path: str,
scheme: Scheme,
download_info: DirectUrl,
is_direct: bool,
pycompile: bool = True,
warn_script_location: bool = True,
direct_url: Optional[DirectUrl] = None,
requested: bool = False,
) -> None:
"""Install a wheel.
Expand Down Expand Up @@ -673,12 +679,25 @@ def _generate_file(path: str, **kwargs: Any) -> Generator[BinaryIO, None, None]:
installer_file.write(b"pip\n")
generated.append(installer_path)

# Record the PEP 610 direct URL reference
if direct_url is not None:
if is_direct:
# Record the PEP 610 direct URL reference
direct_url_path = os.path.join(dest_info_dir, DIRECT_URL_METADATA_NAME)
with _generate_file(direct_url_path) as direct_url_file:
direct_url_file.write(direct_url.to_json().encode("utf-8"))
direct_url_file.write(download_info.to_json().encode("utf-8"))
generated.append(direct_url_path)
else:
# Record the PEP 710 provenance URL reference only if we have hashes for
# the given wheel. They can be missing when wheels are built using an old pip.
assert isinstance(download_info.info, ArchiveInfo)
if download_info.info.hashes:
provenance_url_path = os.path.join(
dest_info_dir, PROVENANCE_URL_METADATA_NAME
)
with _generate_file(provenance_url_path) as provenance_url_file:
provenance_url_file.write(
download_info.to_json(keep_legacy_hash_key=False).encode("utf-8")
)
generated.append(provenance_url_path)

# Record the REQUESTED file
if requested:
Expand Down Expand Up @@ -721,10 +740,11 @@ def install_wheel(
name: str,
wheel_path: str,
scheme: Scheme,
download_info: DirectUrl,
is_direct: bool,
req_description: str,
pycompile: bool = True,
warn_script_location: bool = True,
direct_url: Optional[DirectUrl] = None,
requested: bool = False,
) -> None:
with ZipFile(wheel_path, allowZip64=True) as z:
Expand All @@ -734,8 +754,9 @@ def install_wheel(
wheel_zip=z,
wheel_path=wheel_path,
scheme=scheme,
download_info=download_info,
is_direct=is_direct,
pycompile=pycompile,
warn_script_location=warn_script_location,
direct_url=direct_url,
requested=requested,
)
Loading
Loading