PEP 691: JSON-based simple index API interaction

[dstufft]

No description provided.

[uranusjr]

Makes sense to me from a quick read.

[pfmoore]

LGTM. It's nice how simple this change is as well 🙂

[dstufft]

I did just test this with EpicWink/proxpi#8 as well, since that's easier to use against live PyPI data instead of Warehouse's development data. It was using JSON and it worked like a charm.

[dstufft]

Note: I think this is slightly bugged, it's dropping the hash data on the floor, as I think the Link class depends on it being in the URL.

[dstufft]

Testing this:

$ pip -vvv download requests --no-deps
...
Fetched page https://pypi.org/simple/requests/ as application/vnd.pypi.simple.v1+json
...

[dstufft]

I'm not going to merge this myself, as I don't feel confident in my knowledge of the code base anymore to give myself the 👍, but I believe this should be good to go now if someone wants to review it.

[pfmoore]

I've only desk-checked this but it LGTM. I'll leave it for now in case anyone else has any comments, but if no-one else comes up with any objections, I'll merge it.

[pradyunsg]

Looks like the PyPI implementation landed, so it should be possibel to use that to try this out. :)

[fschulze]

I added PEP 691 support in devpi-server, both for fetching from PyPI and serving installers.

The instance at https://m.devpi.net/ runs the 6.6.0 development version with the added support. For testing with pip you could use -i https://m.devpi.net/root/pypi/+simple/.

An added benefit of the Accept header approach is easy and performant support for other forms of the URL without the trailing slash or the +simple, before we did User-Agent based detection for installers to prevent redirects or serving the devpi-web interface.

You can get the wheels of the development releases for local testing here:
https://m.devpi.net/fschulze/dev/+f/220/2b013a4eadc7f/devpi_common-3.6.1.dev0-py2.py3-none-any.whl#sha256=2202b013a4eadc7f0d41528058db9416c8eab7ddd552fe032c6a4ddfd80c5b5b
https://m.devpi.net/fschulze/dev/+f/a65/b973cb95dd00c/devpi_server-6.6.0.dev0-py3-none-any.whl

[dstufft]

Seems like nobody seems to have any concerns, I think this should be good to land now?

[sbidoul]

LGTM (code review only, no manual test).

Since this will switch to using the new API without transition, is there anything particular to mention in the release notes ?

[sbidoul]

@dstufft I plan to release 22.2 in a week or so. Will you have time to look at the comments above by then -- I'm aware you are super busy on the PyPI side - huges thanks for that 🙏 ❤️ -- ?

[dstufft]

I missed those comments, yea I can address those.

[sbidoul]

Thanks, @dstufft ! Do you see anything we (or users) need to pay attention to on release day ?

[dstufft]

At the risk of jinxing things, this should be entirely transparent.

[notatallshaw]

At the risk of jinxing things, this should be entirely transparent.

I think my only concern would be because you are changing the accept headers it's possible that the software that is being used for workflows inside organizations that are using private repos might give unexpected results.

Unfortunately I no longer work for such an organization otherwise I'd be happy to test it. But something to keep an eye out during the release process.

[fschulze]

@notatallshaw any default configuration should handle the accept headers just fine. With devpi-server it definitely works.

[sbidoul]

We could imagine an index doing a non-standard compliant parsing of the Accept header to test for text/html.

Anyone knows if we can give them a heads up ?

In the meantime I'm going ahead and merge this to move forward in the release process.