Add data structure, parsing and serialisation for direct_url.json

[pradyunsg]

x-ref https://discuss.python.org/t/place-for-a-direct-url-json-parser/26266
Corresponding code in pip: https://github.com/pypa/pip/blob/23.1.2/src/pip/_internal/models/direct_url.py

[brettcannon]

https://packaging.python.org/en/latest/specifications/direct-url-data-structure/

Do we just want a TypedDict for this? Or are you after something fancier?

[ofek]

Would that way support conditional logic? Just asking because I'm unfamiliar with the limitations of typing

[brettcannon]

Would that way support conditional logic? Just asking because I'm unfamiliar with the limitations of typing

What condition are you explicitly trying to model in the data?

[ofek]

I mean that the data is contingent upon what the URL field is

[brettcannon]

I mean that the data is contingent upon what the URL field is

If it's all self-contained and can be determined by a literal, then yes. E.g. {vsc: "git", ...} versus {vcs:"hg", ...} can have different keys. Then type for any key that could be either would be the union and the type checkers would determine what you're working with by seeing if you checked if the literal value was used.

[ofek]

I think it's more dynamic and based on the top level url key as seen here https://packaging.python.org/en/latest/specifications/direct-url-data-structure/#examples

[brettcannon]

I think it's more dynamic and based on the top level url key as seen here https://packaging.python.org/en/latest/specifications/direct-url-data-structure/#examples

Ah, there might be some way to use Annotated, but I'm not sure.

[pradyunsg]

One concern I have with a TypedDict model is that it is non-trivial to validate untrusted data against it, without taking on additional dependencies or having a bunch of extra logic. We can definitely do that tho, if the type system is capable!

If this is something that we want to design for adoption in installers¹, IMO we should have the ability to error out with a decent error message about malformed variants of this file (stuff the existing pip logic raises DirectUrlValidationError for).

Footnotes

1. which is the tooling that generates and parses this information, but I guess someone generating an SBOM might want this too. ↩