Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: pip-specific environment variable expansion in requirements and constraints files #8828

Open
uranusjr opened this issue Aug 31, 2020 · 2 comments
Labels
C: requirement file Using `requirements.txt` type: enhancement Improvements to functionality

Comments

@uranusjr
Copy link
Member

uranusjr commented Aug 31, 2020

What's the problem this feature will solve?
PEP 508 URL specification has been proposed (partly) as the replacement to dependecy_links in setup.py. This however causes problems for artifact vendoring since the file:// protocol does not provide a way to represent relative paths and a project’s requirements.txt must hard-code the project root, or use the pip-specific path format, which is not always viable.

Describe the solution you'd like
The proposal is to introduce pip-specific environment variables that are populated only during the parsing phase of requirements and constraints files, so users can use them in PEP 508 URL specs. These environment variables should have the prefix PIP__ (two underscores to avoid conflicts with the command argument variables), and are guarenteed to use the POSIX path representation (i.e. forward slashes, even on Windows). Two such environment variables are proposed for initial addition:

  • PIP__WORKING_DIRECTORY expands to the direcotry where the pip command is run in, i.e. Path.cwd(). This is the same as the POSIX PWD, but proposed for platform-independent projects.
  • PIP__CURRENT_FILE expands to the requirements or constraints file that contains the requirement line. This makes it easy to e.g. put the requirements.txt and vendored artifacts in the same directory, and refer to an artifact with package @ file://${PIP__CURRENT_FILE}/../package-1.0-py3-none-any.whl.

Alternative Solutions

  • Allow paths in PEP 508. I don’t really like this (filesystem path is a hairy thing to specify in a cross-platform way), and have a feeling others may share the same view.
  • Propose a pip wrapper that automatically generates a requirements.txt from a template. But requirements.txt is underspecified and difficult for a third-party tool to support. Ican develop for only the scenarios I personally need, but it is difficult to suggest it to other people.

Additional context
This would also serve as a solution to #8765. Users can use these environment variables instead of relative paths to avoid the quirky resolution logic. We can even consider deprecating relative paths altogether if this gets adopted well.

@uranusjr uranusjr added C: requirement file Using `requirements.txt` type: enhancement Improvements to functionality labels Aug 31, 2020
@sbidoul
Copy link
Member

sbidoul commented Sep 4, 2020

I may miss something but it looks like relative URLs, if implemented properly as described in #8765, would do the job?

Relative URLs is a fairly well understood concept (in html at least) which fits quite naturally with requirement files.

Of course it is yet another transition to manage, but I'd say it's not worse than deprecating relative requirements.

@uranusjr
Copy link
Member Author

uranusjr commented Sep 4, 2020

relative URLs, if implemented properly as described in #8765, would do the job?

It would do one half of the job. The main problem I have with #8765 is that no matter which logic we choose, people depending on the other resolution logic can no longer do what they’re doing, and I can see both ways being useful in certain workflows. We probably should unify the relative path resolution logic for consistency, but IMO we should also offer an alternative if we do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: requirement file Using `requirements.txt` type: enhancement Improvements to functionality
Projects
None yet
Development

No branches or pull requests

2 participants