BUG: pip install -e . dinged for not using hashes, but it can't

[nedbat]

Describe the bug

My project (https://deps.dev/pypi/coverage) gets a warning about pinned dependencies:

Warn: pipCommand not pinned by hash: .github/workflows/coverage.yml:114

The line in question is:

python -m pip install -e .

I think scorecard is looking for the --require-hashes option, but it's not allowed with -e. This is what happens when I try:

+ python -m pip install --require-hashes -e .
Obtaining file:///home/runner/work/coveragepy/coveragepy
ERROR: The editable requirement file:///home/runner/work/coveragepy/coveragepy cannot be installed when requiring hashes, because there is no single file to hash.

[laurentsimon]

Thanks for the report. So I think the fix is for scorecard to verify that no -e local/path is used in the command. Is that correct?

-e also seems to accept a remote URL as input, which is harder to verify.

Do you think ignoring -e local/path is enough?

[nedbat]

Definitely ignoring -e local/path will help. Why not also allow a URL?

[laurentsimon]

I don't know how the URL works. Does pip fetch the URL and search for a requirement.txt? Or does it simply take the URL as being the source code? Is the source code guaranteed to be immutable (which is the property we're looking for)?

[nedbat]

The local file path or the URL are the same: they are a place to get an installable directory of files.

A difference is that the URL could include a SHA that would make it immutable. So perhaps the rule should be a local file path, or a URL with a SHA?

[laurentsimon]

Yes that would work. Can you provide an example of URL with a SHA? What is the URL format?

[nedbat]

This seems to be the best docs: https://pip.pypa.io/en/stable/topics/vcs-support/

Some syntaxes to support (shell quoting may or may not be needed, and of course -e is possible):

pip install "MyProject @ git+https://git.example.com/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709"
pip install "MyProject[extra1,extra2]@git+https://git.example.com/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709"
pip install git+https://git.example.com/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=MyProject
pip install git+https://git.example.com/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=MyProject==1.2.3
pip install "git+https://git.example.com/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=MyProject[extra1,extra2]==1.2.3"

Are shortened hashes OK?

pip install git+https://git.example.com/MyProject.git@da39a3ee5e#egg=MyProject

[nedbat]

Maybe the simplest solution is to match against @[a-fA-F0-9]{40} and forget the rest of the syntax possibilities?

[laurentsimon]

Thanks for the link!
I think you're right, the regex should be enough. Maybe we also need to match on http(s):// or ssh:// or sftp:// to be sure it's a URL and not a path. Wdut?

[nedbat]

Sure, that couldn't hurt. Ping me on the change, I would like to understand better how this code works.

[laurentsimon]

Actually if you want to take a shot at it, feel free to try it out. Code to change is https://github.com/ossf/scorecard/blob/main/checks/raw/shell_download_validate.go#L469

You can add tests in https://github.com/ossf/scorecard/blob/main/checks/raw/pinned_dependencies_test.go#L550, https://github.com/ossf/scorecard/blob/main/checks/raw/pinned_dependencies_test.go#L710 https://github.com/ossf/scorecard/blob/main/checks/raw/pinned_dependencies_test.go#L813
https://github.com/ossf/scorecard/blob/main/checks/raw/pinned_dependencies_test.go#L855
https://github.com/ossf/scorecard/blob/main/checks/raw/pinned_dependencies_test.go#L935
https://github.com/ossf/scorecard/blob/main/checks/raw/pinned_dependencies_test.go#L983

[nedbat]

Thanks, but I've never written Go, so you should not wait for a pull request from me... :)