Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support VCS dependencies #202

Closed
charliermarsh opened this issue Oct 26, 2023 · 2 comments · Fixed by #283
Closed

Support VCS dependencies #202

charliermarsh opened this issue Oct 26, 2023 · 2 comments · Fixed by #283
Assignees
Labels
enhancement New feature or improvement to existing functionality wish Not on the immediate roadmap

Comments

@charliermarsh
Copy link
Member

No description provided.

@charliermarsh charliermarsh added enhancement New feature or improvement to existing functionality wish Not on the immediate roadmap labels Oct 26, 2023
@charliermarsh charliermarsh added this to the Feature complete milestone Oct 26, 2023
@konstin
Copy link
Member

konstin commented Oct 26, 2023

Monotrail implementation: https://github.com/konstin/poc-monotrail/blob/10730ea1a84c58af6b35fb74c89ed0578ab042b6/crates/monotrail/src/install.rs#L380-L411

For monotrail the openssl linked by libgit2 was a huge pain to build and deploy.

If we can, i'd like to use https://docs.rs/gix/latest/gix/, it's native rust, supposed to be faster, has a rustls option and is stable enough that it's used by cargo (e.g. rust-lang/cargo#11448)

@charliermarsh
Copy link
Member Author

Awesome, thank you.

@charliermarsh charliermarsh removed the wish Not on the immediate roadmap label Oct 29, 2023
@charliermarsh charliermarsh self-assigned this Oct 29, 2023
@charliermarsh charliermarsh added the wish Not on the immediate roadmap label Oct 29, 2023
charliermarsh added a commit that referenced this issue Nov 2, 2023
## Summary

This PR adds support for Git dependencies, like:

```
flask @ git+https://github.com/pallets/flask.git
```

Right now, they're only supported in the resolver (and not the
installer), since the installer doesn't yet support source distributions
at all.

The general approach here is based on Cargo's Git implementation.
Specifically, I adapted Cargo's
[`git`](https://github.com/rust-lang/cargo/blob/23eb492cf920ce051abfc56bbaf838514dc8365c/src/cargo/sources/git/mod.rs)
module to perform the cloning, which is based on `libgit2`.

As compared to Cargo's implementation, I made the following changes:

- Removed any unnecessary code.
- Fixed any Clippy errors for our stricter ruleset.
- Removed the dependency on `curl`, in favor of `reqwest` which we use
elsewhere.
- Removed the ability to use `gix`. Cargo allows the use of `gix` as an
experimental flag, but it only supports a small subset of the
operations. When Cargo fully adopts `gix`, we should plan to do the
same.
- Removed Cargo's host key checking. We need to re-add this! I'll do it
shortly.
- Removed Cargo's progress bars. We should re-add this too, but we use
`indicatif` and Cargo had their own thing.

There are a few follow-ups to consider:

- Adding support in the installer.
- When we lock, we should write out the Git URL that includes the exact
SHA. This lets us cache in perpetuity and avoids dependencies changing
without re-locking.
- When we resolve, we should _always_ try to refresh Git dependencies.
(Right now, we skip if the wheel was already built.)

I'll work on the latter two in follow-up PRs.

Closes #202.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement to existing functionality wish Not on the immediate roadmap
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants