-
-
Notifications
You must be signed in to change notification settings - Fork 241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider using cargo binstall
for rust crates
#2382
Comments
Depending on your answers, we might still benefit from setting up binstall right now even before the packages are available in a release. |
It all sounds reasonable to me, but I don't have any Rust experience. |
I also noticed than rust stuff takes soooooooo much time, if you make your solution work, I'll gladly merge the PR <3 |
Even before my two Sunday PRs were merged I was on that :) in the beginning of that day I had a solution imagined for time taken for tests, I'm not sure yet if it's adequate yet. |
My initial issue, before finishing writing it, was specifically to ask about the two flags used in cargo install, why are they there? |
Me neither, but I started to learn a bit about it. That's why I like working on MegaLinter, it feeds my curiosity 😄 |
@echoix same here, two months ago I didn't even know how to do a Hello World in Python 🤣. |
I learned python especially to build MegaLinter, and I never heard about a quarter of everything that we can lint 🤣 |
I was inspired to read the |
At that point I'd rather the build to fail in order to look at it, if three installation methods, including the one that should work, fails. |
That makes sense to me for |
I would have to retry, but I'm not sure |
To clarify, I was imagining that Cargo might have a config file where one can specify the equivalent setting. |
Oh, what a suprise, 11 hours ago a PR was merged to the cargo-binstall repo, especially about passing the --locked flag to the cargo install command if used: cargo-bins/cargo-binstall#830 |
This issue has been automatically marked as stale because it has not had recent activity. If you think this issue should stay open, please remove the |
This issue has been automatically marked as stale because it has not had recent activity. If you think this issue should stay open, please remove the |
In the weekend I played a lot to see how to reduce Rust crates installing time (first time playing around rust compilation).
Current times are 5-6 minutes for 1 linter (clippy is installed with rust in the first 90 secs) and 2 sarif tools used for other linters. Whereas our total build times are between 20-25 min. So 20-25% of the time is spent on these two tools.
I found that
cargo binstall
would help a lot https://github.com/cargo-bins/cargo-binstall. It is not preinstalled, but downloadable (precompiled -> it is optimized better than when usingcargo install cargo-binstall
as per their docs). cargo binstall downloads the binary from the github releases if available for the required target (the triplet). If not, it will use cargo-quickinstall, another project, and will fall back to cargo install, ie: compile from source. It seems supported by multiple projects as I saw, that's nice to see. The download URL format for the crates are specified in the project's Cargo.toml, and this file is the manifest that is used for crates.io. (theres an override possible too). sarif-fmt and shellcheck-sarif are already configured for binstall. However, they only include two binaries in their releases and they do not include our triplets, so they compile from source.The current command that is used to install our two requirements (appart from clippy, that is a component)
I had one conceptual blocker beforehand:
What where the design choices that lead to adding the
--force
and--locked
options? These didn't really work with binstall, and could figure out if they were really important.If using in the dockerfile RUN step and compile from source using this command,
is faster than the current command (docker step finishes in 120-130 sec instead of 320-380 sec on the github instance i was using (up to 16 cores could have been used). In github actions I think I already saw a 8 minutes for the rust step. I think it might be
--force
that makes recompiling some packages. Why is it needed?For the binstall, I had a proof of concept that worked quite well.
I didn't clean up the binstall binary yet.
At the very end of the time spent yesterday I started working on a workflow for the sarif-rs repo to build the tools as static binaries (ie: with musl with triplet
x86_64-unknown-linux-musl
instead of the dynamic shared libraries version using gcc, tripletx86_64-unknown-linux-gnu
). I really didn't finish, I was just starting.The text was updated successfully, but these errors were encountered: