Reduce the size of vendored dependencies for Rust applications

[rucoder]

When using cargo vendor a lot of unused dependencies are vendored. e.g. all crates related to Windows even though we are interested only in linux platform. The solution is to use https://github.com/coreos/cargo-vendor-filterer as following

cargo vendor-filterer --platform=x86_64-unknown-linux-musl --platform=aarch64-unknown-linux-musl --platform=riscv64gc-unknown-linux-gnu --format tar.zstd

and later uncompress the archive in a Dockerfile. To make it clear: we do not need to use --format. in this case we keep ./vendor folder and the size will still be much smaller. e.g. in case of pkg/installer the size is reduced from 318Mb to 56Mb. If we use compressed ourput we reduce size to 6.6Mb and a single file

[deitch]

Given that all of this is reproducible, like with go, I'd like to see a way to avoid having to commit these to git at all. Maybe we can vendor them but exclude the vendor dir, but require local build to run it?

The one thing we cannot do is enable networking in the container, because that messes up the ability to do reproducible builds and SBoMs, as anything can run.

[rucoder]

Given that all of this is reproducible, like with go, I'd like to see a way to avoid having to commit these to git at all. Maybe we can vendor them but exclude the vendor dir, but require local build to run it?

The one thing we cannot do is enable networking in the container, because that messes up the ability to do reproducible builds and SBoMs, as anything can run.

what is the difference between vendoring at build time and letting cargo to download crate sources itself?

[deitch]

what is the difference between vendoring at build time and letting cargo to download crate sources itself?

We need the build process to always give the exact same results, and even more importantly, know exactly what is included.

When we run the build with network: yes, any program inside can download anything from anywhere. We won't know it, and cannot control it. Unfortunately, build time cargo means an arbitrary program, so it cannot run inside the automated build process. When network: no, the ADD command works, because it is controllable and auditable.

The ideal would be some kind of acceptable download from cargo that works when network: no.