Rust and incremental compilation

[alexcrichton]

With the advent of incremental compilation on stable Rust compile times will hopefully get much better especially in debug mode and hopefully eventually in release mode with rust-lang/rust#48996. Ideally sccache would help to integrate with incremental compilation to even further accelerate CI and builds whenever possible!

Unfortunately it's not super clear how sccache would integrate with incremental compilation in Rust. There will never be an exact cache hit as sccache already otherwise does that, so we sort of need an "almost cache hit" to load something from the cache. One possible idea may be to hash fewer things and upload full compiler incremental cache directories. That way sccache would download the previous cache directory and let rustc take care of what should be reset.

In any case this isn't super pressing for now as incremental for release builds is still aways out, but figured I'd open an issue!

[luser]

I had asked about this a while back when I was doing the Rust work initially but I think incremental compilation was still new enough that I couldn't get a useful answer. It would definitely be interesting to figure out a smart plan here. Maybe the simplest thing we could do would be to store the incremental compilation outputs in the cache entry, so that if you get a full cache hit you can at least do an incremental compile on top of that if you make changes?

You and @michaelwoerister and I should find time at the SF All Hands in June to chat about this.

[alexcrichton]

Maybe the simplest thing we could do would be to store the incremental compilation outputs in the cache entry, so that if you get a full cache hit you can at least do an incremental compile on top of that if you make changes?

For sure! That I think sounds like a great idea and is quite possible to do with -C incremental=.. I think. (in that sccache would recognize such an argument, parse it, and slurp it up)

Unfortunately though I won't actually be at the SF All Hands, but @michaelwoerister if you've got a spare moment in the near future I'd love to chat with you about your thoughts on this :)

[michaelwoerister]

Maybe the simplest thing we could do would be to store the incremental compilation outputs in the cache entry, so that if you get a full cache hit you can at least do an incremental compile on top of that if you make changes?

Yes, that should work. It will at least double the amount of data to download though. We'd also need to review if there's anything non-portable in the cache files (i.e. local file paths or something).

[luser]

Yes, that should work. It will at least double the amount of data to download though. We'd also need to review if there's anything non-portable in the cache files (i.e. local file paths or something).

sccache doesn't currently get cache hits if file paths change (#196), so this isn't any worse than the current situation.

[jakelee8]

@Xuanwo What do you think about supporting Rust incremental builds or disabling incremental compilation (by default or only for crate libraries)?

Caveat docs:

1. https://github.com/mozilla/sccache#rust
2. https://github.com/mozilla/sccache/blob/main/docs/Rust.md

[WieeRd]

I'm a little confused about this caveat, does this mean I need to disable incremental build completely and globally via build.incremental option in ~/.cargo/config.toml if I want to benefit from cached builds?
My primary reason for using sccache is to share 3rd party dependency build caches between projects on my desktop.