Make hashes less dependent on source and build paths

[mbitsnbites]

Currently I was not able to properly use a shared cache between two machines without them using the exact same source and build folders, which seems quite unnecessary.

Apparently (according to #4 ), there are already some thoughts for how to fix this?

My gut feeling is that since the preprocessed result is what's being hashed (together with the command line flags that affect how the compilation is performed), things like file names (input, output) and include paths should not be important at all, so you should be able to remove them from the hash completely, right? (this includes -I, -o, -MT, -MF, -include etc, plus the source file name)

[mbitsnbites]

Forget everything I've said about compiler flags and paths. I did some debugging of generate_hash_key(), and parsed_args.common_args seems to be very clean, containing only flags that affect the post-preprocessing compilation step.

Same thing for the environment flags, where only MACOSX_DEPLOYMENT_TARGET and IPHONEOS_DEPLOYMENT_TARGET affect the hash.

However, the preprocessor output contains lots of paths, at least for gcc. These are typically of the form:

# 1 "/home/alice/git/coolproject/modules/foo/src/cool_source.cpp"
# 1 "/home/alice/git/coolproject-build/Release//"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "/home/alice/git/coolproject/modules/foo/src/cool_source.cpp"
...

A relatively simple solution (that hopefully works) would be to strip the preprocessor output of all lines starting with # .

Edit: Or simply pass -P (Inhibit generation of linemarkers in the output from the preprocessor) to cpp, which seems to do the trick.

[mbitsnbites]

PR #208 addresses this issue for GCC (and hopefully Clang).

I'm not sure if the corresponding problem exists for MSVC.

[mqudsi]

Does this issue affect other languages like rust?

[luser]

Does this issue affect other languages like rust?

It does! #196 has some discussion. There's a PR with a potential fix for Rust, but it'll have the same problem as C compilations (paths in debug info will be wrong), and I'm not sure it'll work properly even with that fix, since it looks like the build directory winds up in one of the internal rlib hashes, which might make rustc unhappy.

[mqudsi]

Thank you for that kind reply. I think cargo could structure things differently on its end to remove some unnecessary duplication and simultaneously make sccache's life easier; I had originally proposed some sort of fs-local cache on cargo's end in a github issue and that's when I was first informed about sccache and the difficult problems it was trying to solve :)

[daniel-sherwood]

Hi - I have just upgraded to a build including these fixes and it seems to work with two directories on the same machine, however I am still getting 100% cache misses when running on a different machine. Can you suggest how I go about debugging this to establish the cause of the cache misses?