All cache misses after recompilation if using Visual Studio directly for compiling instead of cmake

[dav-otero]

Compling from cmake in Windows actually works quite well and there are lots of cache hits for recompilations.
The tricky thing is that if instead I generate the VS solution and then compile. Then I remove the solution and generate it back again and try to build the solution again from VS, I always get cache misses.

Another case: If I compile with the cmake, then delete the project and recompile with cmake, there are lots of cache hits.
If now I delete the project and generate the visual studio solution, open it and build solution, I get 0 cache hits, all cache misses.

[frerich]

A cache miss occurs in case

1. either the invocation is not supported (e.g. because of unsupported command line arguments like /Zi)
2. or the invocation is supported, but no cached entries was found which matches the hashed information

You can tell which of these two are happening for you by monitoring the cache statistics. Cache misses caused by 1. are going to cause increased numbers in the 'passed to real compiler' section.

In case you're bit by number 2., I suggest setting the CLCACHE_LOG=1 variable to see how clcache does its work. In particular, pay attention to the command lines invoked. It may be that the command lines involve paths to temporary build directories or so, and such directories can change often and thus cause the command line (and hence the hash sum identifying the cached object) to change.

[dav-otero]

What I see is during two consecutive VS builds a lot of evictions:

$ clcache -s
clcache statistics:
current cache dir : C:\dev\clcache\cache
cache size : 1,039,011,563 bytes
maximum cache size : 1,073,741,824 bytes
cache entries : 1014
cache hits : 0
cache misses
total : 623
evicted : 623
header changed : 0
source changed : 0
passed to real compiler
called for linking : 0
called for external debug : 7
called w/o source : 0
called w/ multiple sources : 0
called w/ PCH : 0

[frerich]

@sowsem I'm sorry, I must be doing something wrong. I just can't seem to reproduce this problem here - maybe it's because of my CMake version (2.8.11)?

Would it be possible for you to provide a little sample project (some sort of hello world or so) and then let me know how exactly you're building the problem? In particular, which compiler and which version of 'msbuild' do you use?

I'm confident that once I can reproduce the issue on my end, fixing it shouldn't be very hard.

[dav-otero]

Sorry for the late reply due to vacation period.

It is kinda difficult to me to provide a helloworld sample due to the privacy restrictions I have in that code.

I am using cmake 3.4 and lots of specific cmake macros so a helloworld sample would include lot of stuff probably I am not allowed to share unfortunately.

I can investigate myself and probably to not bother you anymore with this specific issue, you can close it. If I find something more specific, I can come back and open a new issue.

Anyway, thanks a lot for all the help on this! You are amazingly responsive!

[webmaster128]

Can this be caused by VS studio removing cache data in a clean step?

[dav-otero]

But cache data belongs to clcache, doesn't it? No matter what VS is doing, as long as the compiler call is being proxied to clcache and that the command is the same... am I wrong? (and more stuff)

[frerich]

@sowsem in issue #33 someone noticed that Visual Studio would seem to delete more files (in particular, files belonging to clcache) than expected. Some research led to #33 (comment) in which I describe that it appears that Visual Studio apparently traces all file accesses done during a project. So maybe that's what's biting you - in that comment, I also mention how to disable this feature and for the reporter it helped.

[dav-otero]

Cool! I will give it a try! Thx!

[dav-otero]

It has worked like a charm! All cache hits now!

Thanks a lot! This can be closed!

It would be good btw to add this info into the main github page, as many people is using VS as IDE and will end up with this issue again.

[frerich]

@sowsem Glad to hear that it helped. Since the README file is getting a bit too long for my taste, I started a wiki page on known caveats and just reference that.

[dav-otero]

Excellent! Again, thanks a lot for the help!