Possible race in CacheDir config

[mwichmann]

This was filed as a comment to closed PR #3353 by @mtvjr. Moving to a fresh issue because I believe it to be a different problem, plus that PR has been closed for over four years.

================

Looks like this might not completely make things thread safe. We just ran into an issue where it appears the cache file is attempted to be read between file creation and the json.dump call finishing. The json.load call failed due to an empty input file.

13:57:22  Traceback (most recent call last):
13:57:22    File "/usr/lib/python3.6/site-packages/SCons/CacheDir.py", line 195, in _readconfig
13:57:22      self.config = json.load(config)
13:57:22    File "/usr/lib64/python3.6/json/__init__.py", line 299, in load
13:57:22      parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
13:57:22    File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
13:57:22      return _default_decoder.decode(s)
13:57:22    File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
13:57:22      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
13:57:22    File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
13:57:22      raise JSONDecodeError("Expecting value", s, err.value) from None
13:57:22  json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
13:57:22  
13:57:22  During handling of the above exception, another exception occurred:
13:57:22  
13:57:22  Traceback (most recent call last):
13:57:22    File "/usr/lib/python3.6/site-packages/SCons/Taskmaster.py", line 230, in execute
13:57:22      if not t.retrieve_from_cache():
13:57:22    File "/usr/lib/python3.6/site-packages/SCons/Node/FS.py", line 2991, in retrieve_from_cache
13:57:22      return self.get_build_env().get_CacheDir().retrieve(self)
13:57:22    File "/usr/lib/python3.6/site-packages/SCons/Environment.py", line 1033, in get_CacheDir
13:57:22      cd = SCons.CacheDir.CacheDir(path)
13:57:22    File "/usr/lib/python3.6/site-packages/SCons/CacheDir.py", line 159, in __init__
13:57:22      self._readconfig(path)
13:57:22    File "/usr/lib/python3.6/site-packages/SCons/CacheDir.py", line 198, in _readconfig
13:57:22      raise SCons.Errors.SConsEnvironmentError(msg)
13:57:22  SCons.Errors.SConsEnvironmentError: Failed to read cache configuration for /var/lib/scons-ext/cache

[mwichmann]

Note the MSVC config caching recently has locking added, using the new FileLock class. We should be able to do the same for the CacheDir config.

[mtvjr]

Thank you for the quick response and my apologies for not responding myself, I tend not to check GitHub very often. Let me know if you need any help or any additional information about the issue.

In our sconstruct, we've bypassed the issue by calling CacheDir directly as soon as we know what directory our cache will use, but before spawning any jobs.

[bdbaddog]

Thank you for the quick response and my apologies for not responding myself, I tend not to check GitHub very often. Let me know if you need any help or any additional information about the issue.

In our sconstruct, we've bypassed the issue by calling CacheDir directly as soon as we know what directory our cache will use, but before spawning any jobs.

Are you spawning off other scons runs? or just regular build tasks?

[mwichmann]

Yeah, I'd say unless you're planning to have distinct CacheDir setups per environment, getting it set up early is good practice anyway.

[mtvjr]

Thank you for the quick response and my apologies for not responding myself, I tend not to check GitHub very often. Let me know if you need any help or any additional information about the issue.
In our sconstruct, we've bypassed the issue by calling CacheDir directly as soon as we know what directory our cache will use, but before spawning any jobs.

Are you spawning off other scons runs? or just regular build tasks?

It should just be regular build tasks, with custom actions.