local-binary-cache-store upsertFile not concurrency-safe

[Ericson2314]

Describe the bug

When the same nar is to be uploaded multiple times (I think via dependencies), the concurrent writes/renames will conflict and failures may occur.

Steps To Reproduce

Still working on this. I did add a bunch of logging and observed the concurrency of multiple atomicWrite calls.

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

Everything is written without errors, redundant nar upserts either do nothing or replace the existing nar.

nix-env --version output

Master branch

Additional context

atomicWrite in src/libstore/local-binary-cache-store.cc uses process ids for temporary files, but I think it needs to use something finer-grained to distinguish threads.

[Ericson2314]

CC @matthewbauer

[Ericson2314]

Based on what I did in obsidiansystems@953ceba, it looks like one trigger is copying a fixed output derivation along with another that depends on the first one.

[matthewbauer]

Is it actually atomicWrite that’s messed up or maybe just the init that’s not being called?

[stale]

I marked this as stale due to inactivity. → More info