-
Notifications
You must be signed in to change notification settings - Fork 372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add process lock to avoid parallel cross issues. #963
Conversation
This seems to handle This is tested to work with 15 simultaneous processes on Windows and Linux. The Windows is slow, but that's to be expected. And the bottleneck is almost certainly Docker. Also, it seems flock itself is guaranteed to release on any form of process exit on Linux. For Windows, we can also see
|
this adds a fair bit of dependencies, is there another (simpler) crate for this? |
I'll see, we probably could use parking_lot directly. Or we could just document this relatively prominently. We could also just use this on Unix-only, where there's significantly less dependencies, since I believe due to how Windows works, this is unlikely to be an issue. Or, we could probably just use |
I've changed it to use |
Docker can have deadlocks or similar issues when invoked multiple times in rapid succession, so this implements a small and correct process lock with an identical API to `named_lock`. This allows us to prevent invoking Docker multiple times. The implementation uses `flock` on Linux/UNIX-like systems, which on Linux is guaranteed to be cleaned up when the process exits. On Windows, it uses `CreateMutexW`, which also guarantees to be cleaned up when the process exits. We use this rather than `fd-lock` because `fd-lock` has more dependencies, and it also uses `LockFile` on Windows which is not guaranteed to be cleaned up in a timely manner. The API is also designed for file locks, not process locks, which is only a good abstraction on UNIX-like systems. We use this rather than `named_lock` since it has numerous, unnecessary dependencies. This allows us to have no external dependencies besides libc and the Windows API, for an extremely lightweight locking library.
Docker can have deadlocks or similar issues when invoked multiple times
in rapid succession, so this implements a small and correct process lock
with an identical API to
named_lock
. This allows us to preventinvoking Docker multiple times.
The implementation uses
flock
on Linux/UNIX-like systems, which onLinux is guaranteed to be cleaned up when the process exits. On Windows, it uses
CreateMutexW
, which also guarantees to be cleaned up when the process exits.We use this rather than
fd-lock
becausefd-lock
has more dependencies, and it also usesLockFile
on Windows which is not guaranteed to be cleaned up in a timely manner. The API is also designed for file locks, not process locks, which is only a good abstraction on UNIX-like systems.We use this rather than
named_lock
since it has numerous, unnecessary dependencies. This allows us to have no external dependencies besides libc and the Windows API, for an extremely lightweight locking library.Fixes #496.