Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add advisory file locking support. #69

Merged
merged 5 commits into from
Dec 2, 2022
Merged

Add advisory file locking support. #69

merged 5 commits into from
Dec 2, 2022

Conversation

sunfishcode
Copy link
Member

Add functions which correspond to flock in Unix. This is not a POSIX interface, but it's sufficiently portable, there's a way to implement it on Windows, and it doesn't have problematic process-associated locks.

Instead of having a single flock function that gets passed and enum, this has separate functions for shared vs. exclusive, and locking vs. non-locking. That's something we can revisit if needed, but it keeps the API simpler for now.

Add functions which correspond to `flock` in Unix. This is not a POSIX
interface, but it's sufficiently portable, there's a way to implement it
on Windows, and it doesn't have problematic process-associated locks.

Instead of having a single `flock` function that gets passed and enum,
this has separate functions for shared vs. exclusive, and locking vs.
non-locking. That's something we can revisit if needed, but it keeps the
API simpler for now.
Also mention that exclusive locks may require write mode on file
descriptors, as described in the [Linux `flock` man page].

[Linux `flock` man page]: https://man7.org/linux/man-pages/man2/flock.2.html
@bjorn3
Copy link

bjorn3 commented Nov 23, 2022

Trying to lock a file in one mode and then locking in another mode inside the same process (not wasm instance!) will silently switch the lock mode even if as I understand it the locking was done through a different fd. In addition this switching is not guaranteed to be atomic. Flock also doesn't work on NFS. Finally it is not clear to me how handling multiple locks for the same file in the same process is supposed behave.

Maybe the best way would be to have a clean slatw design for wasi and have wasi runtimes emulate the right semantics using for example hidden files in combination with a single tightly controlled process global advisory or mandatory lock for the purpose of crash recovery. Or something like that.

@sunfishcode
Copy link
Member Author

Trying to lock a file in one mode and then locking in another mode inside the same process (not wasm instance!) will silently switch the lock mode even if as I understand it the locking was done through a different fd. In addition this switching is not guaranteed to be atomic.

Linux's flock documentation says "If a process uses open(2) (or similar) to obtain more than one file descriptor for the same file, these file descriptors are treated independently by flock()." In my experiments, FreeBSD's flock has this behavior too.

Flock also doesn't work on NFS.

It is emulated to some degree on some platforms, though there are caveats. But my overall idea is, the fact that some people want to use an API not designed to be used over networks, over networks, shouldn't mean that everyone else can't have nice things 😄. I'll add some documentation about this.

Finally it is not clear to me how handling multiple locks for the same file in the same process is supposed behave.

Linux documents that flock locks are per file-description, and not per-process. FreeBSD's appear to behave the same way.

Maybe the best way would be to have a clean slatw design for wasi and have wasi runtimes emulate the right semantics using for example hidden files in combination with a single tightly controlled process global advisory or mandatory lock for the purpose of crash recovery. Or something like that.

We can do that if we need to, but my sense here is that flock is sufficient.

@sunfishcode
Copy link
Member Author

I've now tried an experiment on NFS on Linux, where the man page says that fcntl is emulated using fcntl when on NFS, and it still has per-file-description semantics, rather than per-process.

@sunfishcode
Copy link
Member Author

To protect users from silent data corruption, I've added a note that implementations should know whether the filesystem they're on supports locking or not, and if not, they should make the lock functions here fail with ENOSYS.

@bjorn3
Copy link

bjorn3 commented Nov 23, 2022

fcntl(F_SETLK) is process-global, which is cursed. Linux also has fcntl(F_OFD_SETLK) which has the same semantics as flock wrt to multiple fds in the same process. I presume glibc uses the latter to emulate flock on NFS.

@sunfishcode sunfishcode merged commit b8a6e5a into main Dec 2, 2022
@sunfishcode sunfishcode deleted the sunfishcode/flock branch December 2, 2022 01:31
@bjorn3 bjorn3 mentioned this pull request Dec 2, 2022
sunfishcode added a commit to bytecodealliance/preview2-prototyping that referenced this pull request Dec 8, 2022
sunfishcode added a commit to bytecodealliance/preview2-prototyping that referenced this pull request Dec 8, 2022
sunfishcode added a commit to bytecodealliance/preview2-prototyping that referenced this pull request Dec 8, 2022
pchickey pushed a commit to pchickey/wasmtime that referenced this pull request May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants