Implement shared memory WAL. #71
Conversation
|
Portability is done: only Linux and macOS (on illumos (and other 64-bit Linux archs) could potentially be supported, but they'd be hard to test; BSDs don't work due to locking; Windows is out due to memory remapping; 32-bit doesn't have enough address space. |
|
Mostly done. Still missing read-only mode, tetratelabs/wazero#2157 improvements, and documentation. |
|
Will need an improved design over tetratelabs/wazero#2157. |
ncruces
force-pushed
the
wal
branch
2 times, most recently
from
2e924bb to
faa9f6f
Compare
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.21.0 to 0.22.0. - [Commits](golang/crypto@v0.21.0...v0.22.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.6.0 to 0.7.0. - [Commits](golang/sync@v0.6.0...v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/sync dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
|
This is pretty much ready for testing, @infogulch. This should also make these bindings fully compatible with Litestream, @Koeng101. Suggestions welcome for a Litestream test setup. It also opens the way for SQLite memory mapped IO, and WAL-Mode Blocking Locks. |
|
I'll test out later! Very exciting! |
|
Windows may be possible after all. For reference: Shared memory primitives should work fine on any Unix OS, as implemented. In practice, they're useless without OFD locks, so are unavailable on BSD. illumos could work though, but is disabled for being impossible to test. |
|
tetratelabs/wazero#2177 was merged, and will be in the next release. |
I briefly looked into memory mapped IO, and I won't be pursuing it at this time. It has numerous caveats, especially if not implemented exactly like SQLite does it (mapping the entire file, hoping the size doesn't change often). Trying to implement the API straightforwardly (map/unmap this page), leads to a bunch of |
This intends to implement shared memory WAL for both 64-bit Linux and macOS (
amd64/arm64).This depends on tetratelabs/wazero#2157 (or some potentially better solution) to ensure Wasm memory is
mmaped.Current solution already passes
mptestbut still needs stability, portability and documentation work.