wasm: does not return memory to the OS #59061
Comments
|
CC @neelance |
cherrymui
changed the title
|
Yes, this is a limitation of the current Go Wasm implementation. I also recall there was some discussion about using a strategy to reuse memory similar to what we do on Plan 9, but I don't think it completed. At the point when the Wasm port is implemented (and may be still true?), there was no way to shrink the Wasm linear memory. So the best strategy is probably reuse it. If you (or anyone) would like to implement the Plan 9 strategy, you're welcome to do so. Thanks. cc @golang/wasm |
cherrymui
added
help wanted
NeedsFix
|
I unified the plan9 and wasm memory allocation in https://github.com/achille-roussel/go/compare/master..fix-wasm-memory-leak to confirm the hypothesis. When When running the same test with the change I made: To reproduce: |
|
Happy to send a CL with the fix to unify memory allocation with plan9 👍 |
|
Change https://go.dev/cl/476717 mentions this issue: |
When compiling Go programs to WebAssembly, the memory allocation strategy was neither releasing memory to the OS nor reusing blocks freed by calls to `runtime.sysFreeOS`. This CL unifies the plan9 and wasm memory management strategy since both platforms use a linear memory space and do not have a mechanism for returning memory blocks to the OS. Fixes golang#59061
When compiling Go programs to WebAssembly, the memory allocation strategy was neither releasing memory to the OS nor reusing blocks freed by calls to `runtime.sysFreeOS`. This CL unifies the plan9 and wasm memory management strategy since both platforms use a linear memory space and do not have a mechanism for returning memory blocks to the OS. Fixes golang#59061
When compiling Go programs to WebAssembly, the memory allocation strategy was neither releasing memory to the OS nor reusing blocks freed by calls to runtime.sysFreeOS. This CL unifies the plan9 and wasm memory management strategy since both platforms use a linear memory space and do not have a mechanism for returning memory blocks to the OS. Fixes golang#59061
When compiling Go programs to WebAssembly, the memory allocation strategy was neither releasing memory to the OS nor reusing blocks freed by calls to runtime.sysFreeOS. This CL unifies the plan9 and wasm memory management strategy since both platforms use a linear memory space and do not have a mechanism for returning memory blocks to the OS. Fixes golang#59061
|
May I ask when would this feature be landed? Like Go 1.21? Please have a look, thanks. @achille-roussel |
|
Hello @rickieNodereal I believe this change should be released in Go 1.21 since it has been merged, you could use it on tip already as well. |
|
Thanks a lot! @achille-roussel But may I ask, what is "use it on tip already"? Sorry I'm new to Golang, could you explain a bit more detailed, thanks. |
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
As part of the work on #58141, I came across the runtime code that takes care of interfacing between the Go and system memory allocators, and I believe that Go programs built to WebAssembly are leaking all memory they attempt to release to the OS.
The relevant code is in https://github.com/golang/go/blob/master/src/runtime/mem_js.go
The code comment suggest that the initial plan was either to leverage virtual memory capabilities of the host application if the design discussions recorded at https://github.com/WebAssembly/design/blob/master/FutureFeatures.md#finer-grained-control-over-memory had materialized, or use the same allocation strategy used on Plan9 which also uses a linear memory space (via brk/sbrk) https://github.com/golang/go/blob/master/src/runtime/mem_plan9.go
Since neither of these alternatives were implemented, the current implementation leaks all memory blocks that the GC attempts to release to the OS (via calls to
runtime.sysFreeOS).I have verified this hypothesis by adding print statements in
runtime.sysReserveOSandruntime.sysFreeOSwhich show that theruntime.reserveEndpointer tracking the current size of the WASM memory, is always increasing (verified when running tests for the standardbytespackage). It also shows that the size of the WASM memory is ~7x the size of the Go heap in those tests, which further supports the hypothesis of a leak.I looked for open issues related to WebAssembly and memory usage and it is possible that these might be tied back to this memory leak (if confirmed):
What did you expect to see?
I was expecting the size of the WASM memory to be closer to the size of the Go heap.
I was also expecting
runtime.sysFreeOSto retain the freed memory blocks in a freelist in order to reuse them when memory is needed again.What did you see instead?
The size of the WASM memory is ~7x larger than the size of the Go heap.
When the program needs memory, it always increases the size of the WebAssembly linear memory and never reuses memory blocks that it freed.
The text was updated successfully, but these errors were encountered: