Clear affine slots when dropping a Module
#5321
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit implements a resource usage optimization for Wasmtime with the pooling instance allocator by ensuring that when a `Module` is dropped its backing virtual memory mappings are all removed. Currently when a `Module` is dropped it releases a strong reference to its internal memory image but the memory image may stick around in individual pooling instance allocator slots. When using the `Random` allocation strategy, for example, this means that the memory images could stick around for a long time. While not a pressing issue this has resource usage implications for Wasmtime. Namely removing a `Module` does not guarantee the memfd, if in use for a memory image, is closed and deallocated within the kernel. Unfortunately simply closing the memfd is not sufficient as well as the mappings into the address space additionally all need to be removed for the kernel to release the resources for the memfd. This means that to release all kernel-level resources for a `Module` all slots which have the memory image mapped in must have the slot reset. This problem isn't particularly present when using the `NextAvailable` allocation strategy since the number of lingering memfds is proportional to the maximum concurrent size of wasm instances. With the `Random` and `ReuseAffinity` strategies, however, it's much more prominent because the number of lingering memfds can reach the total number of slots available. This can appear as a leak of kernel-level memory which can cause other system instability. To fix this issue this commit adds necessary instrumentation to `Drop for Module` to purge all references to the module in the pooling instance allocator. All index allocation strategies now maintain affinity tracking to ensure that regardless of the strategy in use a module that is dropped will remove all its memory mappings. A new allocation method was added to the index allocator for allocating an index without setting affinity and only allocating affine slots. This is used to iterate over all the affine slots without holding the global index lock for an unnecessarily long time while mappings are removed.
github-actions
bot
added
the
wasmtime:api
Subscribe to Label Actioncc @peterhuene
This issue or pull request has been labeled: "wasmtime:api"
Thus the following users have been cc'd because of the following labels:
To subscribe or unsubscribe from this label, edit the |
cfallin
approved these changes
Nov 23, 2022
There was a problem hiding this comment.
This looks correct to me -- the unification of the different strategies' state storage is nice too. I like that the "pick and clear the next slot with affinity to this module" operation is O(1) due to the freelist.
I have a few comments below to improve clarity but nothing critical!
crates/runtime/src/instance/allocator/pooling/index_allocator.rs
Outdated
Show resolved
Hide resolved
crates/runtime/src/instance/allocator/pooling/index_allocator.rs
Outdated
Show resolved
Hide resolved
crates/runtime/src/instance/allocator/pooling/index_allocator.rs
Outdated
Show resolved
Hide resolved
crates/runtime/src/instance/allocator/pooling/index_allocator.rs
Outdated
Show resolved
Hide resolved
|
Thanks for the review! I also fixed an issue where |
Ah, good catch; updated diff LGTM. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit implements a resource usage optimization for Wasmtime with the pooling instance allocator by ensuring that when a
Moduleis dropped its backing virtual memory mappings are all removed. Currently when aModuleis dropped it releases a strong reference to its internal memory image but the memory image may stick around in individual pooling instance allocator slots. When using theRandomallocation strategy, for example, this means that the memory images could stick around for a long time.While not a pressing issue this has resource usage implications for Wasmtime. Namely removing a
Moduledoes not guarantee the memfd, if in use for a memory image, is closed and deallocated within the kernel. Unfortunately simply closing the memfd is not sufficient as well as the mappings into the address space additionally all need to be removed for the kernel to release the resources for the memfd. This means that to release all kernel-level resources for aModuleall slots which have the memory image mapped in must have the slot reset.This problem isn't particularly present when using the
NextAvailableallocation strategy since the number of lingering memfds is proportional to the maximum concurrent size of wasm instances. With theRandomandReuseAffinitystrategies, however, it's much more prominent because the number of lingering memfds can reach the total number of slots available. This can appear as a leak of kernel-level memory which can cause other system instability.To fix this issue this commit adds necessary instrumentation to
Drop for Moduleto purge all references to the module in the pooling instance allocator. All index allocation strategies now maintain affinity tracking to ensure that regardless of the strategy in use a module that is dropped will remove all its memory mappings. A new allocation method was added to the index allocator for allocating an index without setting affinity and only allocating affine slots. This is used to iterate over all the affine slots without holding the global index lock for an unnecessarily long time while mappings are removed.