Akka.Streams: memory-optimize ActorMaterializer HOCON injection
#6440
Conversation
creating lots of materializer instances (i.e. in actors that own their own streams) results in a tremendous amount of unreclaimable HOCON-related memory - on the order of 11Gb in https://github.com/Aaronontheweb/AkkaSqlQueryCrushTest This tries to avoid injecting the top level Akka.Streams fallback every time a materializer is used. In addition to that, we try to cache the default materializer settings parsed from the `ActorSystem`s HOCON if none are provided.
|
This should probably be back-ported to v1.4, C#11 syntax not withstanding. |
|
For reference
|
I'll need to rebuild this and integrate it with #6436 to do that, and remove a materializer optimization I also made inside my app. |
|
Ok, the changes are benign so we can merge them anyway |
|
According to test suite, this for sure broke something |
|
Ah, the |
| /// <summary> | ||
| /// Injecting the top-level Materializer HOCON configuration over and over again is expensive, so we want to avoid | ||
| /// doing it each time a materializer is instantiated. This flag will be set to true once the configuration has been | ||
| /// injected the first time. | ||
| /// </summary> | ||
| private static volatile bool _injectedConfig = false; | ||
| private static readonly ConcurrentDictionary<ActorSystem, bool> InjectedConfig = new(); |
There was a problem hiding this comment.
This would be unnecessary if the materializer were an ActorSystemExtension, but since it's not - have to do it this way.
| { | ||
| // Inject the top-level fallback config for the Materializer once, and only once. | ||
| // This is a performance optimization to avoid having to do this on every materialization. | ||
| system.Settings.InjectTopLevelFallback(DefaultConfig()); | ||
| _injectedConfig = true; | ||
|
|
||
| static async Task CleanUp(ActorSystem sys) |
There was a problem hiding this comment.
Added a static delegate to cleanup ActorSystem references after shut down
|
@Arkatufus re-ran my crush test with these changes - the materializer no longer even appears in the memory footprint now. |
|
@Aaronontheweb , in the JVM they have this default system wide materializer which is the recommended, since you typically only need one materializer per application. |
Yeah, I think we should go ahead and just do that too. |
Changes
creating lots of materializer instances (i.e. in actors that own their own streams) results in a tremendous amount of unreclaimable HOCON-related memory - on the order of 11Gb in https://github.com/Aaronontheweb/AkkaSqlQueryCrushTest
This tries to avoid injecting the top level Akka.Streams fallback every time a materializer is used. In addition to that, we try to cache the default materializer settings parsed from the
ActorSystems HOCON if none are provided.Checklist
For significant changes, please ensure that the following have been completed (delete if not relevant):