-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
network_run
hogging lot of memory on call to VoxelManip:read_from_map
#327
Comments
Yeah it does, only thing it currently ignores is cables. Mapblocks for everything else has to be loaded although I've once tested possible caching a bit. Some incomplete info here: However never pushed that forward because as currently pipeworks would anyway need areas to be loaded so in actual game it would be mostly useful for generators. There's also this optimization idea for quarries to release mapblocks they're actually working on, implementing this could help a bit but wouldn't be silver bullet. It would only help in very specific scenarios involving a lot of quarries: But cable count shouldn't really matter, but network cache builder still has to load all mapblocks to find cables while initializing network for first time, after network changes or when admin resets cached data. If high memory usage is because of mapblocks for cables then I'd recommend checking engine cache configuration / memory tuning settings. It should be possible to release these mapblocks immediately after first read, how many at time depends on how much server is able to read in within network cache builder time limits. edit. I think worst case for that cable count is around 2200 mapblocks (assuming cable runs straight at the edge of mapblock). |
You're right about the cables count, it doesn't seem to have any impact and the networking building process is very smooth as far I can see. Taking this into account, my network is actually not that big. I'm pretty much out of my depth here, I still know too few about Minetest. But to me it seem like I've also tested forcing the call to The linked issues where definitely interesting read, but even with a good caching layer and quarry optimisation, I'm not sure this issue will completely disappear? Something else seem to be going on. |
The following line quickly trigger huge memory consumption on larger networks:
https://github.com/mt-mods/technic/blob/master/technic/machines/network.lua#L596C7-L596C7
In my setup, the calls stack up to 10-15GiB of allocated memory before the server shutdown get triggered by the system oom killer.
Server details:
Technic network details:
The consumers triggering the call to
VoxelManip:read_from_map
are all quarries in far away chunks. In the offending network, quarries account for ~60 of the consumers. The crash seem to occur when most of them require the call to run.Commenting out the code for the vm fail to get some of the nodes in some network runs, but fix the performance issues. alternatively, keeping the code intact and reducing the size of the network prevent oom crashes, but still generate intermittent lag spikes on the server.
The text was updated successfully, but these errors were encountered: