Why does the size of the save grow with each save? #79653
Comments
IdleSol
added
the
(S1 - Need confirmation)
|
Unless someone knows, you might be able to try to figure it out by using diff tools to see what's different in the save. On thing I can think of is if you're using the same save as you use as the basis for #79651, in which case it might be egg hatching checks resulting in new animals appearing (and the eggs disappearing in the process). That would be a bug, but there are a number of things that are checked when a map (actually submap) gets loaded into the game, and it could be a result of these checks maybe running more frequently than they should on save/load cycles with no time in between. Note that this is just a guess. |
|
Part 2, which I was going to test initially. While playing in my main world, I noticed a dramatic increase in save size. From ~340 MB to ~468 MB. This surprised me, because I was not traveling to new regions, but completing a military base sweep. Moreover, I was destroying unnecessary corpses and things with a vehicle. Of course there was one controversial point, since on one of the hikes, I used a new route to the base. And so a few new omt were opened. But their number is not comparable to the already opened omt. So I did an experiment:
Clarification. By save I mean “save and exit”. Then I look at the size and load the save.
Here are the results:
As you can see, moving items from one place to another added 1 MB (~+5%) to my save size. At the same time, if I take items out of containers, I reduce the size by 0.5 MB. And also saving to new locations gives an additional increase in size. |
No, it was a fresh test world. All saves happened at the same game moment. Or at the game start time: 8 hours 0 minutes and 0 seconds.
If no one gives me an answer. Then I'll watch which file grows in size and compare them to each other. I'm still hoping someone knows what's wrong though. UPD. Or it will say it can't reproduce and the reason is my machine and/or configuration. Or is this even normal behavior for linux file resaves. |
|
A number of comments:
|
The time didn't go down only for the first test, where all I did was load and save. In the test where I was transferring things, the time was running as normal. Debugging was only used for teleporting a character, spawning a bag and opening a map.
I hadn't thought of that |
|
You can just copy world folders after each save and then compare to see what was modified each time. |
|
Not directly addressing the root cause, but while working on #78857 I did notice files getting re-written that I would expect to be 'static'. Apparently under the hood the actual division of data between files is not as clean as optimal. Maybe after enabling save compression I'll start a longer term project for reorganizing that. |
|
There are a lot of things happening:
|
|
If only I understood which files were responsible for what. But a quick check made me realize that the contents of the files are changing. And also that everything in them is lumped into one line and it's a lot of fun to figure it out. As example 1: uistate.json --"input_history":{},
++"input_history":{"item_filter":[],"list_item_downvote":[],"list_item_priority":[]},As example 2: o.1.1 vs Either there are no matches or just a different order. There are 40k+ lines in this file (after some processing). And trying to understand whether something has changed or just a different order is a hopeless endeavor. If you don't know what it is responsible for and how it is created. I'm still leaning that something is being added since the size keeps increasing This is the processed output of diff: save0 - initial save |
|
Stuff ending up in "random" order is a price you pay for using maps (i.e. the data structure, not the game feature), I believe. I think their hash keys are made up on the spot, and thus have no consistency in their iteration orders between saves. Edit: Incorrect segment: I'd take the smallest of the map files and run a diff on it between saves (before or after manually making it readable by untangling the horrible anti human one liners) in order to try to find things that grow. @ZhilkinSerg's images indicate some things are "refreshed" on reload. In those cases either the regeneration on load should be removed, or the saving should be removed (no point in saving something that's going to be overwritten on load). That unnecessary work shouldn't grow the saves, though, just change them needlessly. Edit: |
Raw |
|
@akrieger is of course correct. The loading code loads into an |
|
Since I deleted the old saves, I had to make new ones. But this time I decided to compare three saves instead of two.
This time there are fewer files that change. But two peculiarities: And if you look at the difference in 219.43.1.map between {
"version":36,
"coordinates":[439,86,1],
"turn_last_touched":5212800,
"temperature":0,
"terrain":[["t_open_air",144]]
++ "radiation":[0,144],
++ "furniture":[],
++ "items":[],
++ "traps":[],
++ "fields":[],
++ "cosmetics":[],
++ "spawns":[],
++ "vehicles":[],
++ "partial_constructions":[]
},That is, for something (tile, omt?) with coordinates 439,86,1 a lot of parameters were missed in the first save. Which were added only in the second save. |
|
The "data" added seems to be null data. My guess would be that unnecessary fields weren't generated originally, but added by the loading code as "missing" and then kept thereafter. I don't think this is the stuff we should be looking at, but rather things that keep growing for no apparent reason. However, the realization that we need 3 saves rather than two to detect what's growing for no reason (adding "missing" empty data is a bad reason, but still a reason) is important. |
This is already from another world. But this time it's a complete map file, created right after saving at the beginning of the game. Note, for the coordinates:
And I don't believe the game is flipping a coin, save here and don't save there. There must be a reason for that. What if we were lucky enough to have empty fields in that location? Maybe there weren't empty fields there, but the game didn't save them the first time and replaced them with empty ones the second time? |
|
The two entries containing the extra empty stuff have treetops in them, while the empty ones only mention open air. That's probably the cause. |
|
Comparing o.1.-1 save iteration 3 to save iteration 2: A large section starting with "predecessors" is growing, but since the entries are in random order it's not really possible to tease out what the additions are (114689 bytes growing to 115654). Edit:
The vector is written to JSON, obviously without knowledge of whether the entries were read from the save or invented from the overmap terrain data originally. Regardless, the code shouldn't recklessly add camps based on overmap ids. overmap Id addition should only add it if it doesn't already exist. If it cannot be guaranteed that the JSON data is read first, the JSON reading should check if the entry already exists and replace it (assuming it's an overmap entry of inferior quality), or perform a check to see which one is the better one (existing saves with multiple entries would probably use the first one to store the faction status, such as quests, etc.). This kind of processing might be useful as a "migration" action to weed out duplicates from existing saves. Edit 2: |
|
The o.1.0 file from the main world weighs 8 MB. A search for And also 69 for UPD. What do these files store? I looked at all the files and the file o.0.0 has a size of 304 MB. But the whole folder weighs 651 MB. If I compare the save for today to the save for February 10th. The file had a weight of 10 MB, with a total weight of 340 MB. |
|
That probably means the save has been save/loaded 68 time since its creation, but it's obviously possible there are other factors we haven't identified involved. My save folder contains a bunch of other files, plus the maps folder and a .mm1 folder. Given that I saw unwarranted growth in the "predecessors" data, there's probably something fishy going on there as well. Edit: |
Describe the bug
I created a world and a character. After which I did:
What kind of extra information is added to the save every time I save and/or quit the game?
I waited an extra few minutes of real time for the 4 tests (between loading and saving) without doing anything in the game. The difference is not significant.
Attach save file
n/a
Steps to reproduce
n/a
Expected behavior
It's okay if it's some useful information.
Screenshots
No response
Versions and configuration
Testing was done on a version of cdda-windows-with-graphics-x64-2025-02-12-1549 running on an emulator. Arch + portproton.
Additionally tested on stable version 0.H for linux
Additional context
Of course the size is small, just a few kilobytes. Which means nothing against the size of the late game save file. Or maybe there is a bug that causes such large sizes.
The text was updated successfully, but these errors were encountered: