Immutable data edits

[Ruin0x11]

When dealing with hotloading, a common issue is how to support mods that transform existing data entries. Suppose there is a mod that turns every character into a putit:

data["base.chara"]:iter():each(function(c) c.class = "elona.slime" end)

The issue is: what happens when you're developing a mod and another mod touches the base.chara table afterwards? If you want to reload your changes immediately, you'd also have to apply the other data transformations in the proper order for the final state of the data to remain correct. The only practical way of doing this currently is to restart the engine, which misses the point of OpenNefia's hotload-first design.

Also, when modifying things like base.config_menu, you usually end up having to append to a list on the data entry.

-- TODO immutable data edits
local menu = { _type = "base.config_menu", _id = "weight_graph.menu" }
table.insert(data["base.config_menu"]:ensure("base.default").items, menu)

But you can't hotload this code without multiple copies of the menu entry appearing in the final data entry. You'd have to go back to the definition of weight_graph.menu and hotload that first to get back to the original state before applying the new changes.

One more issue is that any changes affecting a filtered set of data entries have to be run after all the data has been added, or the changes will not be applied to anything that was added afterward.

---

The basic idea I'm thinking of is to introduce a system similar to the event system, specialized for editing entries in data.

data["base.chara"]:edit("Turn everyone into a slime", "*", 50000, function(c) return { class = "elona.slime" } end)

These functions will be run in a specified order after all mods have finished loading, and every time a relevant data entry/transform is hotloaded afterwards.

The functions might mutate the data itself, or only return the diff of changes to the data. I'm not sure if the benefits will outweigh the performance penalty and inflexibility of the second option. Lua's lack of proper immutable data structures also works against such an idea.

The first argument to data[...]:edit() will be the name of the transform, as with the event system. The second might be a "selector" of some kind, which could match a single ID, a group of IDs with a Lua pattern, or contain a list of IDs to edit.

-- match everything in `base.chara`
data["base.chara"]:edit("Test", ".*", priority, fn)

-- match one ID
data["base.chara"]:edit("Test", "elona.putit", priority, fn)

-- match all characters added by the mod `plus`
data["base.chara"]:edit("Test", "plus%..*", priority, fn)

-- match a set of IDs
data["base.chara"]:edit("Test", { "elona.putit", "elona.red_putit" }, priority, fn)

-- match using a function? (pointless abstraction?)
data["base.chara"]:edit("Test", function(c) return c.class ~= "elona.putit" end, priority, fn)

The third argument is the priority of the transform, as with the event system.

The fourth argument is the transformation callback. I'm still not sure if the selector/callback that operates on one data entry on a time is the best idea. Maybe just returning a list of changes to apply to the entire table is better. (In that case, there would be no need for a selector anymore.)

-- first option: mutation
local function transform(data)
   data["elona.shopkeeper"].class = "elona.putit"
   data["elona.zeome"].class = "elona.putit"
end

-- second option: pure function callbacks returning a minimal diff
local function transform(data)
   return {
      { _id = "elona.shopkeeper", class = "elona.putit" },
      { _id = "elona.zeome", class = "elona.putit" },
   }
end

-- third option: mutate individually
local function transform(c)
   c.class = "elona.putit"
end
local selector = { "elona.shopkeeper", "elona.zeome" }

But ultimately the point is to be able to edit the data table with hotloading in a reproducible manner, even if several mods want to modify the data at different points.

When hotloading any transforms/added data, the entire chain of transforms would be applied to the updated data definition.

• If the per-entry option is chosen, the selector in this case could be used for performance reasons, so the entire set of data won't have to be traversed if only a small set of the data is changed. Or maybe a caching system keeping track of the set of affected IDs for each transform could be used, which gets reset every time new data is hotloaded into the corresponding table.
• If the whole-table option is chosen, we wouldn't be able to tell in the general case which transforms affect what data, so every change to a data entry/transform would have to run the entire suite of transforms for that data type.