deep copy (or merge?)

[stephanecharette]

I see the discussion in issue #661 about update(). This almost does what I was looking for, but not quite.

I have 2 json objects: the user settings, and then the defaults. I would like to merge this together into 1 json. But say the defaults look like this:

{
  "settings": {
    "mouse" : true,
    "colour" : "red",
    stuff : {
      init : "foo",
      shutdown : "bar"
    }
  }
}

If the user settings is empty, then when I call this code everything works fine:

    auto user_settings = nlohmann::json::parse(blah);
    auto combined_settings = get_default_settings();
    combined_settings.update(user_settings);

But if the user has even 1 item, such as:

{
  "settings":{
    "colour":"blue"
  }
}

Then when I call update(), the user's settings object completely overwrites all the other settings, and I lose the default values. What I was looking for in this example is for "blue" to overwrite "red" without losing everything else that came from the defaults.

Does this exist?

[nlohmann]

The update function is indeed "shallow" in the sense it takes and all-or-nothing approach when it comes to object keys. I could image an optional bool recursive=false parameter that, if set to true, will traverse the two objects and only update missing object keys, but does not touch existing keys. Would that make sense?

[stephanecharette]

If it doesn't touch existing keys, then I'd be left with just the defaults and would have no way to merge in the sparse user settings, right?

I think the opposite is needed. If I have a selection of 100 settings in the JSON, but then have 4-5 "user" overrides, how do I merge those few settings into the known large collection of default values?

[stephanecharette]

Oh, nevermind, reversing the order would fix that issue. I would start with the user settings, then merge into it the large "default values" json object, which would only update the ones that the user hasn't explicitly set. Yes, that would work.

[stephanecharette]

If anyone comes across this discussion topic in the future, this is how I got it working:

nlohmann::json merge(const nlohmann::json & lhs, const nlohmann::json & rhs)
{
	// start with the full LHS json, into which we'll copy everything from RHS
	nlohmann::json j = lhs;

	// if a key exists in both LHS and RHS, then we keep RHS

	for (auto it = rhs.cbegin(); it != rhs.cend(); ++it)
	{
		const auto & key = it.key();
		if (it->is_object())
		{
			if (lhs.contains(key))
			{
				// object already exists (must merge)
				j[key] = merge(lhs[key], rhs[key]);
			}
			else
			{
				// object does not exist in LHS, so use the RHS
				j[key] = rhs[key];
			}
		}
		else
		{
			// this is an individual item, use RHS
			j[key] = it.value();
		}
	}

	return j;
}

Then I call it like this:

auto user_settings = nlohmann::json::parse(...);
auto default_settings = get_default_settings(); // obtain defaults as a JSON object from within my app
auto combined_settings = merge(default_settings, user_settings);

[nlohmann]

Thanks for sharing! I'll see if I can make a PR as described in #3006 (comment) that adds a flag to choose this behavior from update.

One note: key() returns a const std::string&, so

const auto key = it.key();

makes unnecessary copies and could be changed to

const auto& key = it.key();

Furthermore,

j[key] = rhs.at(key);

can be changed to

j[key] = rhs[key];

as you already iterating rhs's keys and do not need a bound check.

[nlohmann]

One more thing: be aware that the update function has a bug with the extended diagnostics, see #3007.

[nlohmann]

FYI: I now started a PR to add this functionality to the library, albeit by extending the update function rather than adding a new function: #3069. Feedback welcome!

[stephanecharette]

Thank you, @nlohmann.

[nlohmann]

It's merged now.

[stephanecharette]

Fixed up code above as you commented just in case someone comes across this issue and wants to re-use the code. Thanks for pointing it out.