Automatically register only engine classes whose header has been included

[dsnopek]

This is based on an idea from @maiself at a GDExtension meeting from a couple months ago!

In PR #1050, we added a HashMap that stores the binding callbacks for all engine types, in order to fix a serious bug where the wrapper class used could be an arbitrary parent class (rather than the actual engine class) leading to unexpected behavior in some situations.

Unfortunately, this led to issue #1160 where builds would be ~1-5mb bigger than previously.

This was due to the linker previously being able to optimize away any class that wasn't actually used in the GDExtension. However, if we have to reference every class in order to add their binding callbacks to the HashMap, they are all technically "used", and so none could be optimized away.

@maiself had this really clever idea, which was recorded in the meeting notes as:

// some register header

template<Class>
class Register {
	Register(class_name) {
		hash_map.insert({class_name, type_info(Class)});
	}
};

// node.hpp

static inline Register<Node>("Node"); 

This would mean that classes are only added to the HashMap, if the developer actually includes the header file for them! This should allow the linker to do it's old optimization again (I hope :-))

This PR attempts to implement that, however, I encountered some interesting issues when actually trying to do it:

• static inline variables can't be anonymous -- apparently the variable name is used to tell that each declaration refers to the same object, so all but one can be removed by the linker (which what the inline is aiming to do here).
• We can't directly add the items to the HashMap in the registration object, because the HashMap is keyed by StringName, and we can't create StringName's until after godot-cpp has been initialized. So, instead, the EngineClassRegistration<T> constructor is adding a callback which will do the actual registration at the appropriate time.
• We can't simply declare the std::vector as a static variable at the compilation unit level, because static variables can be initialized in any order, and we can't know that our inline static EngineClassRegistration<T> variables won't be initialized before the std::vector. This is why that is a static variable at the function level, so that it will be initialized on demand.

None of those are really necessarily a problem, but it's why the code in this PR is more complicated than the simple psuedo-code that I started with :-)

Any feedback on how I could simplify this would be appreciated! But, personally, assuming it actually solves the problem, I think even with all these complications this is still an acceptable way to do this.

Fixes #1160

[dsnopek]

Compiling the test GDExtension (used in the automated tests) with the default scons flags (so, it's the debug template) on Ubuntu 20.04 with g++ 11.4.0, I'm seeing:

• 1.6mb (with this PR)
• 4.7mb (on master at ef2f63a)

So, it appears to be working, at least in my case!

I'd be very curious to see what results others get with other compilers on other OS's.

EDIT: Updated after getting editor plugin related classes correctly excluded as well.

[mihe]

My setup in Godot Jolt is admittedly somewhat unorthodox compared to most other godot-cpp usage, so this is not really a representative sample, but here are the numbers I'm seeing on Windows:

	Before	After	Change (KB)	Change (%)
MSVC (debug)	3204 KB	3073 KB	-131 KB	-4.1%
MSVC (release)	3094 KB	2959 KB	-135 KB	-4.4%
clang-cl (debug)	3704 KB	3529 KB	-175 KB	-4.7%
clang-cl (release)	3588 KB	3395 KB	-193 KB	-5.4%

It's certainly an improvement across the board, but nothing astronomical in my particular case. I can give it a try on other platforms later to see if it's specific to the linker/platform perhaps.

For reference, I saw about a ~1.2 MB increase with clang-cl when #1050 was introduced, and a ~2 MB bump with MSVC. I have since added quite a few godot-cpp includes though (the total of which can be seen here) so I assume I wouldn't win it all back even if this PR had cancelled out #1050 completely.

[dsnopek]

Based on a RocketChat conversation with @mihe and @maiself, I've come to realize that I was trying to use static inline variables incorrectly -- I've adjusted it and now the callbacks can be stored in a std::vector rather than std::unordered_set, eliminating one of the complicated workarounds.

Also, I noticed that we were still including some editor plugin classes, even if the GDExtension didn't do anything with editor plugins. So, I rearranged some of that code, and now those will be excluded too! It saved an additional ~200kb in my testing.

[dsnopek]

@mihe Thanks for testing and posting your numbers!

For reference, I saw about a ~1.2 MB increase with clang-cl when #1050 was introduced, and a ~2 MB bump with MSVC. I have since added quite a few godot-cpp includes though (the total of which can be seen here) so I assume I wouldn't win it all back even if this PR had cancelled out #1050 completely.

Hm, you're probably right that you have enough includes now that it won't cancel out all of the size gains from #1050, but the difference still seems quite large (~200kb vs 1.2mb). It'd be really interesting to try reverting #1050 and see if you reclaim the same amount of memory, or if it's somewhat less, although, reverting it would probably be pretty messy at this point given all the changes that have been made since then.

[dsnopek]

I just tried the latest PR with my SG Physics 2D extension (using the same platform, compiler and setup as described in my previous comment) and got this result:

• 3.5mb (with this PR)
• 6.4mb (on master at ef2f63a)

So, I'm personally seeing a pretty dramatic improvement in this case!

[mihe]

It'd be really interesting to try reverting #1050 and see if you reclaim the same amount of memory, or if it's somewhat less, although, reverting it would probably be pretty messy at this point given all the changes that have been made since then.

Seeing as how this PR has gotten rid of the generated register_engine_classes, I guess it largely already has reverted it.

As such, I just now tried simply removing that new inline static variable you added to GDEXTENSION_CLASS_ALIAS, which obviously won't function correctly, but still compiles, and I saw a reduction of about -0.6 MB.

I also tried building on Linux with both Clang and GCC, and with this PR I'm still seeing very modest reductions in my project, in the range of -0.1-ish MB, but when I omit this new inline static variable I'm seeing reductions up to -1.3 MB.

I guess I'll need to fire up SizeBench to better understand what's happening. Either way, clearly this PR is a net positive, so I'm all for it.

[vnen]

I didn't test the size reduction but code-wise this looks good, and the discussion seems to agree it's an improvement.

[AThousandShips]

Just a tiny nitpick, looks good otherwise

[dsnopek]

Cherry-picked for 4.1 in PR #1281