[TRACKER] Unit Test Porting

[RevoluPowered]

We integrated doctest unit test framework and have depreciated old unit tests in favor of headless tests which can run on our CI.

Some useful information and the PR can be found here:
#40148 (comment)

Test status:

• test_string.h (was ported in initial pr)
• test_astar: (ported in Port AStar tests to use doctest #40835).
• test_basis: (ported in Port Basis tests to use doctest. #40940).
• test_class_db: (ported in Port ClassDB tests to use doctest #40980).
• test_gdscript: don't port, likely going to be rewritten, see Enable support for C++ modules tests #40720 (comment).
• test_gui: no assertions found, just random code doing stuff, not sure how to test.
• ⚠️ test_math ⚠️: catastrophic, need a nuclear weapon to clean this up first 🙂, it seems like the code has accumulated random stuff over the span of Godot's existence. Seems to include code for math parsing, geometry (updated 3 months ago), various file access stuff, even platform specific code like: OS::get_singleton()->execute("/bin/ls");.
• test_oa_hash_map: ok, but has some stress tests.
• test_ordered_hash_map: (ported in Port OrderedHashMap tests to doctest #41117).
• test_physics_2d/3d and test_render: no asserts, just code, not sure how to test.
• test_shader_lang: need someone from core devs for best results, but likely can be ported without issues.

*updated from Xrayez[m] below
New tests to be written:

• local_vector.h
• networking?
• audio?
• ... etc
• every data container we implement

General guide:

1. Delete the .cpp file and remove the old test harness after copying contents to the header
2. include the doctest header
3. if you have to use a specific feature like the classdb you must initialize this manually recommended using init_class_db() and deinit_class_db() at the start and end of each function, you are responsible for initializing that portion of the engine in the test, if you require the renderer, you can see the logic required for this in main.cpp/main.h in main/ directory.
4. If you have questions check the doctest documentation then if you're stuck ping us on IRC we don't bite.
5. Remove ALL print logic from the tests, use INFO() when you need a print, but do not use this in place of a print, it's not required to print, just use CHECK() output of the tests is normal for INFO but not in a 'spammed context'

For safety, we actually initialized the os::singleton properly for you so that logging works fine.

Cool stuff we can now do which we couldn't before
./godot --test -d (profile your tests time)
./godot --test --reporters=xml,junit,console (report some details in other formats for CI or Coverage checkers)
./godot --test -s (show what checks are in the tests)

[RevoluPowered]

crash bug on vscode #40666

[Anutrix]

@RevoluPowered Maybe change those bullets to checkboxes.

[RevoluPowered]

An update:

• scons -j16 tools=yes tests=yes is required to build tests now :) Thanks @Xrayez for resurrecting your PR
• tests are now top level directory in tests/ by @Xrayez

[Xrayez]

I went through some existing tests to see the feasibility of them to be ported:

• test_astar: (ported in Port AStar tests to use doctest #40835).
• test_basis: (ported in Port Basis tests to use doctest. #40940).
• test_class_db: (ported in Port ClassDB tests to use doctest #40980).
• test_gdscript: don't port, likely going to be rewritten, see Enable support for C++ modules tests #40720 (comment).
• test_gui: no assertions found, just random code doing stuff, not sure how to test.
• ⚠️ test_math ⚠️: catastrophic, need a nuclear weapon to clean this up first 🙂, it seems like the code has accumulated random stuff over the span of Godot's existence. Seems to include code for math parsing, geometry (updated 3 months ago), various file access stuff, even platform specific code like: OS::get_singleton()->execute("/bin/ls");.
• test_oa_hash_map: ok, but has some stress tests.
• test_ordered_hash_map: (ported in Port OrderedHashMap tests to doctest #41117).
• test_physics_2d/3d and test_render: no asserts, just code, not sure how to test.
• test_shader_lang: need someone from core devs for best results, but likely can be ported without issues.

For stress tests I propose prepending "[Stress]" tags to test case names for now, so that they can be excluded to have deterministic results on GitHub's CI, I think those are more useful to run on separate instances to find more bugs rather than regressions.

[fire]

Porting basis currently.

[fire]

Basis is ported #40940

[3akev]

OrderedHashMap is ported in #41117

[Xrayez]

ClassDB tests ported in #40980. This opens up possibility to write unit tests for classes which inherit Object (directly or indirectly), so feel free to write some new tests for those!

[RevoluPowered]

test_fbx will be ported by me after we can get the 3.2 pr out of the way I've backed up the branch with the originals to RevoluPowered/backup_fbx_tests (they're in the old folder)

[Xrayez]

Preliminary documentation is written in godotengine/godot-docs#4017, once merged, should be available under Engine Development → Unit Testing.

---

Update: available at https://docs.godotengine.org/en/latest/development/cpp/unit_testing.html.