WIP: Conversions between Python's native (stdlib) enum and C++ enums.

[rwgk]

Description

Please ignore for now.

Suggested changelog entry:

[Skylion007]

@rwgk There has been a lot of discussion about doing this and other limitation with the enum class for a while btw: see especially #2332 and #1177 (for a long standing enum issue). Also see this gist from #2332 https://gist.github.com/anntzer/96f27c0b88634dbc61862d08bff46d10 . 2332 has a lot of ideas for how to do it properly and how people do it in their downstream project.

[rwgk]

@rwgk There has been a lot of discussion about doing this and other limitation with the enum class for a while btw: see especially #2332 and #1177 (for a long standing enum issue). Also see this gist from #2332 https://gist.github.com/anntzer/96f27c0b88634dbc61862d08bff46d10 . 2332 has a lot of ideas for how to do it properly and how people do it in their downstream project.

Thanks for the pointers. I was vaguely aware of #2332 but didn't actually look very much...

... because I basically don't have a choice: the native_enum is for PyCLIF compatibility. We have hundreds (hand-waving estimate) of extensions that use a native_enum equivalent. Global-test failures due to pybind11:enum_ vs native enum mismatches are one of the biggest remaining blockers for us.

Note that I'm not aiming to replace pybind11::enum_, but to add an alternative option.

I'll focus on making the native_enum implementation complete for all corner cases, then with that in hand

1. the PyCLIF-pybind11 integration has one blocker less,
2. we can step back and review what we want to do long term.

[Skylion007]

@rwgk There has been a lot of discussion about doing this and other limitation with the enum class for a while btw: see especially #2332 and #1177 (for a long standing enum issue). Also see this gist from #2332 https://gist.github.com/anntzer/96f27c0b88634dbc61862d08bff46d10 . 2332 has a lot of ideas for how to do it properly and how people do it in their downstream project.

Thanks for the pointers. I was vaguely aware of #2332 but didn't actually look very much...

... because I basically don't have a choice: the native_enum is for PyCLIF compatibility. We have hundreds (hand-waving estimate) of extensions that use a native_enum equivalent. Global-test failures due to pybind11:enum_ vs native enum mismatches are one of the biggest remaining blockers for us.

Note that I'm not aiming to replace pybind11::enum_, but to add an alternative option.

I'll focus on making the native_enum implementation complete for all corner cases, then with that in hand

1. the PyCLIF-pybind11 integration has one blocker less,

2. we can step back and review what we want to do long term.

Now that we only support Python >3.5, I don't see why we can't implement some of those changes to replace the old enum if needed.

[Skylion007]

Also, some of the proposed methods don't require an ABI break. The Python Enum objects are singletons so we just need to generate and map between them in the caster somehow.

[rwgk]

Also, some of the proposed methods don't require an ABI break. The Python Enum objects are singletons so we just need to generate and map between them in the caster somehow.

I think it needs to go into "something like internals", but not necessarily internals.

@wjakob I think it would be great to have #4293 (with the tiny #4307 tweak) merged, for the get_internals_state_dict() function, which I could reuse for something like get_native_enums_from_state_dict(). (Maybe a better name for the helper function would be get_python_state_dict().)

[rwgk]

EDIT: It turned out, fixing the new leak was much easier than expected:

commit 3debb51

Organize internals-like functionality as struct native_enum_type_map_v1, including cleanup in embed.h (no more leaking).

Note that the old leaks still exist.

---

This PR currently includes this code (in pybind11.h):

        // Intentionally leak Python reference.
        detail::get_native_enum_type_map()[data.enum_type_index] = py_enum.release().ptr();

This will only matter for embedding.

This small experiment shows that it currently does not even matter for embedding:

https://github.com/rwgk/issues/tree/0e7ebbda7494a3d1165aaf45523fad4c506199f7/pybind11/initialize_finalize_loop

That experiment shows:

• A simple initialize-finalize loop does not leak with Python 3.10.
• But if class_<>(...); is called it leaks substantially.

top output files are included under the link above.

Conclusion: Any effort trying to avoid the "Intentional leak" above would have no tangible benefit. There is a bigger general problem that would need to be solved first, and probably that solution would make it straightforward to also avoid this documented leak.

For future easy reference, this was the state @ master: https://github.com/pybind/pybind11/tree/296615ad34f9d8f680efbab22553881ad9843063