Enable subclassing nb_type

[makslevental]

I would like to be able to subclass nb_type. Currently this is possible in pybind and I use this feature to perform some "magic" when a pybind class pops out from C++ (see here). The upstream project (MLIR) recently switched pybind -> nanobind so I'd like to restore this functionality.

Let me know if you'd like me to add a test case.

[wjakob]

Doing anything with the nb_type metaclasses is dangerous and intentionally disallowed. Only libnanobind can safely create new nanobind types. First of all, new instances need to be correctly registered in the internals data structure. Secondly, nanobind types also use more memory than standard Python types, which the library internally uses for bookkeeping. This requires a special allocation sequence and correct initialization of this supplemental data. All of this is orchestrated by the metaclasses, which are part of the secret sauce to make nanobind fast, and that also means that they may change in unpredictable ways in the future. This is why they are locked away.

Things that required certain workarounds in pybind11 should often be done differently (potentially more directly) in nanobind, so a direct 1:1 port of "magic" is generally not recommended. I suggest that you create a discussion post that explains your use case (ideally reduced a bit, the MLIR implementation file you posted was difficult to follow)