-
Notifications
You must be signed in to change notification settings - Fork 741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Prototype] Python stubs generation #2379
Conversation
Thanks for working on this! Have you got an example of a tool that would process these outputs into As this branch progresses I'd prefer it to be part of the |
@@ -0,0 +1,67 @@ | |||
use syn::{GenericArgument, PathArguments, Type}; | |||
|
|||
pub fn map_rust_type_to_python(rust: &Type) -> String { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than embedding this information into the macro, it would be cool if we could somehow use a trait to calculate this. I think that'd mean code generating a function (maybe as part of #[pymodule]
) which could be run to build stubs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, but I have no idea where to extract that information from.
I liked the compile-time approach because it would be easier to transparently include into Maturin (it could automatically enable the cargo feature, collect the generated files, merge them together and continue on without any code modifications), but I'm not yet sure that all information is available then.
Is your idea something like this?
trait PythonType {
fn python_type() -> &'static str
}
It would probably be easy to implement for all built-in PyO3 types, and I guess #[pyclass]
and #[derive(FromPyObject)]
can generate it as well.
I'm not sure how the other needed information (full method signature, etc) can be passed to the runtime though.
After including the
Originally I thought it would be good to have something like |
Looking at the |
Sorry for the delay. Do you mean that this generates the |
Yes. It generates pyi fragments that are unordered (they all start with Currently, it is able to export most of the class and method/function/attribute signatures, however the conversion from Rust types to Python types is hard-coded (and really ugly).
At the moment, it doesn't. I hope that it's possible to get the structure at compile-time, but I doubt it is. Ideally, I'd like: |
Ah, cool. I've thought about doing similar in the past, though I got a bit spooked. How does it handle developers making edits? Do the fragments have to be removed manually before each build?
I think the only way that this can be done reliably is to make a trait, and instead of generating fragments directly from the macros, generate code which then uses that trait to generate the fragments. Maybe we can also solve the module discovery at the same time, by writing a configuration file which says what
What do you think of that? |
If it has to work at runtime, would it be easier to just have a function I'm not really a fan of duplicating the module structure in code and in configuration, but I don't really know if we have a choice either. |
TBH that may well be the right approach, that way the user can check the generated Reminds me of https://matklad.github.io/2022/03/26/self-modifying-code.html (at least the part of running generation at test time). Having the file permanently present will probably work very nicely for development (rather than by generating at install time, local development may not work so well depending on how things are set up). In cases where the generated |
That sounds like a good solution for user interaction. Now the two missing problems... I do not know how stubs handle modules. My understanding is that the recommended way is to something like this:
What's the structure expected by Maturin? And the other question would be how to extract the information we want from the Rust code. Creating a trait seems fine but I haven't really used proc-macros before, any recommendations on how to get started? |
After looking at it some more, the main problem is linking the function metadata with their class. I can statically generate the required data for each function, but I don't see how to give it to the class. |
Closing in favor of #2447 which is much cleaner. |
Hi, this is a prototype to see if it's possible to generate a Python stub (.pyi) automatically from PyO3.
This is a very early prototype and definitely should not be merged as-is, but I'd be grateful for feedback on how to improve it.
The idea would be to generate parts of the stub as PyO3 parses the different classes, then to use an external program (e.g. included in Maturin) to combine the parts into a single file. Currently the parts are printed to stdout for ease of debugging.