Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docstring code formatter: add benchmarks #8909

Open
BurntSushi opened this issue Nov 29, 2023 · 0 comments
Open

docstring code formatter: add benchmarks #8909

BurntSushi opened this issue Nov 29, 2023 · 0 comments
Labels
docstring Related to docstring linting or formatting formatter Related to the formatter performance Potential performance improvement

Comments

@BurntSushi
Copy link
Member

BurntSushi commented Nov 29, 2023

There is some concern over whether the new docstring code snippet formatting could slow down the overall formatting process. To ameliorate this and understand its performance characteristics, we should try to write some micro-benchmarks that will help us track performance over time. @MichaReiser in particular had some suggestions here: #8857 (comment)

I think we should try to include some "real world" examples of code snippets, but it might also be useful to try to include some "tortured" examples too. That is, try to devise the absolute worst case for performance and see how it does. This will give us an idea of what perf might look like at the extremes. For example, this might help us characterize the costs of us checking whether the reformatted code is valid or not. See #8857.

At present, I have done some very loose ad hoc benchmarking to get a general sense of things:

For an ad hoc benchmark, if I run the formatter with and without docstring-code enabled, then runtime is about the same. I ran it a few times and couldn't notice a difference. (Not that this is a good substitute for a real benchmark, but perhaps suggestive that docstring code formatting doesn't have a huge impact on perf.)

In particular, I did that on CPython and polars.

@BurntSushi BurntSushi added performance Potential performance improvement docstring Related to docstring linting or formatting formatter Related to the formatter labels Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docstring Related to docstring linting or formatting formatter Related to the formatter performance Potential performance improvement
Projects
None yet
Development

No branches or pull requests

1 participant