Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce CPU usage by implementing custom Copy funcs for schema #509

Closed
radeksimko opened this issue May 13, 2021 · 2 comments · Fixed by #513
Closed

Reduce CPU usage by implementing custom Copy funcs for schema #509

radeksimko opened this issue May 13, 2021 · 2 comments · Fixed by #513
Labels
enhancement New feature or request

Comments

@radeksimko
Copy link
Member

Current Version

0.16.2

Background

A schema cache was introduced as part of 0.16.0 release. memdb was also introduced to make operations above the state easier to handle. One side-effect of these changes was that we would create a full copy of every provider schema before storing that into memdb. This should have negligible impact on memory usage (as GC should cleanup the original copy), however because we make the copy using copystructure which in turn makes heavy use of the reflect library, copying these schemas consumes significant amount of CPU.

This can be seen from CPU profiles collected by @mattduguid as part of a debugging session per hashicorp/vscode-terraform#591 , where about 70% of the CPU is spent in schemas.PreloadSchemasToStore() and a big part of those 70% is spent in reflectwalk.

Screenshot 2021-05-13 at 12 00 57

Use-cases

Reduce CPU spikes upon initial language server launch.

Proposal

Investigate whether replacing copystructure with explicit Copy() implementations on schema structures would help reduce the CPU usage as Go won't have to use the (presumably) costly reflection at runtime.

@radeksimko radeksimko added the enhancement New feature or request label May 13, 2021
@radeksimko
Copy link
Member Author

FWIW I was able to obtain a very similar profile on my Mac with fresh 0.16.2 - at least percentage-wise. I'm unsure why the total CPU time spent is about half, but the profile clearly points to the same bottleneck.

Screenshot 2021-05-13 at 12 07 52

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant