Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep the stores open all the time #665

Merged
merged 2 commits into from
Feb 4, 2020
Merged

Keep the stores open all the time #665

merged 2 commits into from
Feb 4, 2020

Conversation

badboy
Copy link
Member

@badboy badboy commented Jan 21, 2020

A "store" is a handle to the database.
Glean has 3 different ones and can keep them open throughout its lifetime.

Not frequently re-opening database handles also might be a performance
improvement.

Extracted from #664 because it's unrelated. Not directly related to a filed bug.

@auto-assign auto-assign bot requested a review from travis79 January 21, 2020 13:19
@codecov-io
Copy link

codecov-io commented Jan 21, 2020

Codecov Report

Merging #665 into master will increase coverage by 0.03%.
The diff coverage is 55.26%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #665      +/-   ##
============================================
+ Coverage     74.65%   74.69%   +0.03%     
  Complexity      266      266              
============================================
  Files            96       96              
  Lines          5926     5931       +5     
  Branches        730      725       -5     
============================================
+ Hits           4424     4430       +6     
- Misses          966      969       +3     
+ Partials        536      532       -4
Impacted Files Coverage Δ Complexity Δ
glean-core/src/database/mod.rs 62.79% <55.26%> (+0.49%) 0 <0> (ø) ⬇️
...etry/glean/private/CustomDistributionMetricType.kt 97.67% <0%> (+0.05%) 12% <0%> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53fb434...6c8e416. Read the comment docs.

Copy link
Member

@travis79 travis79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, what was the performance gain in doing this (or do we know yet)?

@badboy
Copy link
Member Author

badboy commented Jan 22, 2020

We didn't know at submission time, it was mostly a hunch from Dexter.
I did ran a very simplistic benchmark (setting 2 metrics and submitting a ping) and get reliable 5% improvements compared to what's in current master.

glean set and submit    time:   [94.437 us 94.502 us 94.577 us]
                        change: [-5.3865% -5.1006% -4.8247%] (p = 0.00 < 0.05)
                        Performance has improved.

That is on my Linux machine (on my Mac it's unreliable and significantly slower, milliseconds instead of microseconds)

I'll work on adding the benchmark as part of this as well.

A "store" is a handle to the database.
Glean has 3 different ones and can keep them open throughout its lifetime.

Not frequently re-opening database handles also might be a performance
improvement.
@badboy badboy force-pushed the leave-it-open branch 2 times, most recently from b0cf598 to 6c8e416 Compare January 22, 2020 13:33
@badboy
Copy link
Member Author

badboy commented Jan 22, 2020

So the numbers are unreliable even on my Linux workstation. The benchmark is also very simplistic and nowhere near a real use case.
Nonetheless I included the benchmark now as an extra crate so we can re-run/extend it if needed.

@badboy badboy requested a review from travis79 January 22, 2020 13:36
@badboy badboy added the blocked Blocked pull requests and issues label Jan 23, 2020
@badboy
Copy link
Member Author

badboy commented Jan 23, 2020

holding back on this to investigate what this means for I/O operations.

@badboy badboy removed the blocked Blocked pull requests and issues label Jan 24, 2020
@badboy
Copy link
Member Author

badboy commented Jan 24, 2020

All good, I/O operations unchanged for the startup case.

/cc @travis79, please give it a final review

Copy link
Member

@travis79 travis79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor nit but looks good to me!

glean-core/benchmark/README.md Outdated Show resolved Hide resolved
Currently it only contains one very simplistic benchmark:
Setting 2 metrics and submitting a custom ping.

Why an additional crate?
This way we don't add any new (dev) dependencies to the crates that get released.
@badboy badboy merged commit 1a8c347 into master Feb 4, 2020
@badboy badboy deleted the leave-it-open branch February 4, 2020 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants