Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark for nested/wide model using definitions #997

Merged
merged 2 commits into from
Sep 30, 2023

Conversation

adriangb
Copy link
Member

No description provided.

@adriangb adriangb enabled auto-merge (squash) September 30, 2023 02:22
@adriangb adriangb disabled auto-merge September 30, 2023 02:24
@codecov
Copy link

codecov bot commented Sep 30, 2023

Codecov Report

Merging #997 (82d194b) into main (f3c5714) will decrease coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 82d194b differs from pull request most recent head 4eeec88. Consider uploading reports for the commit 4eeec88 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #997      +/-   ##
==========================================
- Coverage   93.04%   93.03%   -0.01%     
==========================================
  Files         106      106              
  Lines       15701    15701              
  Branches       35       35              
==========================================
- Hits        14609    14608       -1     
- Misses       1085     1086       +1     
  Partials        7        7              

see 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3c5714...4eeec88. Read the comment docs.

@codspeed-hq
Copy link

codspeed-hq bot commented Sep 30, 2023

CodSpeed Performance Report

Merging #997 will degrade performances by 24.83%

Comparing add-nested-benchmark (4eeec88) with main (f3c5714)

Summary

❌ 1 regressions
✅ 137 untouched benchmarks

🆕 2 new benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main add-nested-benchmark Change
test_dont_raise_error 29.7 µs 39.5 µs -24.83%
🆕 test_nested_schema_using_defs N/A 75.6 ms N/A
🆕 test_nested_schema_inlined N/A 68.9 ms N/A

@adriangb adriangb enabled auto-merge (squash) September 30, 2023 03:00
@adriangb adriangb merged commit 49126b0 into main Sep 30, 2023
@adriangb adriangb deleted the add-nested-benchmark branch September 30, 2023 03:05
@adriangb
Copy link
Member Author

@davidhewitt I noticed the Rust benchmarks report wild variations like 9,477,991 ns/iter (+/- 87,410,750). Any idea what could be causing that? The Python versions seem to run with a reasonable duration / variation ratio, I'd expect the Rust version to be even better!

@davidhewitt
Copy link
Contributor

If I had to guess without actually profiling, it may be related to PyO3/pyo3#1056. If we don't explicitly clear the PyO3 pool on each iteration then the Python GC will probably jam up on increasingly large collections of garbage which can't be freed.

I'm so looking forward to shipping PyO3 0.20 and getting on with reworking PyO3's API 😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants