Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add derived vs. non-derived benchmark #96

Merged
merged 1 commit into from
Nov 9, 2015

Conversation

travisbrown
Copy link
Member

I've been meaning to add a benchmark comparing derived and non-derived codecs for a while, and this question from @adelbertc was a good reminder.

The non-derived decoders and encoders are intended to be idiomatic, and one of the interesting things the benchmarks show is that Json.obj (used in the non-derived encoder but not the derived one) needs some work—the derived encoder is actually faster than the hand-written one. The throughput of the derived decoder is about 3% lower than the non-derived one, which is within the margin of error.

Benchmark                                   Mode  Cnt        Score       Error  Units
CirceDerivationBenchmark.decodeDerived     thrpt   80   788832.569 ± 11653.684  ops/s
CirceDerivationBenchmark.decodeNonDerived  thrpt   80   813443.980 ± 13189.326  ops/s
CirceDerivationBenchmark.encodeDerived     thrpt   80  1010433.746 ± 12266.961  ops/s
CirceDerivationBenchmark.encodeNonDerived  thrpt   80   975425.816 ±  9003.887  ops/s

The derived decoders and encoders also look pretty good in terms of allocations:

Benchmark                                                       Mode  Cnt        Score       Error   Units
CirceDerivationBenchmark.decodeDerived:gc.alloc.rate.norm      thrpt   20     4984.002 ±    35.636    B/op
CirceDerivationBenchmark.decodeNonDerived:gc.alloc.rate.norm   thrpt   20     4844.002 ±   110.473    B/op
CirceDerivationBenchmark.encodeDerived:gc.alloc.rate.norm      thrpt   20     2752.002 ±     0.003    B/op
CirceDerivationBenchmark.encodeNonDerived:gc.alloc.rate.norm   thrpt   20     2992.002 ±     0.003    B/op

So performance probably isn't usually a good reason to avoid generic derivation here.

@codecov-io
Copy link

Current coverage is 70.06%

Merging #96 into master will increase coverage by +0.79% as of 967a10e

Powered by Codecov. Updated on successful CI builds.

travisbrown added a commit that referenced this pull request Nov 9, 2015
@travisbrown travisbrown merged commit 3a8f271 into master Nov 9, 2015
@travisbrown travisbrown deleted the topic/hand-rolled-benchmark branch November 12, 2015 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants