Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decode reports from a byte buffer #2386

Merged
merged 3 commits into from
Nov 30, 2017
Merged

Decode reports from a byte buffer #2386

merged 3 commits into from
Nov 30, 2017

Conversation

bboreham
Copy link
Collaborator

This is speculative: empirically it goes faster but theoretically it could make things worse.

So I would like to try it out by merging into master hence into dev, and if we don't like it we can revert.

@rade
Copy link
Member

rade commented Mar 24, 2017

So I would like to try it out by merging into master hence into dev, and if we don't like it we can revert.

The cleaner way to do this is to de-automate the relevant services in flux, and then deploy an image from this branch. cc @2opremio who has done this a few times.

@2opremio
Copy link
Contributor

Yep, I would rather do that before merging. @bboreham mind giving it a try? I can guide you. It basically entails:

  1. Modifying flux's config in dev to stop deploying scope automatically
  2. Deploying you image manually.

rade and others added 3 commits November 29, 2017 21:48
this new version is less efficient... but not for much longer
Reading and uncompressing the entire message into memory first allows
the decoder to avoid memory allocations in field names, etc.
Copy link
Member

@rade rade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feeding 30s of prod reports to the benchmark with

go test -tags 'netgo unsafe' -run=x -cpu=2 -bench=ReportUnmarshal --bench-report-path=/tmp/prod-20171129/

cdbc01e shows

BenchmarkReportUnmarshal-2   	       1	23049267656 ns/op	1239968760 B/op	21422077 allocs/op

and the final commit

BenchmarkReportUnmarshal-2   	       1	11998462779 ns/op	2679079128 B/op	19529968 allocs/op

That's a massive improvement!

@bboreham bboreham merged commit d6e47c0 into master Nov 30, 2017
@rade rade deleted the decode-bytes branch December 25, 2017 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants