Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Transmit reports using gob with websockets ... #957

Closed
wants to merge 2 commits into from

Conversation

2opremio
Copy link
Contributor

... with the intention of caching the gob engines.

Very hacky for now, just a proof of concept.

As an alternative to improve #854

... with the intention of caching the gob engines.

Very hacky, just a proof of concept.
@2opremio
Copy link
Contributor Author

Unfortunately this doesn't seem to help at all. When running Scope locally (with a single app and probe), it takes double as much CPU as the generated JSON decoders ( #916 ) and I get this app CPU profile:

pprof.172.16.0.3:4040.samples.cpu.011.pb.gz

app

Surprisingly, even if a single encoding/gob.Decoder is used, compileDec() (which creates the gob engines) takes 40% of the execution time. This isn't expected, since the engines should be cached within the Decoder.

@2opremio 2opremio changed the title [WIP] Use gob with websockets ... [WIP] Transmit reports using gob with websockets ... Feb 11, 2016
@2opremio
Copy link
Contributor Author

Surprisingly, even if a single encoding/gob.Decoder is used ...

This is not true. LatestMap, Counters etc ... all provide their own GobEncode()/GobDecode() functions, which in turn create buffers and their own encoders/decoders. Let's see if I can pass the top-level encoder/decoder. I believe this should also help quite a bit with garbage collection.

This is also the case for JSON, which is probably hurting performance in #916 too

@2opremio 2opremio mentioned this pull request Feb 11, 2016
4 tasks
@2opremio
Copy link
Contributor Author

There isn't a way to pass in the same encoder. Using encoding/gob.Register to avoid intermediate representations for ps.Map (11d517a) didn't fly either. I ended up getting:

runtime: goroutine stack exceeds 1000000000-byte limit

In a nutshell, gob not only sucks with used in request-response communications (which admittedly wasn't designed for) but also when you want to serialize intermediate representations of your types/interfaces.

@2opremio
Copy link
Contributor Author

Closing in favor of #916

@2opremio 2opremio closed this Feb 16, 2016
@2opremio
Copy link
Contributor Author

2opremio commented Mar 3, 2016

PS: it seems like the Golang maintainers don't have any plans to improve the interface in order to allow more efficient serialization of custom intermediate types. https://groups.google.com/forum/#!topic/golang-nuts/BQsVtMiu0W4

@tomwilkie tomwilkie deleted the 854-gob-ws branch April 19, 2016 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant