Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Compression of SSE message data #479

Closed
nickchomey opened this issue Jan 13, 2025 · 8 comments
Closed

Support for Compression of SSE message data #479

nickchomey opened this issue Jan 13, 2025 · 8 comments

Comments

@nickchomey
Copy link

nickchomey commented Jan 13, 2025

I just messed around with the TS SDK and added support for the CompressionStream API (gzip, deflate, deflate-raw) and tested it on some html fragments - small and large. It makes a very meaningful difference.

First, all three have the same compression level. The difference seems to be with regards to checksums or something - gzip being the most robust, deflate some, deflate-raw having none.

It also seems that, at least with the crude implementation, deflate-raw isnt actually usable by D*/the browser (perhaps needs extra processing/parsing due to the lack of the framing data).

Stream of 100 fragments

  • compressed: 12.3 kB
  • uncompressed; 57.4 kB

Single bulk fragment of the 100 fragments:

  • compressed: 10.4 kB
  • uncompressed; 51.8 kB

Speed differences are negligible between compressed/uncompressed. Though, interestingly, the streamed version is MUCH slower than the bulk - I suppose that makes sense, given that it is sending a new message for each one. I assume that the time doesnt include idiomorph either. I suppose you'd generally want to use bulk unless each fragment chunk is very expensive to generate.

Here's a screen recording of it.

chrome_MqVYpccOHu.mp4

I'm not sure how languages other than TS/JS would implement this, since it is using the CompressionStream API. But perhaps they can just use whatever gzip/deflate compression is native to the language and the browser knows what to do with it if the content-encoding header is appropriately set to deflate/gzip.

I'm happy to share the mess of code that I have if there's interest in this. Its definitely not the basis for an sdk implementation, but the idea is correct, simple and effective.

@Superpat
Copy link
Contributor

I've seen some user libraries that minify html fragments and remove comments from executescript code. I think compression is probably a better tactic for reducing event sizes then those other methods.

But I'm a bit confused by the video, it seems to be showing that the only real difference is with a bulk update vs multiple updates and that even if the uncompressed updates are 5 times larger, they still transfer in about the same time. Can you showcase something that demonstrates a real difference in query time between compressed and uncompressed updates ? I'm curious how big a fragment needs to be for it be noticeable.

@nickchomey
Copy link
Author

nickchomey commented Jan 13, 2025

I intend to Minify AND compress. They're separate, but compatible/harmonious things.

Bandwidth is what is important here, not time. And these fragments were sufficiently large to show an impact (5x)

Though, this is localhost with unlimited bandwidth/speed. If a mobile phone on a metered 3G connection is making the request, less data will absolutely make a big performance and wallet impact.

Does that clarify things?

p.s. if anyone tries to say that 3G and metered/expensive bandwidth dont exist/matter, youre disgracefully wrong. I'll let these articles speak for me https://infrequently.org/series/performance-inequality/ The 2025 edition will be coming out soon, and will say much of the same (https://webperf.social/@slightlyoff@toot.cafe/113802946434336908)

@Superpat
Copy link
Contributor

Ah I see, I thought you where talking about time, not bandwith, hence the confusion.

@nickchomey
Copy link
Author

nickchomey commented Jan 13, 2025

Again, on slow connections (which is most of the world), bandwidth and time are the same thing. Its only on high speed connections - which even rich people have to suffer regularly - where this doesn't matter.

That article series (and all his other posts) is tremendous. Cant recommend it more highly

@nickchomey
Copy link
Author

nickchomey commented Jan 13, 2025

Delaney made some good points in discord - I think i continue to fail to fully appreciate how SSE is just HTTP. Therefore, any reverse proxy - caddy, nginx, cloudflare, etc... - can just compress SSE traffic as it comes through and then the browser decompresses as it normally would. Much easier (and probably more performant) to just let it happen at that layer rather than in the application, let alone each of the D* sdks.

He said he does this and gets brotli and zstd compression as well. I'll start experimenting with caddy and report back with what I find.

There can be some gotchas with it though - they apparently tend to buffer the responses before compressing, so if you are sending small responses (a small html fragment or signal), they might mysteriously get delayed. That might be an argument for allowing it in the application layer, but surely not strong enough to warrant the effort

@nickchomey
Copy link
Author

nickchomey commented Jan 14, 2025

So i have tested it proxied through Caddy. The app is different as I reverted to a different commit and then just implemented a the fragment streaming.

Anywhere, here's the results. The first in each compression pair is stream, 2nd is bulk

image

Its going vastly slower for the streamed version - not sure if that's caddy or just the app being different. But the compression ratio is VASTLY higher for both gzip and zstd than when it was implemented in TS. Both are comparable though in terms of speed and compression.

I'll see if I can recover the previous version and test it out to isolate the performance difference to the caddy or the app change

@nickchomey
Copy link
Author

nickchomey commented Jan 14, 2025

I salvaged the compression version and did uncompressed requests. Evidently somethign was weird in the new app implementation to make the streamed response times 1.6s. Now its the same 650ms as before.

Gzip and zstd are still extremely small - much smaller than the TS compression impementation.

image

Very rough conclusions:

  1. best to do compression in the proxy layer - dead-simple and more compression.
  2. And also best to just batch the fragments in the app and send as one big fragment rather than stream them as they become ready (unless, perhaps, they are very costly to generate). And i have to figure the timing shown doesnt take idiomorph into account, which would surely be much slower for 100 small fragments than 1 big fragment (especially since they're all just an insert rather than an actual merge).

Caddy has minify modules, but i couldnt be bothered to make a custom build to test it. But i see no reason why it couldnt minify then compress.

Closing the issue now

@nickchomey
Copy link
Author

nickchomey commented Jan 14, 2025

Perhaps the real conclusion, though, is just that its not something that should be implemented in the sdks. Because there's plenty of reasons to not want/need a proxy.

A good example is the D* Bad Apple example - https://data-star.dev/examples/bad_apple (code)

It does compression via a Go package in the application. Zstd reduces the ascii video size about 10x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants