Optimising connections per second

[v-flashpoint]

Hi all,

I'm using uWebSockets.js to run a quiz platform. Due to the nature of the beast, most people are only going to connect in the minutes/seconds running up to the quiz, so I've been trying to optimise the number of clients we can get connected per second.

We're using SSL (as I don't know a way to get around it from a HTTPS site!), and in testing at the moment we've been able to connect 300 clients every second on an m5a.2xlarge on AWS. The only thing we do on connecting a client is make a call to redis to get that player's information - there's very little in the way of expensive logic happening in our benchmark demo. At a rate of 300 clients per second, node is using almost 100% of its core.

What I was wondering was, is there anything we can do to optimise how quickly we can open new connections? In a best case scenario, how many connections per second should we be expecting (are there any benchmarks I've missed)?

Thanks!

[e3dio]

SSL handshake is CPU expensive, you are running on m5a.2xlarge which is 8 vCPU and you are only using 1 thread, this is where more cores helps, add a uWS.js thread for each vCPU. It is more effort than 1 thread but can get you better performance x8, and Node makes it easy

[e3dio]

Another option you could try which I use is this open source Caddy proxy https://github.com/caddyserver/caddy which specializes in SSL/TLS termination and automatic certificates written in Go so it should spread work across all cores, pass that traffic to uWS.js app and see what numbers you get

[joshxyzhimself]

Using Caddy is one of our favourite solutions.

• Automated TLS of course
• Can easily switch between TLS 1.2 & TLS 1.3
• Can easily route by origin to specific backend servers
• Can easily route /pathnames to other backend servers
• Can modify headers

Maybe even better if you benchmark the other ones, e.g. HAProxy, NGINX, Traefik, Envoy

https://www.loggly.com/blog/benchmarking-5-popular-load-balancers-nginx-haproxy-envoy-traefik-and-alb/

[uNetworkingAB]

300 per second sounds pretty reasonable (I think?). SSL handshakes are seriously CPU-intensive.

[tayler-king]

300 per second sounds pretty reasonable (I think?). SSL handshakes are seriously CPU-intensive.

In which case, what would be your recommended approach for queuing upgrade requests?

Sudden bursts of traffic seeing up to 1000s of req/s can be a pain to deal with as it seems to cause socket connections to hang.