Is it any alternative to run deno http server in cluster mode?

[devalexqt]

Is it possible to run deno http server on all available cpu?

[ry]

No, we don't have that functionality, yet, unfortunately.

[alxtsg]

Does using Worker count? Here is a prototype:

app.ts:

const START_PORT = 8000;
// Adjust to the number of CPU cores.
const WORKER_COUNT = 4;
for (let i = 0; i < WORKER_COUNT; i += 1) {
  const worker = new Worker(
    "./server_worker.ts",
    {
      type: "module",
      deno: true,
    },
  );
  worker.postMessage({
    port: START_PORT + i,
  });
}

server_worker.ts:

import { serve } from "https://deno.land/std@0.54.0/http/server.ts";

self.onmessage = async (e) => {
  const { port } = e.data;
  const server = serve({
    hostname: "127.0.0.1",
    port,
  });
  console.log(`Started an HTTP server listening on port ${port}.`);
  for await (const req of server) {
    req.respond({ body: "Hello World\n" });
  }
};

Start the server:

# Running Deno in worker is an unstable feature, so we need to add the
# --unstable flag.
deno run --allow-read --allow-net --unstable app.ts

In the example above, 4 HTTP servers will be started.

Started new HTTP server listening on port 8001.
Started new HTTP server listening on port 8000.
Started new HTTP server listening on port 8002.
Started new HTTP server listening on port 8003.

You can put a load balancer in front of the application and distribute the requests to each of the HTTP servers.

This is not the same as using cluster in Node.js to start multiple HTTP servers, because the HTTP servers in the Workers cannot listen on the same port.

[devalexqt]

Thanks for reply, but do You guys planing add this features or its out of focus for now?

[devalexqt]

"same port" is a key

[alxtsg]

See #3403 and #5377, looks like not much progress on that area yet.

[ry]

@devalexqt We don’t have plans to add that feature - but I’d be open to discussing how it could be done with web workers. Basically we need the ability to send resources to other workers - then you would share the server resource...

[devalexqt]

Yea, resource sharing with workers will be elegant solution!

[ngot]

I had done a similar cluster module using the worker feature in another JS runtime called fibjs. Transferring an object's ownership to another isolation(or call it worker) is the key point. (fibjs only supports transfer network socket and Buffer, tbh, socket is enough)

This is the module I wrote https://github.com/fibjs-modules/cluster-server#usage, the key implementation is https://github.com/fibjs-modules/cluster-server/blob/master/lib/cluster.js#L92

According to

• https://developer.mozilla.org/en-US/docs/Web/API/Worker/postMessage
• https://nodejs.org/api/worker_threads.html#worker_threads_port_postmessage_value_transferlist
  Both Browser and Node support passing Transferable objects. But it seems they don't support transfering Socket object.

Anyway, if it is possible to support transfer Deno.Conn(or something smiliar) between workers, it will be feasible to implement cluster.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kedicesur]

@ry What @alxtsg suggested may work by starting a load balancer ahead. Such as gobetween which is very simple. I just tried it with 4 sessions of denotrain working simultaneously on localhost at 4 different ports and req/sec increased from ~17000 req/sec to ~55,000 req/sec when tested with wrk.

[abaldawa]

@alxtsg What's the point of starting HTTP servers within workers if you are gonna put a manual load balancer in the front? Just start 4 separate Deno HTTP server processes and manually put load balancer on the front then for even better performance. The whole point of cluster module was they do port sharing and load balancing for you. If Deno does not have that then there is no point in using workers to start listening on separate port when workers are not as efficient in IO as main thread.

[alxtsg]

@abaldawa It depends on what you want to do and the requirements. Maybe in some cases you can't put a load balancer in front of the application for whatever reasons.

[matthewharwood]

Cluster allows to vertical scale compute e.g 1 instance with many cores. We've been doing significant work on node and found that vertical scaling an instance with 6 core and say 4 horizontal instances with load balancing has significant improvements in RPS and latency than 1 core with 24 instances and load balancing. The trade off is the memory increases but it's worth it

[devalexqt]

I agree what in production we need to have load balancer in front of actual service (for our projects we use nginx ) and then just spawn local js servers on different ports as many as we need. Also, all static assets, uploads and downloads served by nginx side because it's already have good builtin workers and we don't hit our js server for each request to just send staic files... As result js server only respond to API requests .