Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webworkers for io.js #180

Closed
iankronquist opened this issue Dec 18, 2014 · 9 comments
Closed

Webworkers for io.js #180

iankronquist opened this issue Dec 18, 2014 · 9 comments

Comments

@iankronquist
Copy link
Contributor

What exists and what is lacking

io.js, understandably, does not have any threading primitives, and gives the outward appearance of being single-threaded. There do exist multi-process primitives which act like popen. Threads and processes serve a very similar purpose but are fundamentally different.

Where webworkers fit

Implementing webworkers in io.js would allow developers to share code between the server and the client. Webworkers are designed for exactly the type of expensive and long running work which server side web applications often perform. It should be noted that webworkers are not part of any javascript standards. Instead, they are part of the HTML standard, and in some companies managed by the teams which work on the DOM.

Use in the community

There already exists an NPM package which emulates the behavior of webworkers. On the one hand, this indicates need from the community, on the other hand it indicates that there exists a method of using webworker APIs and sharing code, potentially obviating the need for a native code implementation. However, a native code implementation could be more efficient.
In a similar vein, there exists a plethora of packages for managing web workers, using web workers in iframes, and other purposes. Web workers are widely used by the JS community.
Members of the TC have indicated interest in integrating web workers into io.js.
I cannot find any bugs in the joyent/node bugtracker which reference web workers. This may indicate that web workers are not actually desired by the community, or that webworkers are a newly emerging standard which arose after node.

What belongs in io.js

Obviously the io.js community doesn't want to implement the monstrosities which lurk in the DOM complicated standards which do not serve the focus of the community. Nor do we want to implement things for which there already exist easy to use and powerful alternatives. However, webworkers offer a powerful, standardized API for running simultaneous javascript tasks.

@iankronquist
Copy link
Contributor Author

Apparently this wasn't a serious proposal. I'll close this.

@bnoordhuis
Copy link
Member

For the record, we're more likely to go down the path of full isolate support. That's more or less equivalent to the cluster module but with everything contained in a single process.

It's not on any roadmap because it's mainly an optimization, not a feature. It would allow optimizing operations like sending or receiving binary data between processes; instead of several read/write system calls, just a memory copy or maybe even a pointer swap would do.

We could hack something like that into the current cluster module if libuv supported shared memory; but it doesn't and making it work reliably on all platforms is not trivial.

@phpnode
Copy link

phpnode commented Dec 18, 2014

@bnoordhuis +1 for that optimisation - the poor performance of IPC in the cluster module is really unfortunate, it means that it's basically impossible to use things like leveldb effectively.
The problem: I want to create a data store with node, using leveldb as a storage backend. Leveldb can only be used by a single process, so ideally I'd run that in the master process and share it between N worker processes. The workers would do most of the work, dealing with things like accepting HTTP requests, query parsing, serialisation, validation etc, before sending the query off to leveldb and then formatting the response for the request. Unfortunately the IPC call that requires at the moment totally kills performance. The node leveldb community has come up with things like multilevel to hack around the issue but it's around 10x slower than using leveldb directly. Having a fast way to do this stuff would enable a lot of applications which are currently just too damn expensive.

@basarat
Copy link

basarat commented Mar 12, 2015

Is there something fundamental that prevents https://www.npmjs.com/package/webworker-threads from being rolled into iojs/node

@vkurchatkin
Copy link
Contributor

@basarat most threading packages on npm just spin up a new isolate inside a thread with some custom API for message passing. What is desirable is to have full-blown instance instead (event loop, require, etc)

@Samreay
Copy link

Samreay commented Aug 31, 2015

Has there been any movement on WebWorkers in io.js - any available implementations that work properly?

@Samreay
Copy link

Samreay commented Sep 6, 2015

Wow, that github thread is massive. I can't figure out if the change got merged in, whether it is still behind an experimental flag or not.

It would be perfect for me - at the moment I have my own pseudo-worker implementation using cluster.fork(), but the performance is roughly ten times slower in message passing that I see when using a web worker inside Chrome.

eti-p-doray pushed a commit to eti-p-doray/node that referenced this issue May 28, 2024
The new callback should return v8::Intercepted::kYes/kNo to indicate
whether the operation was intercepted. This replaces the old approach
where the callback had to leave the return value unset or set it to
an empty handle to indicate that the the request wasn't intercepted.

See https://crrev.com/c/5465509 and https://crrev.com/c/5465513.
# Conflicts:
#	src/node_contextify.cc
syg pushed a commit to syg/node that referenced this issue Jun 20, 2024
The new callback should return v8::Intercepted::kYes/kNo to indicate
whether the operation was intercepted. This replaces the old approach
where the callback had to leave the return value unset or set it to
an empty handle to indicate that the the request wasn't intercepted.

See https://crrev.com/c/5465509 and https://crrev.com/c/5465513.
# Conflicts:
#	src/node_contextify.cc

# Conflicts:
#	src/node_contextify.cc
#	src/node_env_var.cc
eti-p-doray pushed a commit to eti-p-doray/node that referenced this issue Aug 28, 2024
The new callback should return v8::Intercepted::kYes/kNo to indicate
whether the operation was intercepted. This replaces the old approach
where the callback had to leave the return value unset or set it to
an empty handle to indicate that the the request wasn't intercepted.

See https://crrev.com/c/5465509 and https://crrev.com/c/5465513.
# Conflicts:
#	src/node_contextify.cc

# Conflicts:
#	src/node_contextify.cc
#	src/node_env_var.cc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants
@bnoordhuis @phpnode @basarat @vkurchatkin @iankronquist @Samreay and others