Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workers: initial implementation #1159

Closed
wants to merge 389 commits into from

Conversation

petkaantonov
Copy link
Contributor

Workers are light-weight processes that are backed by OS threads.

This is an initial implementation that is behind the flag --experimental-workers. It should have very little effect on other io.js components, even if the flag is enabled. The module can be required by doing var Worker = require('worker'); - see how the tests do it in test/workers folder.


What is currently implemented:

Worker constructor which takes an entry module and an optional options object. Currently (I have some ideas for more) the only option is keepAlive:

var worker = new Worker('worker.js', {keepAlive: false})

The option defaults to true, which means the worker thread will stay alive even when its event loop is completely empty. This is because you might not have
anything to send to the worker right away when a server is started for instance.

A worker object is an event emitter, with the events 'message', 'exit', and 'error' and public methods postMessage, terminate, ref and unref:

  • 'message' event will receive currently a single argument, that is directly the message posted from the worker thread.
  • 'exit' event is emitted when the worker thread exits and receives the exit code as argument.
  • 'error' event is emitted when there is an uncaught exception in the worker which would abort a normal node process. The argument is the error thrown,
    if it's an error object, its builtin type, stack, name and message are retained in the copy. It will additionally contain extra properties that were in the
    original error, such as .code.
  • postMessage([message]) where message is structured clonable* can be used to send a message to the worker thread. Additionally internal messaging support is implemented
    that doesn't need to copy data or go through JS. (* currently implemented as JSON to keep PR small, more details later)
  • terminate([callback]) terminates the worker by using V8 TerminateExecution and uv_async_send, so the only thing that
    can defer a worker's termination is when it is executing C code which shouldn't contain any infinite loop or such :-).
  • unref and ref work like timer's respective methods, however keep in mind that a worker will terminate if its owner terminates.

Inside a worker thread, several process-wide features are disabled for example setting the process' current working directory, title, environment variables or umask. Workers can still read these, but only the main thread can change them. Otherwise all core modules and javascript works normally inside a worker, including console.log.

Inside worker thread, the Worker constructor object is an event emitter, with the event 'message' and the method postMessage() to communicate with its owner (see tests for usage). The worker constructor can be used to construct nested workers with the worker being their owner. Passing messages from grand child to grand parent requires the child's parent to pass its message through, however transferable message ports can be implemented if use cases emerge.

Nested workers were implemented because Web Workers support them too (except Chrome) and I envision that a setup with ${CPU cores} amount of web servers with N amounts of sub-workers for each will be very useful.

2 new public process read-only properties are introduced process.isMainInstance and process.isWorkerInstance.


Advantages of workers over processes:

  • Internal ITC (not passing JS objects) is ridiculously fast with the wait-free message channel
  • User ITC is probably faster as well, not having the overhead of JSON parsing and serialiazing
    • And can also transfer more kinds of objects, such as Dates, Maps, Sets, objects with circular references...
    • Additionally ownership transfer of external resources like ArrayBuffers and external strings can be done without copying
    • Ownership transfer of uv handles will probably require support from uv so that handles can be detached from an event loop
      and adopted by another.
  • Threads have less context-switch overhead, making it feasible to run many more workers than there are CPU cores
    which is more graceful in high load situations. And if we implement worker-cluster, there shouldn't be the need
    to use home-made round-robing scheduler.
  • Free of problems like process.send not always being synchronous
  • There is just one process to kill and no runaway processes

Disadvantages:

  • Cannot rely on OS to free resources upon worker exit. However the architecture that
    enables freeing all resources upon Environment destruction is there and
    is already used to free all resources that a new worker that doesn't do
    anything but exit immediately allocates.
  • C code maintainers need to think about thread-safety. However, there is generally
    little amount of shared resources as most stuff is tied to a v8 isolate or a uv event loop
    which are exclusive to a worker.

Compared to traditional threading:

  • Workers are expensive to create, one does not simply spin up a worker to do 1 task and throw it away.
  • Workers cannot share any JS memory (this is impossible in V8), to JS users they appear pretty much as isolated as real processes

To keep the PR size small (believe it or not, deep copying will take much more code than workers alone), objects are still copied by JSON serialization. I want to implement something like the structured clone with some differences as the spec has some seriously nasty parts that are not really needed in practice but force the implementation to be so slow as to defeat the whole point of using it in the first place.

So what cannot be implemented from the algorithm:

  • The html5 objects like File (since we don't have those)
  • Named properties of Arrays (inconsistent and extremely nasty to implement using v8's api)
  • (implicit) If you have sparse array and define indexed properties in protototypes, then those values in prototypes are used in the resulting array. V8 has internal check that array and object prototypes are clean of indexed properties, but we don't have access to that.

What could be implemented but don't want to implement from the algorithm:

  • Normal properties of Map and Set. This would be inconsistent and add more stuff to implement, but certainly doable.

What I mean by inconsistent is that copying map, set and array properties is inconsistent because the properties of Date, RegExp, Boolean, Number, and String objects are dropped.


To make reviewing this easier, a description of changes in each file follows:

  • *test/workers/**_, _tools/test.py, Makefile
    • For testing. Run make test-workers. Worker tests are not ran as a part of make test
  • worker.cc, worker.h, worker.js
    • Majority of the worker implementation is here
  • util.h, util-inl.h, util.cc
    • Added STATIC_ASSERT
    • Added some threading helpers
    • Added a hack to get a char* from js string (TODO)
    • Added Remove method to ListHead
  • smalloc.cc, smalloc.h
    • Extracted CallbackInfo class declaration to a header
    • Moved ALLOC_ID to env.h
  • producer-consumer-queue.h
    • Modified folly's wait-free single-producer single-consumer queue.
  • persistent-handle-cleanup.cc, persistent-handle-cleanup.h.
    • Because finalizers are not guaranteed to be called even when a v8 isolate is disposed, these implement PersistentHandleVisitor that is used to walk all unfinalized weak persistent handles to release their C resources upon Environment destruction
  • notification-channel.cc, notification-channel.h
    • These work around uv_async_send unreliability when async handles need to be unreffed
  • node-internals.h
    • Removed NodeInstanceData - the event loop code for worker became too different from the main thread event loop code for this to be useful
  • node-contextify.cc
    • Extracted out ContextifyScript class declaration to a header. The diff looks horrible but there is no real changes
    • Added check to prevent cancellation of termination in EvalMachine method
  • node.js
    • Added worker mode startup and cleanup registration for stdio and signal handles. (environment->CleanupHandles() is actually unused by main instance, but workers use it)
  • node.cc
    • Added locking around process-wide methods. (If uv's rwlock still causes deadlocks in Windows 2003, it should be changed to mutex).
    • Added worker-only locking for module initialization, the ApiLock() is null for main instance but workers need it because module initialization isn't termination safe.
    • Made MakeCallback safe to call inside workers that are being abruptly terminated
    • Changed all exit calls to env->Exit() (which only exits the process if called from main thread)
    • Added an export for process._registerHandleCleanup()
    • Moved atexit from LoadEnvironment to Init (probably not the right place, but it cannot be in LoadEnvironment)
    • Added CreateEnvironment overload to be called from worker threads
    • Extracted Environment initialization to InitializeEnvironment
    • Removed all worker checks from StartNodeInstance and renamed it to RunMainInstance. Added a call to TerminateSubWorkers() before exit event is emitted.
    • Added ShutdownPlatform and delete platform at the end of Start method.
  • handle_wrap.cc, handle_wrap.h
    • Added a way to register handle cleanups upon Environment destruction.
  • env.h, env-inl.h
    • Introduced a ClassId enum which weak persistent wrappers can use so that they can be walked upon Environment destruction and cleaned up for sure.
    • Added some worker related methods
    • Added set_using_cares(), which means that in Environment destructor destroy_cares is called
    • Added CanCallIntoJs() method: main instance can always call into js but workers can be in the middle of termination in which case one cannot call into js anymore
    • Added Exit method which does process exit if called from main thread and worker termination if called from worker thread
  • cares_wrap.cc
    • Added InitAresOnce to do one-time process-wide initialization
    • Added a call to env->set_using_cares() (see env.h above)
  • async-wrap.cc
    • Made same changes to MakeCallback as were made to node::MakeCallback
  • timers.js
    • Added handle cleanup registrations for timer handles so that when a worker Environment is destructed, the handles won't leak
  • child_process.js
    • Added handle cleanup registration for the pipe handle so that when a worker Environment is destructed, the handle won't leak. The other handles probably leak too but this was the only one that is caught by current tests. There is currently an assertion in worker termination code for leaked handles.
  • LICENSE
    • Added ProducerConsumerQueue's license

@mscdex
Copy link
Contributor

mscdex commented Mar 15, 2015

What exactly is meant by "Named properties of Arrays?"

}, null, WorkerContextBinding.ERROR);
};

require('util')._extend(Worker, EventEmitter.prototype);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this reuse the existing cached require('util')?

@petkaantonov
Copy link
Contributor Author

Property that is a string or symbol as opposed to an integer (which are called indexed properties)

@domenic
Copy link
Contributor

domenic commented Mar 16, 2015

Epic feature :D.

Regarding structured clone, @dslomov-chromium had plans to bring that into the ES spec and presumably also into V8 directly. I am not sure what the status of that is though. It would be good to collaborate to avoid future back-compat breakage if we do ever get the chance to switch to V8's structured clone.

@bnoordhuis
Copy link
Member

That's the ES7 typed objects strawman, isn't it?

@vkurchatkin
Copy link
Contributor

@petkaantonov
Copy link
Contributor Author

@domenic Yea, the thing should have been implemented in V8 to begin with. Firefox for example implements it in SpiderMonkey, with callbacks for host objects.

Another solution can be patching V8 with some API additions that are already possible in v8 but just not done. E.g.:

  • Array::HaveIndexedPropertiesInPrototypeChain
  • Array::IsSparse()
  • Object::GetOwnRealNamedProperties()
  • Object::GetRealIndexedOwnProperty

Also what would be nice to have:

  • Object::GetRealNamedOwnProperty

But of course most optimal solution is implementing structured clone in v8, the above additions would just enable avoiding completely ridiculous things like having to call GetPropertyNames() on an array.

@bnoordhuis
Copy link
Member

Object::GetRealNamedOwnProperty

You should be able to emulate that like this:

Local<Object> object = /* ... */;
Local<String> key = /* ... */;
Local<Value> property = object->GetRealNamedProperty(key);
const bool is_own_property =
    !property.IsEmpty() && object->GetRealNamedPropertyInPrototypeChain(key).IsEmpty();

Array::HaveIndexedPropertiesInPrototypeChain

Local<Array> array = /* ... */;
const uint32_t index = /* ... */;
const bool is_from_prototype =
    !array->HasRealIndexedProperty(index) &&
    !array->HasIndexedLookupInterceptor() &&
    array->Has(index);

(Or just check HasRealIndexedProperty for each element in the prototype chain.)

var context = this._workerContext;
this._workerContext = null;
if (typeof callback === 'function') {
this.on('exit', function(exitCode) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this.once() so the closure doesn't stay around?

@petkaantonov
Copy link
Contributor Author

@bnoordhuis The check should be much cheaper than emulation, I mean at the cost of calling HasRealIndexedProperty alone you could have already retrieved a real own indexed property. And only if it's undefined would you need to check if it's a hole, which is a rare case.

to_remove = hc;
break;
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An O(n) operation per closed handle is kind of unfortunate...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I thought about that and we should consider changing this to a structure that allows faster removal

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or if the HandleCleanup object is available to the registerer, removal is constant time

Fishrock123 and others added 14 commits July 3, 2015 17:10
Notable changes

* deps: Fixed an out-of-band write in utf8 decoder.
This is an important security update as it can be used to cause a
denial of service attack.
See next commit for the actual fix.

PR-URL: nodejs#1373
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Update AUTHORS list using tools/update-authors.sh

PR-URL: nodejs#2100
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James Hartig <fastest963@gmail.com>
Improve performance by:
+ Not leaking the `arguments` object!
+ Getting the last character of a string by index, instead of
  with `.substr()` or `.slice()`

Improve code consistency by:
+ Using `[]` instead of `.charAt()` where possible
+ Using a function declaration instead of a var declaration
+ Using `.slice()` with clearer arguments
+ Checking if `dir` is truthy in `win32.format`
  (added tests for this)

Improve both by:
+ Making the reusable `trimArray()` function
+ Standardizing getting certain path statistics with
  the new `win32StatPath()` function

PR-URL: nodejs#1778
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
Path functions being benchmarked are:
* format
* isAbsolute
* join
* normalize
* relative
* resolve

PR-URL: nodejs#1778
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
PR-URL: nodejs#2112
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Every npm version bump requires a few patches to be floated on
node-gyp for io.js compatibility. These patches are found in
03d1992,
5de334c, and
da730c7. This commit squashes
them into a single commit.

PR-URL: nodejs#990
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
The delay-load hook allows node.exe/iojs.exe to be renamed. See efadffe
for more background.

PR-URL: nodejs#1433
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
PR-URL: nodejs#2099
Author: Jeremiah Senkpiel <fishrock123@rocketmail.com>
The original test uses a variable to explicitly count how many
times the callback is invoked. This patch uses common.mustCall()
to track if the callback is called or not. This makes the test
more robust, as we don't explicitly hardcode the number of times
to be called.

PR-URL: nodejs#2122
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
gdbinit provided by V8 can be very useful for low-level debugging of
crashes in node and in binary addons. Most useful commands at 'jst'
for JS stack traces and 'job' for printing a heap object.

This patch installs the file at $PREFIX/share/doc/node/gdbinit.

Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
PR-URL: nodejs#2123
Add a check for crypto before using it, similar to how
other tests work.

PR-URL: nodejs#2129
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
@alubbe
Copy link

alubbe commented Jul 8, 2015

Can we do anything to help out @petkaantonov? It would be awesome to play around with this!

@bnoordhuis
Copy link
Member

@alubbe There are still some outstanding issues / comments. I don't think anyone will object if you adopt this PR and make the necessary changes. Just fork Petka's branch and add your changes on top.

@petkaantonov
Copy link
Contributor Author

#2133 replaces this PR

@GnorTech
Copy link
Member

For those who want to write Node.js code in multithread program: NW.js implemented this by enabling Node.js in Web Workers: https://nwjs.io/blog/v0.18.4/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-minor PRs that contain new features and should be released in the next minor version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.