Win: Request to officially build node.dll with releases

[justinmchase]

There are a lot of discussions around building native addons and the problems with renaming node.exe to iojs.exe, I tried to re-read as many of those as I could before I came here to write this. I am familiar with the delay_load_hook flag workaround.

I just want to point out that there are multiple applications now embedding iojs into their applications (e.g. nwjs.exe, electron.exe). Electron actually builds and ships their own node.dll for this reason, I believe. Electron users typically rename electron.exe to my-app.exe before shipping their product as well. Additionally, you have to rebuild any 3rd party modules yourself, you can't just use node-gyp in a straightforward way as the module developer intended. The end result is that you have to build your module one way to get it to run with iojs via command line or through unit tests and then you have to compile it again, another way, to get it to actually run in Electron and nwjs.

Needless to say, this is all extremely complicated and a real barrier to getting native modules in electron / nwjs.

If, however, io.js shipped with node.exe, iojs.exe & node.dll, and all native addons, when built via node-gyp linked specifically to node.dll instead of to an .exe then this would allow any application to embed node without having to worry about the process name. Additionally, it would allow any native addon compiled with node-gyp, without any extra work, to automatically be compatible with any such embedded application. I could then run unit tests against the same binary that I am running in electron, for example.

I know the argument against adding a .dll is that it is aesthetically pleasing to have node bundled all into a single executable and I don't disagree, but the complexity of managing native addons and building them multiple times to run against multiple executables is outweighing that benefit in my mind. Also, we already have node.exe and iojs.exe, so why not have a 3rd as well? On windows people will rarely use node through any means other than the installer anyway, and the fact that there are 1, 3 or 100 files is largely unappreciated.

[Fishrock123]

cc @piscisaureus, also @zcbenz if he has further details.

[benjamingr]

Since the merge is happening won't this become a non-issue?

[justinmchase]

I don't think that changes anything for apps that embed node. Electron and nwjs for example.

[justinmchase]

Actually I think I may have found a simpler way to support electron or anybody embedding node, without having to ship another binary :)

Just include node.dll in the DelayLoadDLLs list:

'msvs_settings': {
    'VCLinkerTool': {
    'DelayLoadDLLs': ['node.dll', 'iojs.exe', 'node.exe'],
    'AdditionalOptions': ['/ignore:4199' ],
  }
}

And then modify the load_exe_hook function like so:

static FARPROC WINAPI load_exe_hook(unsigned int event, DelayLoadInfo* info)
{
  if (event != dliNotePreLoadLibrary)
    return NULL;

  if (_stricmp(info->szDll, "iojs.exe") != 0 &&
      _stricmp(info->szDll, "node.exe") != 0 &&
      _stricmp(info->szDll, "node.dll") != 0)
    return NULL;

  HMODULE m = GetModuleHandle("node.dll");
  if (m == NULL) m = GetModuleHandle(NULL);
  return (FARPROC) m;
}

This will add support for delay loading node.dll. You don't actually have to ship it with node, but it will allow anyone who is embedding node into their app to optionally compile their own node.dll and ship that with their app (which is what electron does) to automatically support native node modules.

[refack]

@justinmchase I'm digging in the past looking for a simple embedding example code.

1. To give as an example
2. To turn into a test harness for node.dll

Was wondering if you got/know-of one?

[justinmchase]

Its been a while since I thought about this but this was maybe the simplest one I did with some good examples in it:

https://github.com/EvolveLabs/node-mumble-audio

Are you dealing with electron or just straight up making a native module for node? Because electron is a little more complicated but manageable. I would do things a differently now, than I did in the past but I may be able to help.

There are a number of other projects with native components you could look at, for example: node-expat

I don't know where you're getting stuck but based on the node-mumble-audio repo I'll try to give you a few pointers on how to start.

Use Bindings

In your entrypoint use the bindings library. All it basically does is give you a way to load a binary module by simple name and it goes and finds the actual native module by trying a variety of naming convention based paths.

e.g.:
https://github.com/EvolveLabs/node-mumble-audio/blob/master/index.js#L1

Use node-gyp To Compile

Gyp can be a bear to deal with but it is easier the supported happy path and it is better than some other solutions I've seen. Basically you setup a binding.gyp file in your repo, which is essentially a project file describing where your code files are as well as platform specific variables.

e.g.:
https://github.com/EvolveLabs/node-mumble-audio/blob/master/binding.gyp#L4

That .gyp file has some electron specific stuff in it which I will go into if you want but I will skip it for now. If you're not doing electron you can skip the win_delay_load_hook stuff.

Use nan

It may seem a little confusing at first but trust me, use it. It is an abstraction over v8 and libuv. Perhaps the future won't be as bad as the past but major version bumps of node have come with non-backwards compatible versions of v8 which can cause your library to become non-forwards compatible and therefore obsolete pretty fast. It can also dramatically decrease the time it takes for you to update your package when newer versions of node come out.

That being said, use an engines declaration in your package.json.

Similarly use the os and cpu sections in your package.json to declare what you support or not during install.

Use abstractions over os apis

Here is an example of using compiler variables to detect windows or not and abstracting over slighly different OS apis: sleep_ms.

Also use the std library since it does much of this for you and works well with v8 and nan.

Use nan's built-in classes

One really powerful class I recommend looking at is the AsyncProgressWorker. This allows you to begin a job which runs in parallel in an actual different thread than the UI thread v8 is running in. Its a remarkably easy way to get true parallelism in node with your native code, which is absolutely necessary for some scenarios (such as audio processing like this package does).

Documentation.

protip: read up on scope's too!

Use Travis.ci and appveyor

I was using both. This allowed me to get CI testing on Windows, Linux and osx. They are both free for OSS and very reasonably priced for commercial use.

Conventions

Name your folders as src for your native code and lib for node code. Put all binaries that you depend on under the folder deps. You should have binaries for every platform / arch that you support and your folder structure should look something like:

example-app/deps/bin/{arch}/{os}/{libname}
example-app/deps/lib/{arch}/{os}/{libname}
example-app/deps/includes/{libname}

Binaries should go under bin, which includes .dll, .dylib or linux binaries. .lib files should go under the lib folder and all other files such as .h files should go under includes.

Name the output folder of your build dist. Use the postbuild script to copy binary dependencies into the dist folder based on the local arch/os.

Electron

For this the only tricks I had to figure out was how to work around the delay_load_hook as well as the sub-par (in my opinion) way of building at the time.

I believe there have been a number of advances in this area since I last looked so I think I would probably explore those other tools before sharing what I went through. Here are a couple that have either changed significantly or are new that I haven't had a chance to look into yet:

• https://www.npmjs.com/package/electron-builder
• https://www.npmjs.com/package/electron-compile
• https://www.npmjs.com/package/node-ninja
• https://www.npmjs.com/package/nbind

Good Luck!

[refack]

Thanks for the excellent write up, I'm sure I can make a good HOWTO for native addons from it!

Are you dealing with electron or just straight up making a native module for node?

1. A few people asked for simple (i.e. not electron) examples on how to embed node, and I'm looking to write a HOWTO on that
2. We don't do automated testing for the node.dll target, so we need a simple harness that embeds the dll that we can run the tests on.

So I'm looking for more for the other way around, embedding node.dll in another exe...

[justinmchase]

Well just to be clear, with the win_delay_load_hook trick you can essentially compile against node.dll but then at runtime bind that to any other executable, including electron.exe or node.exe or my cool new app.exe.

There is built-in logic in node-gyp to solve that problem for node.exe and node.dll but you can override it. I was able to solve that by using this delay_load_hook instead:
https://github.com/EvolveLabs/electron-updater-tools/blob/master/src/delay_load_hook.h

I don't recommend using that library per-se but you could snipe that file and integrate it into your build processes. I had a pull request to add this into node-gyp directly but they never merged it.

[justinmchase]

If you are creating an executable that links to node as a static library then you can use the win_load_delay_hook trick to build a node binary module that will bind to either node.exe, node.dll or myexecutable.exe at runtime. This will allow you to unit test your module in mocha or whatever through node as well as run in your actual app without requiring recompilation. The same binary will run in both places.

To do this, see this library: https://github.com/EvolveLabs/electron-updater-tools

If you look into delay_load_hook.h you'll see that the delay load hook prevents the loading of the binary the static library is bound to from happening at load, and instead calls a hook function which allows you to do more dynamic resolution of the module. So long as the binary you are loading has all of the node api exported, it will work.

Here you can see the logic for trying to find the binary that exports node libraries:

// If the current process is node or iojs, then just return the proccess module.
if (_stricmp(processName, "node.exe") == 0 ||
    _stricmp(processName, "iojs.exe") == 0) {
  return (FARPROC)processModule;
}

// If it is another process, attempt to load 'node.dll' from the same directory.
PathRemoveFileSpec(processPath);
PathAppend(processPath, "node.dll");

HMODULE nodeDllModule = GetModuleHandle(processPath);
if(nodeDllModule != NULL) {
  // This application has a node.dll in the same directory as the executable, use that.
  return (FARPROC)nodeDllModule;
}

// Fallback to the current executable, which must statically link to node.lib.
return (FARPROC)processModule;

Therefore the above will allow your native module to load in any process that has statically linked in node.lib, a node.dll present in the same directory as the current executable or happens to a be a process named node.exe specifically.

So I would recommend copy+paste that .h file into your module, then in that modules .gyp file add the following:

• Disable default win_delay_load_hook
• Add an include path to your .h file
• Delay load dll's in windows

Also, I just want to add that there was a pull request for this, which would enable this scenario to be supported in node-gyp by default with no extra work on your part but the PR was declined because the author didn't understand the use case (my wording). You may want to comment there if you have any extra weight or use case definition you want to add.

[santhiya-v]

@justinmchase , Thanks for this very useful explanation. We are trying to build a cross-platform native node addon in windows using CMake-JS library. Currently it supports only MSVC compiler, we are adding support for MinGW. They use DELAYLOAD option of the MSVC, we are trying to find an equivalent for MinGW. Any suggestions would greatly help. Thanks

cmake-js/cmake-js@68f92e1

[justinmchase]

@santhiya-v Sorry I don't know enough about cmake to be helpful :(

[PaTiToMaSteR]

I just want to say, in the name of god for all the industry to embed this incredible creation called nodejs... make an official node.dll please. You can easily run js code, but we cannot bind/load addons, so the current embedded solution is a decaf nodejs.