Node.js crypto and the WebCrypto API

[daviddias]

I've been looking for documentation and/or notes about the plans for the Node.js core crypto module and somethings are still unclear to me (but on that I'm sure I'm simply missing a lot of context, which I hope to get :)), namely:

• Since there is a WebCrypto Standard now, is there any goal of accessing the WebCryptoAPI available in Chrome from Node.js?
• Node.js crypto module stands on top of openssl lib, what would be the obstacles if we wanted to move to the WebCryptoAPI, not being dependent on the openssl lib that is installed in the host (and from that, enable crypto code to move between Node.js and the Browser without any trouble).
• What is the state of ECDSA in Node.js
• Is there plans for a Crypto WG? Or is there a channel that I should be posting these questions instead of here?

Thank you :)

[Fishrock123]

cc @nodejs/crypto

[mscdex]

1. It hasn't really been discussed before, but IMHO this is something that I think could easily be done in userland. crypto provides most of the features of the Web Cryptography API already, it's just a difference in function names.
2. By default, the bundled OpenSSL is used for the crypto module, not the host's copy of OpenSSL. The obstacles would be creating a compatibility layer that maps the Web Cryptography API to the crypto API counterparts. There's probably a few features that do not currently exist in the crypto module. Technically those things could be polyfilled in userland too.
3. I'm not sure what you mean here, but you can already sign/verify data using ecdsa keys in node.
4. There is no crypto WG currently, but there is a crypto team (as @Fishrock123 showed). I'm not exactly sure what the goals of a crypto WG would be.

[daviddias]

Thank you for getting back to me, @mscdex :)

Here goes some comments:

1. Having a 'webcrypto' module that picks the right API depending of where the code is being run would definitely be nice.
2. Good to know, hadn't realised that.
3. Could you share an example? I was unable to get it working and started using the ecdsa module instead
4. Apparently the team is private, not sure if that is intended. I just wanted to understand if there was a best place to post my questions

[rvagg]

Private just as a result of the way github does teams, anyone who feels qualified to weigh in and help whenever that team is pingged is welcome to join, if you have the time and interest @diasdavid then I'd happily add you to that team.

There is no WG, there hasn't been a need for it, but anyone who wants to spend the time to start and help run a working group is welcome to do so, you can bootstrap it here by getting some discussion happening and figuring out what that WG might do, the level of interest and response might tell you whether such a group would even be practical. Again, you're more than welcome to try and make that happen but be patient obviously.

[indutny]

@mscdex goals of crypto WG should be cryptic...

[indutny]

I'm overall +1 for the WebCrypto, sounds like a good idea.

[daviddias]

Thank you for clarifying that, @rvagg :) and thank you, @indutny, for validating the idea :)

I'm always glad to help and crypto is definitely on my pool of subjects of interest! What are the current endeavours that the crypto team is taking?

I've added the WebCrypto module to my list of things to build for libp2p + node-ipfs and what I think its best for me to do is to continue following up with the crypto team if I find anything or simply for clarifications.

[rvagg]

What are the current endeavours that the crypto team is taking?

I believe the team primarily exists as support for all things crypto rather than undertaking specific endeavours, beyond upgrading and maintaining openssl that is. So it's up to you to propose a way forward that would turn this into a slightly more formalised team rather than just a crypto-support team.

[tomgco]

I have also come across various quirks around the crypto subsystem, specifically not having an API that can create ECDSA private and public keys (unless I am mistaken), and format the exported keys into "raw", "pkcs8" and "spki", would also be nice to include JsonWebKey's as an auxiliary input and output key format in node core.

Would we be interested in allowing the generation of certificates from within the node crypto submodule, something like this would help?

[tomgco]

specifically not having an API that can create ECDSA private and public keys (unless I am mistaken)

By this I mean that It is not simple to be able to create these keys without having to go through the convoluted way of using ECDH.generateKeys, even then we are able to grab the private key as a BN_bn2bin and place it into a buffer. But we then lack the ability to format into PEM, thus meaning we cannot pass it as an argument into sign.sign. Meaning that the api is not compatible with itself.

[mikeal]

perhaps the best way forward is for someone to write a module that meets the standard API and give this ticket input if anything in the API isn't possible with the existing functionality exposed by crypto.

[tomgco]

perhaps the best way forward is for someone to write a module that meets the standard API and give this ticket input if anything in the API isn't possible with the existing functionality exposed by crypto.

@diasdavid I will happily help you with creating this compatibility layer between crypto and your module WebCrypto.

[sam-github]

#2833 (comment) should be reported as a bug, it seems to me!

[rmhrisk]

@tomgco once there is a webcrypto module for node our http://pkijs.org/ project should work for creating and validating certificates amongst other things. We have been discussing creating a module like discussed here but have not had the time as we have been focused on some other things but if we can support this in anyway we would like to.

[rmhrisk]

I should add this project might be of interest while looking at doing this : http://gostcrypto.com/doc/SubtleCrypto.html

[calvinmetcalf]

ecdsa signatures with sha2 digests is one (source am trying to write a module that exposes an API that uses either node or web crypto)

[rmhrisk]

We have put together a node module based on typescript and OpenSSL that provides WebCrypto interfaces, you can find it here - https://github.com/PeculiarVentures/node-webcrypto-ossl

[jasnell]

@nodejs/crypto ... is there reason to keep this open? it's not likely that this would be something we'd do in core unless someone happened to pop up with a PR (and even then it's doubtful)

[bnoordhuis]

I'll close the issue. The answer to 'is the WebCrypto API going to be supported in core' is 'probably not'.

If there are gaps in the crypto module that stop a user-land module from implementing the WebCrypto API, please file new issues for them.

[christiansmith]

@jasnell @bnoordhuis

it's not likely that this would be something we'd do in core unless someone happened to pop up with a PR (and even then it's doubtful)

The answer to 'is the WebCrypto API going to be supported in core' is 'probably not'.

As someone implementing open standards that depend heavily on cryptographic functions and target both Node and the browser, I have to say this is extremely disappointing.

Since WebCrypto API is already widely adopted by browsers, there is zero chance of going in the opposite direction and pushing Node's native crypto interface into that world. So incorporating WebCrypto into Node is the only viable option if we want to write isomorphic crypto-dependent JavaScript

The kinds of libraries that depend on cryptography would be so much easier to develop, audit, and maintain if there were just one common crypto API and no wrappers, proxies, shims, or polyfills involved.

Relatively few people are coding directly with these crypto APIs. However, the higher level libraries they create are useful to a broad audience of developers and end users.

It's understandable, given the relatively esoteric nature of the subject, that this isn't an immediate priority for the core team. I'd hope that at the very least if someone submitted a credible PR it would be a welcome contribution.

Please reconsider your position on this.

[bnoordhuis]

The kinds of libraries that depend on cryptography would be so much easier to develop, audit, and maintain if there were just one common crypto API and no wrappers, proxies, shims, or polyfills involved.

@christiansmith You will need to come up with a stronger argument than this. Convenience is not a good reason for inclusion in core. The guideline is: if it can be done as a third-party module, it should be done as a third-party module.

A good argument against WebCrypto is that the existing crypto APIs can never go away. Adding WebCrypto would result in a large amount of duplicate functionality.

[jasnell]

This isn't to say that core developers don't want this, it's only to say
that there is a very high bar to adding new API into the core library. I'm
sure many would love to see this work and it makes sense for us to
collaborate to make sure it works well, but having things as ecosystem
modules is a good thing.

On Wednesday, July 27, 2016, Ben Noordhuis notifications@github.com wrote:

The kinds of libraries that depend on cryptography would be so much easier
to develop, audit, and maintain if there were just one common crypto API
and no wrappers, proxies, shims, or polyfills involved.

@christiansmith https://github.com/christiansmith You will need to come
up with a stronger argument than this. Convenience is not a good reason for
inclusion in core. The guideline is: if it can be done as a third-party
module, it should be done as a third-party module.

A good argument against WebCrypto is that the existing crypto APIs can
never go away. Adding WebCrypto would result in a large amount of duplicate
functionality.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2833 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAa2ea7zyt2e8VfQ1nm1w4Rex5gHKlTtks5qZ5RGgaJpZM4F8Q1y
.

[christiansmith]

Thanks for your prompt reply :)

@bnoordhuis

You will need to come up with a stronger argument than this. Convenience is not a good reason for inclusion in core.

This isn't about simple convenience. It's about increasing the likelihood of shipping reasonably correct code where the impact of getting it even slightly wrong can be orders of magnitude greater than a typical bug in some library.

Audit-ability and maintainability are of paramount concern in the application of cryptography. Having to maintain two separate implementations of the same low level logic, which we currently do, is an invitation for subtle bugs and security flaws.

Shims and wrappers don't eliminate the need to rigorously test, audit and maintain that duplication of error-prone low level code. The investment of time and effort that goes into that is substantially greater than what you'd typically expect in building an app, and the duplication is no minor inconvenience.

The guideline is: if it can be done as a third-party module, it should be done as a third-party module.

First, if crypto should just be done as a third party module, why exactly does Node have a core crypto module to begin with? Cryptography is becoming important enough throughout the ecosystem that all the major browser vendors have actually adopted, implemented, and shipped a standard interface. It's part of the browser. Not a third party module.

Second, have you actually tried using any pure JavaScript crypto libs? While admirable as an intellectual exercise, they're unsuitable for production use on the server, if only for performance reasons. The only way for crypto modules to be viable in Node is as wrappers over c/c++ code. As a third party module, this means dealing with compilation issues that pop up in various platforms. It's great that Node supports native modules, but in my experience they create as many problems as they solve and should be used sparingly.

We experience real challenges in supporting and maintaining all the platforms that Node itself supports using native modules. In practice, it's such a distraction trying to keep up that we have to recommend people run our server code in the controlled environment of containers. This diminishes the viability of our work for no small number of prospective users.

A good argument against WebCrypto is that the existing crypto APIs can never go away. Adding WebCrypto would result in a large amount of duplicate functionality.

It strikes me as absurd to posit that anything can never go away. Languages and platforms deprecate out of date features all the time. Node itself has gone through several differing implementations of core features like Streams, if I remember correctly.

At the time Node's crypto library was created, there was no specified standard for what a crypto API in JavaScript should look like. Now there is, and while we could bikeshed over it all day long, the fact of the matter is it's here to stay and it changes the landscape in a way that warrants re-evaluating early assumptions about how Node's under-the-hood crypto should be exposed to developers.

I understand and agree with the argument for keeping the core of Node.js light and tight, but I also think you're greatly underestimating the benefits of having an interoperable core crypto module.

@jasnell

there is a very high bar to adding new API into the core library

And there should be, especially in this case. All I'm suggesting is that it may be worth keeping open minds about the prospect rather than giving the community a discouraging signal that Node will not adapt to this change in the world around it. Why would a contributor want to take on the challenge if there's a low probability that the core team will be receptive to a contribution?

[bnoordhuis]

This isn't about simple convenience. It's about increasing the likelihood of shipping reasonably correct code where the impact of getting it even slightly wrong can be orders of magnitude greater than a typical bug in some library.

Okay, but why would you care if the code is in core or in a npm module? Core is not a fairy tale land of unicorns and bug-free code, the people working on it aren't infallible robots. (At least, I'm not - can't speak for the others.)

First, if crypto should just be done as a third party module, why exactly does Node have a core crypto module to begin with?

Because it predates npm. It's been part of node since practically the beginning.

The only way for crypto modules to be viable in Node is as wrappers over c/c++ code. As a third party module, this means dealing with compilation issues that pop up in various platforms. It's great that Node supports native modules, but in my experience they create as many problems as they solve and should be used sparingly.

That is essentially a convenience argument. Compiler trouble isn't a good argument, native modules can be shipped as precompiled binaries.

It strikes me as absurd to posit that anything can never go away.

Even if we deprecated the crypto and tls modules today, we'd be well into the 2020s before we could reasonably remove them. That's far enough into the future that it might just as well be never. In the mean time we'd be stuck maintaining two code bases.

[christiansmith]

Okay, but why would you care if the code is in core or in a npm module?

First class support for cryptographic primitives is far too important to be maintained informally in the free time of disorganized volunteers or subject to the fickle interests of commercial enterprise.

And why should we have to bundle and install another version of OpenSSL when one already ships with Node out of the box? Half the work is already done!

I think the Node.js Foundation should take responsibility for the foundations of cryptography in Node. What other organization can be trusted to do it with best long term interests of all Node users in mind? The problem requires that level of purposeful attention and commitment.

Core is not a fairy tale land of unicorns and bug-free code, the people working on it aren't infallible robots. (At least, I'm not - can't speak for the others.)

Reasonably bug-free crypto code is exactly why this should be a concern. It's one thing to write a shim around some new DOM feature. But crypto? A mapping between two different peer level interfaces to the same underlying C implementation is far from ideal. We risk creating new places for vulnerabilities to lurk opportunistically.

Because it predates npm. It's been part of node since practically the beginning.

But it's here now. It's part of the core team's responsibility now.

Compiler trouble isn't a good argument, native modules can be shipped as precompiled binaries.

Which still explodes the problem of supporting multiple platforms from one place which already does it to everyone who writes a library depending on it.

That is essentially a convenience argument.

Based on the last few years of experience with this, I would rather ask our users to run a version of Node.js that has a complete, standard and interoperable crypto interface baked in than deal with ongoing npm crypto dependency hell.

Even if we deprecated the crypto and tls modules today, we'd be well into the 2020s before we could reasonably remove them. That's far enough into the future that it might just as well be never. In the mean time we'd be stuck maintaining two code bases.

I suspect most of the people who will actually read this thread have been working with JavaScript in some form or another since it was called LiveScript and have witnessed the many seismic shifts along the way. Given the reach of Node.js, it's not unreasonable to ask the stewards of the platform to think in very long time frames.

[Qard]

Can we lift the browser implementation like we did with the profiler? If we encourage browser vendors to keep things modular, we can benefit from the existing codebases.

[calvinmetcalf]

@christiansmith in my experience writing a module that used native crypto methods in node or the browser when possible and falling back to js when not, the fact that node has a slightly different API is the least of your worries. The web crypto api is huge and since it is so broad and features so many optional algorithms, simply writing a browser only crypto code requires quite a lot of shims due to different browser support (e.g. safari not supporting anything, chrome not supporting 192 bit keys, and firefox not supporting the export of private ECDSA keys and those are off the top of my head). Meaning you're going to have to use a library anyway for the foreseeable future maintaining 2 versions (one using brower crypto, one doing it in JS) and that's not caring about node

Notice that the web crypto api is behind the keyword subtle that's because it's full of subtle traps and is explicitly designed to be wrapped by libraries and instead of being used directly meaning at the end of the day if node added browser crypto all it would do would make those library authors lives slightly easier, but not that much easier because you can do almost all of the browser crypto in terms of node crypto modules.

[bnoordhuis]

Can we lift the browser implementation like we did with the profiler?

Not without pulling in a metric ton of dependencies. Cryptography in Firefox is built on top of NSS, Chromium uses BoringSSL (which is not 100% compatible with OpenSSL.) In both cases, we'd end up with two crypto libraries in core, and that's not counting the 10,000s lines of glue code.

I don't know Safari/WebKit well enough to comment but I imagine it's the same; browsers are complex beasts.

[Qard]

Ah, right. Forgot about the whole BoringSSL thing...

[oliverjanik]

Please reopen. I think node should ship all the API's browser does with the exception of UI related APIs (DOM).

Node is not our primary environment. We only care about node in context of compiling front-end code-bases and server side rendering. As such we deeply care about having consistent runtime for our code base, whether it's browser or node. Adding yet another environment to deal with (and special case) is a big burden for a small team.