Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto: add key object API #24234

Closed
wants to merge 16 commits into from

Conversation

tniessen
Copy link
Member

@tniessen tniessen commented Nov 7, 2018

Preamble: This is a huge change and I will do my best to help with reviewing it. There might still be dozens of places that need some work, but so far, everything seems to be working nicely. I summarized the motivation behind this change in #15113 (comment). There are also lots of possible discussions around this, e.g. whether key derivation should consume / produce key objects etc.


This commit makes multiple important changes:

  1. A new key object API is introduced. The KeyObject class itself is
    not exposed to users, instead, several new APIs can be used to
    construct key objects: createSecretKey, createPrivateKey and
    createPublicKey. The new API also allows to convert between
    different key formats, and even though the API itself is not
    compatible to the WebCrypto standard in any way, it makes
    interoperability much simpler.

  2. Key objects can be used instead of the raw key material in all
    relevant crypto APIs.

  3. The handling of asymmetric keys has been unified and greatly
    improved. Node.js now fully supports both PEM-encoded and
    DER-encoded public and private keys.

  4. Conversions between buffers and strings have been moved to native
    code for sensitive data such as symmetric keys due to security
    considerations such as zeroing temporary buffers.

  5. For compatibility with older versions of the crypto API, this
    change allows to specify Buffers and strings as the "passphrase"
    option when reading or writing an encoded key. Note that this
    can result in unexpected behavior if the password contains a
    null byte.


cc @nodejs/crypto @nodejs/security-wg @nodejs/security

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@tniessen tniessen added the semver-minor PRs that contain new features and should be released in the next minor version. label Nov 7, 2018
@nodejs-github-bot nodejs-github-bot added the lib / src Issues and PRs related to general changes in the lib or src directory. label Nov 7, 2018
@tniessen tniessen added crypto Issues and PRs related to the crypto subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. labels Nov 7, 2018
Copy link
Contributor

@refack refack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass, looks good.
Some guideline nits.

lib/internal/crypto/sig.js Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
@TimothyGu
Copy link
Member

The new API also allows to convert between different key formats, and even though the API itself is not compatible to the WebCrypto standard in any way, it makes interoperability much simpler.

What are the differences between this new API and the API exposed through web crypto? Could we realistically change this API to be compatible?

@tniessen
Copy link
Member Author

tniessen commented Nov 10, 2018

What are the differences between this new API and the API exposed through web crypto? Could we realistically change this API to be compatible?

WebCrypto has a very different design. I originally called the new API CryptoKey but changed it to KeyObject to avoid confusion with the WebCrypto CryptoKey class. Some of the key differences:

  • CryptoKey objects can be exported as raw/SPKI/PKCS#8 only and that is done using a separate function, exportKey, not using an instance method. Our options are much more complex, e.g. to support encrypted keys, and I think it makes sense for this to be an instance method.

  • The type property is actually compatible to the getType() function of the KeyObject class, and if we decide to implement @addaleax suggestion and turn it into a property, it will match the WebCrypto spec. However, CryptoKey instances have three additional properties that would have a huge impact on the way people can use key objects:

    readonly attribute boolean extractable;
    readonly attribute object algorithm;
    readonly attribute object usages;
    

    We could implement them, but they don't make sense unless we adopt the whole WebCrypto API in my opinion, and as I said before, I don't think we should do that.

  • Symmetric CryptoKey objects don't have a property describing the size of the underlying key, and the only way to get the type of an asymmetric key is to parse the algorithm identifier associated with the key.


Rebased, old HEAD was 6c92496a7cf304785162ad5cd37632cea916b22b.

@tniessen tniessen force-pushed the crypto-add-key-objects branch from 6c92496 to 29e1fe7 Compare November 12, 2018 17:20
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
Copy link
Contributor

@sam-github sam-github left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the direction, I think its a good API. Left some comments, mostly on the API. The C++ is a lot to read through, sorry, I ran out of time. I would prefer this not be experimental. I assume it doesn't intentionally break current APIs, and it also isn't particularly complex in terms of its API, so its hard to see it generating much comment or controversy or need to change. Would be great if it can go into 11.x, get some mileage there, and be stable in 12.xx. I doubt it can make it into 10.x, its a lot of churn.

doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Outdated Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
@jasnell
Copy link
Member

jasnell commented Nov 19, 2018

Definite +1 on this moving forward. Code generally looking good but won't sign off until it's further along.

@tniessen tniessen force-pushed the crypto-add-key-objects branch from c115514 to e68c257 Compare November 19, 2018 20:53
@ChALkeR
Copy link
Member

ChALkeR commented Nov 23, 2018

/cc @joepie91

@tniessen
Copy link
Member Author

One question has come up multiple times: Should the key object API permit X509 certificates when constructing public keys? Personally, I think it makes more sense to provide an API for certificates that allows to extract the public key as a key object because a certificate is not the same as an asymmetric key, but others might feel differently.

cc @nodejs/crypto

src/node_crypto.h Show resolved Hide resolved
src/node_crypto.cc Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
doc/api/crypto.md Outdated Show resolved Hide resolved
This commit makes multiple important changes:

1. A new key object API is introduced. The KeyObject class itself is
   not exposed to users, instead, several new APIs can be used to
   construct key objects: createSecretKey, createPrivateKey and
   createPublicKey. The new API also allows to convert between
   different key formats, and even though the API itself is not
   compatible to the WebCrypto standard in any way, it makes
   interoperability much simpler.

2. Key objects can be used instead of the raw key material in all
   relevant crypto APIs.

3. The handling of asymmetric keys has been unified and greatly
   improved. Node.js now fully supports both PEM-encoded and
   DER-encoded public and private keys.

4. Conversions between buffers and strings have been moved to native
   code for sensitive data such as symmetric keys due to security
   considerations such as zeroing temporary buffers.

5. For compatibility with older versions of the crypto API, this
   change allows to specify Buffers and strings as the "passphrase"
   option when reading or writing an encoded key. Note that this
   can result in unexpected behavior if the password contains a
   null byte.
MylesBorins pushed a commit that referenced this pull request Dec 25, 2018
This commit makes multiple important changes:

1. A new key object API is introduced. The KeyObject class itself is
   not exposed to users, instead, several new APIs can be used to
   construct key objects: createSecretKey, createPrivateKey and
   createPublicKey. The new API also allows to convert between
   different key formats, and even though the API itself is not
   compatible to the WebCrypto standard in any way, it makes
   interoperability much simpler.

2. Key objects can be used instead of the raw key material in all
   relevant crypto APIs.

3. The handling of asymmetric keys has been unified and greatly
   improved. Node.js now fully supports both PEM-encoded and
   DER-encoded public and private keys.

4. Conversions between buffers and strings have been moved to native
   code for sensitive data such as symmetric keys due to security
   considerations such as zeroing temporary buffers.

5. For compatibility with older versions of the crypto API, this
   change allows to specify Buffers and strings as the "passphrase"
   option when reading or writing an encoded key. Note that this
   can result in unexpected behavior if the password contains a
   null byte.

PR-URL: #24234
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
MylesBorins pushed a commit that referenced this pull request Dec 25, 2018
PR-URL: #24234
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
MylesBorins added a commit that referenced this pull request Dec 25, 2018
Notable Changes:

* cli:
  - add --max-http-header-size flag (cjihrig)
    #24811
* crypto:
  - always accept certificates as public keys (Tobias Nießen)
    #24234
  - add key object API (Tobias Nießen) [#24234](#24234)
  - update root certificates (Sam Roberts)
    #25113
* deps:
  - upgrade to libuv 1.24.1 (cjihrig)
    #25078
  - upgrade npm to 6.5.0 (Audrey Eschright)
    #24734
* http:
  - add maxHeaderSize property (cjihrig)
    #24860

PR-URL: #25175
@MylesBorins MylesBorins mentioned this pull request Dec 25, 2018
MylesBorins pushed a commit that referenced this pull request Dec 26, 2018
This commit makes multiple important changes:

1. A new key object API is introduced. The KeyObject class itself is
   not exposed to users, instead, several new APIs can be used to
   construct key objects: createSecretKey, createPrivateKey and
   createPublicKey. The new API also allows to convert between
   different key formats, and even though the API itself is not
   compatible to the WebCrypto standard in any way, it makes
   interoperability much simpler.

2. Key objects can be used instead of the raw key material in all
   relevant crypto APIs.

3. The handling of asymmetric keys has been unified and greatly
   improved. Node.js now fully supports both PEM-encoded and
   DER-encoded public and private keys.

4. Conversions between buffers and strings have been moved to native
   code for sensitive data such as symmetric keys due to security
   considerations such as zeroing temporary buffers.

5. For compatibility with older versions of the crypto API, this
   change allows to specify Buffers and strings as the "passphrase"
   option when reading or writing an encoded key. Note that this
   can result in unexpected behavior if the password contains a
   null byte.

PR-URL: #24234
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
MylesBorins pushed a commit that referenced this pull request Dec 26, 2018
PR-URL: #24234
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
MylesBorins added a commit that referenced this pull request Dec 26, 2018
Notable Changes:

* cli:
  - add --max-http-header-size flag (cjihrig)
    #24811
* crypto:
  - always accept certificates as public keys (Tobias Nießen)
    #24234
  - add key object API (Tobias Nießen) [#24234](#24234)
  - update root certificates (Sam Roberts)
    #25113
* deps:
  - upgrade to libuv 1.24.1 (cjihrig)
    #25078
  - upgrade npm to 6.5.0 (Audrey Eschright)
    #24734
* http:
  - add maxHeaderSize property (cjihrig)
    #24860

PR-URL: #25175
MylesBorins added a commit that referenced this pull request Dec 26, 2018
Notable Changes:

* cli:
  - add --max-http-header-size flag (cjihrig)
    #24811
* crypto:
  - always accept certificates as public keys (Tobias Nießen)
    #24234
  - add key object API (Tobias Nießen) [#24234](#24234)
  - update root certificates (Sam Roberts)
    #25113
* deps:
  - upgrade to libuv 1.24.1 (cjihrig)
    #25078
  - upgrade npm to 6.5.0 (Audrey Eschright)
    #24734
* http:
  - add maxHeaderSize property (cjihrig)
    #24860

PR-URL: #25175
cjihrig added a commit to cjihrig/node that referenced this pull request Dec 26, 2018
During the time between nodejs#24234
being opened and it landing, a V8 update occurred that deprecated
several APIs. This commit fixes the following compiler warnings:

../src/node_crypto.cc:3342:11:
  warning: 'Set' is deprecated: Use maybe version

../src/node_crypto.cc:3345:13:
  warning: 'GetFunction' is deprecated: Use maybe version

PR-URL: nodejs#25205
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
MylesBorins added a commit that referenced this pull request Dec 26, 2018
Notable Changes:

* cli:
  - add --max-http-header-size flag (cjihrig)
    #24811
* crypto:
  - always accept certificates as public keys (Tobias Nießen)
    #24234
  - add key object API (Tobias Nießen) [#24234](#24234)
  - update root certificates (Sam Roberts)
    #25113
* deps:
  - upgrade to libuv 1.24.1 (cjihrig)
    #25078
  - upgrade npm to 6.5.0 (Audrey Eschright)
    #24734
* http:
  - add maxHeaderSize property (cjihrig)
    #24860

PR-URL: #25175
targos pushed a commit that referenced this pull request Jan 1, 2019
During the time between #24234
being opened and it landing, a V8 update occurred that deprecated
several APIs. This commit fixes the following compiler warnings:

../src/node_crypto.cc:3342:11:
  warning: 'Set' is deprecated: Use maybe version

../src/node_crypto.cc:3345:13:
  warning: 'GetFunction' is deprecated: Use maybe version

PR-URL: #25205
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
This commit makes multiple important changes:

1. A new key object API is introduced. The KeyObject class itself is
   not exposed to users, instead, several new APIs can be used to
   construct key objects: createSecretKey, createPrivateKey and
   createPublicKey. The new API also allows to convert between
   different key formats, and even though the API itself is not
   compatible to the WebCrypto standard in any way, it makes
   interoperability much simpler.

2. Key objects can be used instead of the raw key material in all
   relevant crypto APIs.

3. The handling of asymmetric keys has been unified and greatly
   improved. Node.js now fully supports both PEM-encoded and
   DER-encoded public and private keys.

4. Conversions between buffers and strings have been moved to native
   code for sensitive data such as symmetric keys due to security
   considerations such as zeroing temporary buffers.

5. For compatibility with older versions of the crypto API, this
   change allows to specify Buffers and strings as the "passphrase"
   option when reading or writing an encoded key. Note that this
   can result in unexpected behavior if the password contains a
   null byte.

PR-URL: nodejs#24234
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
PR-URL: nodejs#24234
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
During the time between nodejs#24234
being opened and it landing, a V8 update occurred that deprecated
several APIs. This commit fixes the following compiler warnings:

../src/node_crypto.cc:3342:11:
  warning: 'Set' is deprecated: Use maybe version

../src/node_crypto.cc:3345:13:
  warning: 'GetFunction' is deprecated: Use maybe version

PR-URL: nodejs#25205
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
Notable Changes:

* cli:
  - add --max-http-header-size flag (cjihrig)
    nodejs#24811
* crypto:
  - always accept certificates as public keys (Tobias Nießen)
    nodejs#24234
  - add key object API (Tobias Nießen) [nodejs#24234](nodejs#24234)
  - update root certificates (Sam Roberts)
    nodejs#25113
* deps:
  - upgrade to libuv 1.24.1 (cjihrig)
    nodejs#25078
  - upgrade npm to 6.5.0 (Audrey Eschright)
    nodejs#24734
* http:
  - add maxHeaderSize property (cjihrig)
    nodejs#24860

PR-URL: nodejs#25175
paroga added a commit to paroga/node that referenced this pull request Mar 2, 2019
Expose the size of asymetric keys of crypto key object from the
crypto module added in v11.6.0 (nodejs#24234)
BridgeAR pushed a commit to BridgeAR/node that referenced this pull request Mar 6, 2019
Expose the size of asymetric keys of crypto key object from the
crypto module added in v11.6.0.

PR-URL: nodejs#26387
Refs: nodejs#24234
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
BridgeAR pushed a commit to BridgeAR/node that referenced this pull request Mar 12, 2019
Expose the size of asymetric keys of crypto key object from the
crypto module added in v11.6.0.

PR-URL: nodejs#26387
Refs: nodejs#24234
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
@tniessen tniessen mentioned this pull request Mar 29, 2019
16 tasks
@benbucksch
Copy link
Contributor

benbucksch commented May 20, 2020

The new API introduced here is extremely ambiguous and confusing.
We have a "KeyObject" and a "key object", which are not the same type.
We have KeyObject type, key object, key parameter, and key property. All different.
We have key.key.key. Not kidding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. crypto Issues and PRs related to the crypto subsystem. lib / src Issues and PRs related to general changes in the lib or src directory. notable-change PRs with changes that should be highlighted in changelogs. semver-minor PRs that contain new features and should be released in the next minor version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.