-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
crypto: use openssl's own memory BIOs in crypto_context.cc #47160
Conversation
Review requested:
|
There was an error requiring running CLANG_FORMAT_START=$(git merge-base HEAD main) make format-cpp |
Please help! I ran the |
Oh, thanks a lot! |
Is it possible to add a test for this? That feels necessary for a change like this (otherwise a future PR that undoes these changes with the goal of simplifying code probably also has a good chance of being approved). |
I can give you a test key but you have to install gost-engine (additional OpenSSL engine) to check it... |
I've tried environment without gost-engine installed and found that I can't make TLS connection but I can create secure context with my key (it fails without my patch and successes with it). So I'll try to write a test. |
By the way, what is a reason to statically link OpenSSL during Node.JS compilation by default? I mean that's somehow weird that in the rest of platform you can load any openssl config with --openssl-config option, load any engine with crypto.setEngine, but, for example, tls part is statically linked to a certain OpenSSL. |
If you've ever tried to ship a dynamically linked binary across a multitude of platforms and distros, you know the answer. ^_^ |
Done. |
I just wonder why statically linked Node.JS does support engines in crypto.createSign but not in tls.connect. |
It does. Not sure where you get that notion from but |
Haha) tls.getCiphers() doesn't see additional ciphers until OpenSSL is dynamically linked, Looks like there will be another investigation and another issue)) |
Oh, is that what you mean. That's indeed a separate issue but the reason is simple: SSL_get_ciphers() is called once and the result is cached. If you call it before your engine is loaded, you won't see any of the new ciphers it adds. |
So do you need more of my help to approve this PL? |
|
Done. |
Ok, it looks like my test fails on some configurations. I'll investigate further and rewrite it. |
Test failed because specified key cannot be processed without gost-engine. |
Yep) I understood it, that's why now I'm rewriting the test. The trick was that Node.js with statically linked OpenSSL loads engine even without "crypto.setEngine" so I was sure that engine would be needed later to create connection but not to load a .pem. That's another oddity, I wrote about it in issue #43653 . |
OK, I've rewritten test. It's rather low-level but works. |
Hi! Could you re-run workflows please? |
@bnoordhuis Should I fix anything else? |
It looks like the new CI is successful. Should you add the "commit-queue" label again? |
Landed in 10b21e5 |
NodeBIO's memory buffer structure does not support BIO_C_FILE_SEEK and B IO_C_FILE_TELL. This prevents OpenSSL PEM_read_bio_PrivateKey from readi ng some private keys. So I switched to OpenSSL'w own protected memory bu ffers. Fixes: nodejs#47008 PR-URL: nodejs#47160 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
NodeBIO's memory buffer structure does not support BIO_C_FILE_SEEK and B IO_C_FILE_TELL. This prevents OpenSSL PEM_read_bio_PrivateKey from readi ng some private keys. So I switched to OpenSSL'w own protected memory bu ffers. Fixes: #47008 PR-URL: #47160 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
NodeBIO's memory buffer structure does not support BIO_C_FILE_SEEK and B IO_C_FILE_TELL. This prevents OpenSSL PEM_read_bio_PrivateKey from readi ng some private keys. So I switched to OpenSSL'w own protected memory bu ffers. Fixes: #47008 PR-URL: #47160 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
NodeBIO's memory buffer structure does not support BIO_C_FILE_SEEK and B IO_C_FILE_TELL. This prevents OpenSSL PEM_read_bio_PrivateKey from readi ng some private keys. So I switched to OpenSSL'w own protected memory bu ffers. Fixes: nodejs#47008 PR-URL: nodejs#47160 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309
Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309
Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309
Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309
Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309
Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309
* chore: bump node in DEPS to v18.17.0 * chore: update build_modify_js2c_py_to_allow_injection_of_original-fs_and_custom_embedder_js.patch Xref: nodejs/node#46930 manually sync patch to minor upstream code shear * chore: update build_ensure_native_module_compilation_fails_if_not_using_a_new.patch Xref: nodejs/node#48248 manually sync patch to minor upstream code shear * chore: update fix_expose_the_built-in_electron_module_via_the_esm_loader.patch Xref: nodejs/node#47824 chore: upstream func throwIfUnsupportedURLProtocol() has been removed, so no need to patch it * chore: update api_pass_oomdetails_to_oomerrorcallback.patch Xref: nodejs/node#47695 manually sync patch to minor upstream code shear * chore: remove fix_prevent_changing_functiontemplateinfo_after_publish.patch Xref: nodejs/node#46979 (upstreamed patch) Xref: https://chromium-review.googlesource.com/c/v8/v8/+/2718147 (related) * chore: update fix_adapt_debugger_tests_for_upstream_v8_changes.patch Xref: nodejs/node#47274 manually sync patch to minor upstream code shear some tests moved from sequential to parallel * chore: remove fix_libc_buffer_overflow_in_string_view_ctor.patch Xref: fix_libc_buffer_overflow_in_string_view_ctor.patch patch is no longer needed due to upstream bump to ada 2.2.0 * chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.patch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0 * chore: rebuild filenames.json several files removed/added/changed upstream * chore: update build_add_gn_build_files.patch upstream dep histogram 0.11.7 moved its include path from src/ to include/ Xref: nodejs/node#47742 * chore: update fix_crypto_tests_to_run_with_bssl.patch Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: ftbfs in node dep ada * fix: ftbfs in node dep uvwasi * chore: rebuild patches * chore: update fix_handle_boringssl_and_openssl_incompatibilities.patch Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: work around Node 18 isURL() regression * chore: sort script/node-disabled-tests.json alphabetically * test: add parallel/test-snapshot-argv1 to disabled list test: add parallel/test-snapshot-namespaced-builtin to disabled list We don't support that type of snapshotting at the moment. * chore: disable flaky node test parallel/test-dgram-send-cb-quelches-error fails upstream in v18.x on my box as well * ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set * fixup! fix: work around Node 18 isURL() regression fix: infinite loop regression * fixup! fix: work around Node 18 isURL() regression * chore: patch fixtures/errors/force_colors.snapshot The line numbers in the stacktrace from our v8 build don't match what Node's tests are expecting, so update the stacktrace to match our build. The specific numbers probably aren't t needed for the force_colors test, which is trying to see whether or not the lines are greyed out. One option is to upstream a test change to stop hardcoding the stacktrace. * fixup! fix: work around Node 18 isURL() regression fix; pull in upstream bugfix * fixup! ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set chore: do not inject ELECTRON_RUN_AS_NODE in test-assert-colors.js * chore: disable flaky node test parallel/test-debugger-random-port-with-inspect-port --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com>
This fixes a warning in line 26: "warning: value computed is not used" when calling BIO_seek(). Refs: nodejs#47160
* chore: bump node in DEPS to v18.17.0 * chore: update build_modify_js2c_py_to_allow_injection_of_original-fs_and_custom_embedder_js.patch Xref: nodejs/node#46930 manually sync patch to minor upstream code shear * chore: update build_ensure_native_module_compilation_fails_if_not_using_a_new.patch Xref: nodejs/node#48248 manually sync patch to minor upstream code shear * chore: update fix_expose_the_built-in_electron_module_via_the_esm_loader.patch Xref: nodejs/node#47824 chore: upstream func throwIfUnsupportedURLProtocol() has been removed, so no need to patch it * chore: update api_pass_oomdetails_to_oomerrorcallback.patch Xref: nodejs/node#47695 manually sync patch to minor upstream code shear * chore: remove fix_prevent_changing_functiontemplateinfo_after_publish.patch Xref: nodejs/node#46979 (upstreamed patch) Xref: https://chromium-review.googlesource.com/c/v8/v8/+/2718147 (related) * chore: update fix_adapt_debugger_tests_for_upstream_v8_changes.patch Xref: nodejs/node#47274 manually sync patch to minor upstream code shear some tests moved from sequential to parallel * chore: remove fix_libc_buffer_overflow_in_string_view_ctor.patch Xref: fix_libc_buffer_overflow_in_string_view_ctor.patch patch is no longer needed due to upstream bump to ada 2.2.0 * chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.patch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0 * chore: rebuild filenames.json several files removed/added/changed upstream * chore: update build_add_gn_build_files.patch upstream dep histogram 0.11.7 moved its include path from src/ to include/ Xref: nodejs/node#47742 * chore: update fix_crypto_tests_to_run_with_bssl.patch Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: ftbfs in node dep ada * fix: ftbfs in node dep uvwasi * chore: rebuild patches * chore: update fix_handle_boringssl_and_openssl_incompatibilities.patch Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: work around Node 18 isURL() regression * chore: sort script/node-disabled-tests.json alphabetically * test: add parallel/test-snapshot-argv1 to disabled list test: add parallel/test-snapshot-namespaced-builtin to disabled list We don't support that type of snapshotting at the moment. * chore: disable flaky node test parallel/test-dgram-send-cb-quelches-error fails upstream in v18.x on my box as well * ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set * fixup! fix: work around Node 18 isURL() regression fix: infinite loop regression * fixup! fix: work around Node 18 isURL() regression * chore: patch fixtures/errors/force_colors.snapshot The line numbers in the stacktrace from our v8 build don't match what Node's tests are expecting, so update the stacktrace to match our build. The specific numbers probably aren't t needed for the force_colors test, which is trying to see whether or not the lines are greyed out. One option is to upstream a test change to stop hardcoding the stacktrace. * fixup! fix: work around Node 18 isURL() regression fix; pull in upstream bugfix * fixup! ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set chore: do not inject ELECTRON_RUN_AS_NODE in test-assert-colors.js * chore: disable flaky node test parallel/test-debugger-random-port-with-inspect-port --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com>
This fixes a warning in line 26: "warning: value computed is not used" when calling BIO_seek(). Refs: nodejs#47160 PR-URL: nodejs#49206 Reviewed-By: Michael Dawson <midawson@redhat.com>
* chore: bump node in DEPS to v18.17.0 * chore: update build_modify_js2c_py_to_allow_injection_of_original-fs_and_custom_embedder_js.patch Xref: nodejs/node#46930 manually sync patch to minor upstream code shear * chore: update build_ensure_native_module_compilation_fails_if_not_using_a_new.patch Xref: nodejs/node#48248 manually sync patch to minor upstream code shear * chore: update fix_expose_the_built-in_electron_module_via_the_esm_loader.patch Xref: nodejs/node#47824 chore: upstream func throwIfUnsupportedURLProtocol() has been removed, so no need to patch it * chore: update api_pass_oomdetails_to_oomerrorcallback.patch Xref: nodejs/node#47695 manually sync patch to minor upstream code shear * chore: remove fix_prevent_changing_functiontemplateinfo_after_publish.patch Xref: nodejs/node#46979 (upstreamed patch) Xref: https://chromium-review.googlesource.com/c/v8/v8/+/2718147 (related) * chore: update fix_adapt_debugger_tests_for_upstream_v8_changes.patch Xref: nodejs/node#47274 manually sync patch to minor upstream code shear some tests moved from sequential to parallel * chore: remove fix_libc_buffer_overflow_in_string_view_ctor.patch Xref: fix_libc_buffer_overflow_in_string_view_ctor.patch patch is no longer needed due to upstream bump to ada 2.2.0 * chore: remove fix_preventing_potential_oob_in_ada_no_scheme_parsing.patch Xref: nodejs/node#47339 patch is no longer needed due to upstream bump to ada 2.2.0 * chore: rebuild filenames.json several files removed/added/changed upstream * chore: update build_add_gn_build_files.patch upstream dep histogram 0.11.7 moved its include path from src/ to include/ Xref: nodejs/node#47742 * chore: update fix_crypto_tests_to_run_with_bssl.patch Xref: nodejs/node#47160 BoringSSL doesn't support BIO_s_secmem() (a secure heap variant of BIO_s_mem()), so use BIO_s_mem() instead. Related discussion of secure heap support in BoringSSL: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: ftbfs in node dep ada * fix: ftbfs in node dep uvwasi * chore: rebuild patches * chore: update fix_handle_boringssl_and_openssl_incompatibilities.patch Upstream used `BIO_s_secmem()`, a secure heap variant of `BIO_s_mem()`. BoringSSL doesn't support it, so this PR opts for `BIO_s_mem()` instead. Upstream Node.js change that prompted this: nodejs/node#47160 Related discussion of BoringSSL support of secure heap: https://boringssl-review.googlesource.com/c/boringssl/+/54309 * fix: work around Node 18 isURL() regression * chore: sort script/node-disabled-tests.json alphabetically * test: add parallel/test-snapshot-argv1 to disabled list test: add parallel/test-snapshot-namespaced-builtin to disabled list We don't support that type of snapshotting at the moment. * chore: disable flaky node test parallel/test-dgram-send-cb-quelches-error fails upstream in v18.x on my box as well * ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set * fixup! fix: work around Node 18 isURL() regression fix: infinite loop regression * fixup! fix: work around Node 18 isURL() regression * chore: patch fixtures/errors/force_colors.snapshot The line numbers in the stacktrace from our v8 build don't match what Node's tests are expecting, so update the stacktrace to match our build. The specific numbers probably aren't t needed for the force_colors test, which is trying to see whether or not the lines are greyed out. One option is to upstream a test change to stop hardcoding the stacktrace. * fixup! fix: work around Node 18 isURL() regression fix; pull in upstream bugfix * fixup! ci: ensure spawned node tests have ELECTRON_RUN_AS_NODE set chore: do not inject ELECTRON_RUN_AS_NODE in test-assert-colors.js * chore: disable flaky node test parallel/test-debugger-random-port-with-inspect-port --------- Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: Charles Kerr <charles@charleskerr.com>
This fixes a warning in line 26: "warning: value computed is not used" when calling BIO_seek(). Refs: nodejs/node#47160 PR-URL: nodejs/node#49206 Reviewed-By: Michael Dawson <midawson@redhat.com>
This fixes a warning in line 26: "warning: value computed is not used" when calling BIO_seek(). Refs: nodejs/node#47160 PR-URL: nodejs/node#49206 Reviewed-By: Michael Dawson <midawson@redhat.com>
NodeBIO's memory buffer structure does not support BIO_C_FILE_SEEK and B IO_C_FILE_TELL. This prevents OpenSSL PEM_read_bio_PrivateKey from readi ng some private keys. So I switched to OpenSSL's own protected memory bu ffers.
Fixes: #47008