include/node/openssl/archs/ has ~50MB of (I hope?) unused headers

[asottile]

it looks like the prebuilt builds contain openssl headers for each architecture, even though they are only built for one -- potentially saving ~50MB ish:

$ du -hs node-v18.13.0-linux-x64/include/node/openssl/archs/*
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/aix64-gcc-as
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/BSD-x86
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/BSD-x86_64
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/darwin64-arm64-cc
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/darwin64-x86_64-cc
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/darwin-i386-cc
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux32-s390x
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux64-mips64
936K	node-v18.13.0-linux-x64/include/node/openssl/archs/linux64-riscv64
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux64-s390x
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux-aarch64
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux-armv4
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux-elf
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux-ppc64le
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/linux-x86_64
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/solaris64-x86_64-gcc
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/solaris-x86-gcc
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/VC-WIN32
2.8M	node-v18.13.0-linux-x64/include/node/openssl/archs/VC-WIN64A
936K	node-v18.13.0-linux-x64/include/node/openssl/archs/VC-WIN64-ARM

[UlisesGascon]

Thanks for the suggestion @asottile. I believe this issue will be more relevant in nodejs/node. I don't have the capabilities to transfer it, but I hope other colleagues in the @nodejs/build can do it :)

[mhdawson]

@UlisesGascon transferred

[mhdawson]

@asottile is that the same case for 14.x builds as well? Just wondering if you know if this is new since our OpenSSL 3 integration or its always been that way.

[asottile]

looks like 14.x has the same issue -- but it is much less pronounced there:

$ ls -d node-v14.19.3-linux-x64/include/node/openssl/archs/*
node-v14.19.3-linux-x64/include/node/openssl/archs/aix64-gcc
node-v14.19.3-linux-x64/include/node/openssl/archs/aix-gcc
node-v14.19.3-linux-x64/include/node/openssl/archs/BSD-x86
node-v14.19.3-linux-x64/include/node/openssl/archs/BSD-x86_64
node-v14.19.3-linux-x64/include/node/openssl/archs/darwin64-arm64-cc
node-v14.19.3-linux-x64/include/node/openssl/archs/darwin64-x86_64-cc
node-v14.19.3-linux-x64/include/node/openssl/archs/darwin-i386-cc
node-v14.19.3-linux-x64/include/node/openssl/archs/linux32-s390x
node-v14.19.3-linux-x64/include/node/openssl/archs/linux64-mips64
node-v14.19.3-linux-x64/include/node/openssl/archs/linux64-s390x
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-aarch64
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-armv4
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-elf
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-ppc
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-ppc64
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-ppc64le
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-x32
node-v14.19.3-linux-x64/include/node/openssl/archs/linux-x86_64
node-v14.19.3-linux-x64/include/node/openssl/archs/solaris64-x86_64-gcc
node-v14.19.3-linux-x64/include/node/openssl/archs/solaris-x86-gcc
node-v14.19.3-linux-x64/include/node/openssl/archs/VC-WIN32
node-v14.19.3-linux-x64/include/node/openssl/archs/VC-WIN64A
node-v14.19.3-linux-x64/include/node/openssl/archs/VC-WIN64-ARM

[mhdawson]

@asottile thanks for the quick response, the headers are larger in OpenSSL 3 so it might make more sense to prune now than before.

[richardlau]

Some history:

• Node.js OpenSSL headers much larger after OpenSSL 3 switch over #42081
• deps: add template for generated headers #42616 attempted some deduplication but it caused a regression deps: regression in openssl #42976 so was reverted Revert "deps: add template for generated headers" #42978.

[mhdawson]

@richardlau I think the deduplication attempted before might be a bit different. In the headers tarball I think we need headers for all supported platforms but in the tar for a specific architecture/os it may be ok to just have the one for that architecture/os

[bnoordhuis]

In the headers tarball I think we need headers for all supported platforms

I touched on it earlier this week in #46401 but many files in deps/openssl/config/archs/*/no-asm are duplicates. I count 1018 files but only 183 unique files.

My shell-fu says we can shave off 16,068,792 bytes just by deduping those, more if the linked PR lands.

[andymac4182]

Is there any reason these files can't be deleted completely for a slimmed down runtime only production image?

Is there anything that can be done to help move this work along?

[bnoordhuis]

Yes: send pull requests. This is basically waiting for someone to do the work.

[andymac4182]

I will be honest I am not 100% over what needs to happen. If you can give me a 2 second summary I will go through the code and write up what I think I need to do then do it :)

[jbergstroem]

@andymac4182 the prior work, related issue and revert is a bit higher up in this thread. You probabaly want to stand on top of that should you work towards a new PR! #46451 (comment)

[andymac4182]

Thanks. I will give it a read over 👍

[SimenB]

FWIW, nodejs/docker-node#1923 has landed

[LaurentGoderre]

We might need to do it for slim and alpine as well