Can't locate IPC/Cmd.pm in @INC with openssl v0.9.93

[ChristopherRabotin]

Until version 0.9.93, openssl would build perfectly fine in a Github workflow without additional dependencies. This is no longer the case since v0.9.93.

I've tried at least five different packages to install in the ubuntu-latest Github workflow, as evidenced here, and still openssl fails to build.

  RANLIBFLAGS = None
  running cd "/home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/build/src" && AR="ar" CC="cc" RANLIB="ranlib" "perl" "./Configure" "--prefix=/home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/install" "--openssldir=/usr/local/ssl" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "linux-x86_64" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64"

  --- stderr
  Can't locate IPC/Cmd.pm in @INC (@INC contains: /home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/build/src/util/perl /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . /home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/build/src/external/perl/Text-Template-1.56/lib) at /home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/build/src/util/perl/OpenSSL/config.pm line 19.
  BEGIN failed--compilation aborted at /home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/build/src/util/perl/OpenSSL/config.pm line 19.
  Compilation failed in require at ./Configure line 23.
  BEGIN failed--compilation aborted at ./Configure line 23.
  thread 'main' panicked at '


  Error configuring OpenSSL build:
      Command: cd "/home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/build/src" && AR="ar" CC="cc" RANLIB="ranlib" "perl" "./Configure" "--prefix=/home/runner/work/nyx/nyx/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-5d1194676ac32302/out/openssl-build/install" "--openssldir=/usr/local/ssl" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "linux-x86_64" "-O2" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64"
      Exit status: exit status: 2


      ', /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssl-src-300.1.3+3.1.2/src/lib.rs:573:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `PYO3_ENVIRONMENT_SIGNATURE="cpython-3.7-64bit" PYO3_PYTHON="/usr/local/bin/python3.7" PYTHON_SYS_EXECUTABLE="/usr/local/bin/python3.7" "cargo" "rustc" "--features" "python" "--target" "x86_64-unknown-linux-gnu" "--message-format" "json-render-diagnostics" "--manifest-path" "/home/runner/work/nyx/nyx/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"`
Error: The process '/usr/bin/docker' failed with exit code 1
Error: The process '/usr/bin/docker' failed with exit code 1
    at ExecState._setResult (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1702:25)
    at ExecState.CheckComplete (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1685:18)
    at ChildProcess.<anonymous> (/home/runner/work/_actions/PyO3/maturin-action/v1/dist/index.js:1579:27)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

This is obviously related to #1550, but the answer is stunningly vague as to which "required perl packages" should be installed.

I suspect that this is as simple as installing the correct packages. The following are not the ones to install:

- name: Install Perl for openssl
        run: | # Perl should just die by now
          sudo apt-get update
          sudo apt-get install -y libipc-run-safehandles-perl perl

I appreciate if someone could quickly answer since I've been headbutting on this for a short while now and have lost all hope in figure out what Perl package is needed (and also wonder why Perl hasn't died yet, but that's another conversation).

[sfackler]

IIRC maturin runs the build in a manylinux Docker image. Installing dependencies in the host Ubuntu install will not make them available in the container.

[ChristopherRabotin]

Ah ok, that would exain why my attempts have failed so far. Let me see if I can add the perl installation to the maturin build, not quite sure how I'd do that immediately. Thanks for your quick reply

…

On Wed, Sep 6, 2023, 20:13 Steven Fackler ***@***.***> wrote: IIRC maturin runs the build in a manylinux Docker image. Installing dependencies in the host Ubuntu install will not make them available in the container. — Reply to this email directly, view it on GitHub <#2036 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEZV2ABL5JMVEX6XH6KM5TXZEUVZANCNFSM6AAAAAA4OE6JH4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[ChristopherRabotin]

For those who encounter this issue, remove the manylinux: auto from the maturin action. It is also possible that there was something wrong with the sscache: 'true' instruction. Removing that for one of the builds showed that everything was being rebuilt. Aborting that run and reenabling it picked up the build from scratch again.

So I'm not quite sure what was going on: it's either some cache that wasn't invalidated when it should have, or something with the manylinux.

Thanks sfackler for pointing me toward the right directly.

Successful build here.

[s3rius]

@ChristopherRabotin. Disabling 'manylinux: auto' would make your package uninstallable on old systems.

So, the actual problem is that if you use manylinux: auto option, the maturin-action creates a docker container that runs minimal needed manylinux version specified by the manylinux PEP. In some containers openssl-devel is not installed by default. To surpass this issue, you need to install it manually.

You can find list of all containers that runs for specific targets in maturin-action's README here (Note that x86 containers are spelled as i686). Interesting thing here is that some containers are debian-based, and others are RHEL-based. So you cannot simply put apt-get ...

I was struggling wit this problem for days. You solution initially helped to fast-release the package, but then I couldn't install it inside my docker-containers, so I was trying to find another one.

You can see the working workflow here: https://github.com/Intreecom/scyllapy/blob/develop/.github/workflows/release.yaml
And here's job logs: https://github.com/Intreecom/scyllapy/actions/runs/6227492557

Also I depend on libatomic, and I'm not sure if it's for openssl. But if you find any problems linking with -latomic flag, use my if statement from the script to solve it.

[ChristopherRabotin]

@s3rius A thousand thanks! I encountered this issue again just this week and my previous fix no longer worked, but yours does! Thank you, thank you, thank you!

[thorrester]

@s3rius - THANK YOU! I spent about 2 hours trying to debug an openssl issue until I stumbled across your solution. Works perfectly.