Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Interpreter breaks due to not being able to load libpthread #14195

Open
straight-shoota opened this issue Jan 9, 2024 · 9 comments
Open

Interpreter breaks due to not being able to load libpthread #14195

straight-shoota opened this issue Jan 9, 2024 · 9 comments
Labels
kind:bug kind:regression Something that used to correctly work but no longer works topic:compiler:interpreter

Comments

@straight-shoota
Copy link
Member

straight-shoota commented Jan 9, 2024 •
edited
Loading

Since Crystal 1.11.0 the interpreter breaks while booting up because it cannot load libpthread.

It works fine when pkg-config is unavailable.
Reproduction steps (using Arch Linux, but it appears on other systems as well):

$ pacman -Sy crystal make llvm gcc pkg-config
$ make crystal interpreter=1
$ bin/crystal i hello-world.cr
Using compiled compiler at .build/crystal
Crystal interpreter 1.11.0 (2024-01-08).
EXPERIMENTAL SOFTWARE: if you find a bug, please consider opening an issue in
https://github.com/crystal-lang/crystal/issues/new/
cannot find -lpthread (/usr/lib/libpthread.so: cannot open shared object file: No such file or directory)
Linker arguments: -L/usr/bin/../lib/crystal -lpcre2-8 -lpthread -ldl -levent
Search path: /usr/bin/../lib/crystal:/lib64:/usr/lib64:/lib:/usr/lib (Crystal::Loader::LoadError)
  from src/compiler/crystal/loader/unix.cr:113:31 in 'load_library'
  from src/compiler/crystal/loader.cr:40:7 in 'new'
  from src/compiler/crystal/loader/unix.cr:76:7 in 'parse'
...

Without pkg-config it works.

https://forum.crystal-lang.org/t/crystal-1-11-has-been-released/6284/10?u=straight-shoota
@straight-shoota straight-shoota added kind:bug kind:regression Something that used to correctly work but no longer works topic:compiler:interpreter labels Jan 9, 2024
@straight-shoota
Copy link
Member Author

The first thing I'm wondering about is why the interpreter even tries to load libpthread. This should be disabled in interpreted mode (ref #11807).

Maybe pkg-config for some other lib asks for -lpthread? This would explain why it works when pkg-config is missing.

@MistressRemilia
Copy link
Contributor

I seem to get something similar, but with a different error message, on Slackware 15.0. But this only happens with certain programs, not all of them.

Clipped for length 
[alexa@sakuya benben]$ crystal i src/main.cr
Crystal interpreter 1.11.0-dev [e3200d9eb] (2024-01-04).
EXPERIMENTAL SOFTWARE: if you find a bug, please consider opening an issue in
https://github.com/crystal-lang/crystal/issues/new/
Monkey patching the ZStandard bindings to fix a memory leak
Invalid option: -pthread (OptionParser::InvalidOption)
  from /home/alexa/src/crystal/src/option_parser.cr:130:45 in '->'
  from /home/alexa/src/crystal/src/option_parser.cr:496:27 in 'parse'
  from /home/alexa/src/crystal/src/option_parser.cr:118:5 in 'loader'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/context.cr:431:5 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1293:8 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1293:8 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2100:5 in 'create_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1898:22 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:472:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1497:5 in 'get_const_index_and_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:2645:27 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'visit'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'compile_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:1497:5 in 'get_const_index_and_compiled_def'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:685:31 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/syntax/visitor.cr:27:12 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3196:5 in 'accept'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'visit'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/semantic/bindings.cr:16:7 in 'visit'
  from /home/alexa/src/crystal/src/enumerable.cr:510:7 in 'accept'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/compiler.cr:3388:11 in 'compile'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/interpreter.cr:232:5 in 'interpret'
  from /home/alexa/src/crystal/src/compiler/crystal/interpreter/repl.cr:96:5 in 'interpret_and_exit_on_error'
  from /home/alexa/src/crystal/src/gc/boehm.cr:141:5 in 'repl'
  from /home/alexa/src/crystal/src/compiler/crystal/command.cr:103:7 in 'run'
  from /home/alexa/src/crystal/src/compiler/crystal/command.cr:54:5 in '__crystal_main'
  from /home/alexa/src/crystal/src/crystal/main.cr:129:5 in 'main'
  from /lib64/libc.so.6 in '__libc_start_main'
  from ../sysdeps/x86_64/start.S:122 in '_start'
  from ???
[alexa@sakuya benben]$

@straight-shoota straight-shoota added this to the 1.11.1 milestone Jan 9, 2024
@straight-shoota
Copy link
Member Author

straight-shoota commented Jan 9, 2024 •
edited
Loading

The culprit seems to be #14130 which fixed the pkg-config name for libgc (=bdw-gc) and since then pkg-config --libs bdw-gc outputs the correct linker flags for libgc, which includes -lpthread.

@straight-shoota
Copy link
Member Author

I think the easiest way to get back into working order is to revert the part of #14130 which defines the (correct) pkg-config name.
That was just a bonus and the rest of that PR should continue to work without it. Linking libgc worked fine sofwar without the proper pkg-config name, so we should be good with it for a little longer.

Then for a proper fix we'll need to improve the loader to function when -lpthread is given but the library doesn't exist. In compiled mode this is not an issue because libgc.a is available (it's empty but fulfils the link flag).

@straight-shoota straight-shoota mentioned this issue Jan 10, 2024
Merged
@straight-shoota
Copy link
Member Author

straight-shoota commented Jan 10, 2024 •
edited
Loading

This issue went undetected because our build images (where interpreter CI runs) do not have libgc-dev installed. Thus they're missing the pkg-config file for bdw-gc and -lpthread does not appear in the linker flags.
Everything still works because the interpreter actually ignores -lgc as a workaround to avoid clashes with the compiler's own garbage collector. The symbols are still available to the interpreted program through the program handle because they're linked into the compiler.

@straight-shoota straight-shoota mentioned this issue Jan 10, 2024
Merged
@straight-shoota
Copy link
Member Author

straight-shoota commented Jan 10, 2024 •
edited
Loading

In order to resolve this issue for real, we'll need to make some changes.

Until now we have defined most1 of the special cases for the interpreter linking with conditional @[Link] annotations. This is an easy solution because it works in stdlib and doesn't need a compiler update. But it only gets us so far as it does not work for linker flags that are outside our control (e.g. from pkg-config).

In order to cover that, we need to change the loader to have special rules for some libraries. Basically, for some lib flags it should ignore any errors when the library file cannot be found. This should be fine as long as all symbols can be found. These optional libs would be pthread and dl.
Perhaps we could activate this behaviour only if the gnu flag is set.

Footnotes

  1. The loader ignores -lgc entirely due to issues with loading libgc twice in the same program. ↩

@straight-shoota
Copy link
Member Author

For reference, I don't think there's any easy way to clearly detect when pthread and dl can be skipped.

We could try to mimic what happens in a compile time linker: It links the empty static library, and everything works because the symbols are already available through libc.
The loader could check if the respective static libraries are available. But it would be more complicated to verify that they're empty.
I think it's fine to skip this extra validation. As long as the loader can find all symbols, we're good.

@straight-shoota straight-shoota removed this from the 1.11.1 milestone Jan 10, 2024
@zw963 zw963 mentioned this issue Jan 10, 2024
Closed
@straight-shoota
Copy link
Member Author

Another approach could be to tailor the linker arguments more exactly to the libc requirements. For glibc >= 2.34 we should not add link annotations for the removed libraries.
This could be the default with a compile time flag to enable these linker arguments when working with older versions of glibc.
Or we try to recognize the library version via finding and inspecting libc.so.

@straight-shoota
Copy link
Member Author

I have a branch for #14195 (comment): https://github.com/straight-shoota/crystal/tree/feat/loader-ignore-missing

But I'm wondering if #14195 (comment) isn't a better solution in general.
I suppose it's feasibility depends on if we can check libc version easily or can get away without checking.

@Vici37 Vici37 mentioned this issue Feb 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind:bug kind:regression Something that used to correctly work but no longer works topic:compiler:interpreter
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@straight-shoota @MistressRemilia