dlsym not implemented for: __pthread_get_minstack #789

petr-tik · 2019-06-24T21:37:39Z

Hey, this is a great project!

I appreciate you are currently not supporting FFI, but thought this might be a useful data point for posterity. I expect you will consider this low on your list of priorities.

I decided to run it on tantivy

** Versions of rustc and miri

petr_tik@merluza:~$ cargo +nightly miri -V
miri 0.1.0 (bc0c76d 2019-05-13)
petr_tik@merluza:~$ rustc +nightly -Vv
rustc 1.36.0-nightly (7d5aa4332 2019-05-16)
binary: rustc
commit-hash: 7d5aa43325ad7629766b1183011f5bf5b2a1ea26
commit-date: 2019-05-16
host: x86_64-unknown-linux-gnu
release: 1.36.0-nightly
LLVM version: 8.0

** How to reproduce

get the current upstream/master of tantivy

petr_tik@merluza:~/Coding/tantivy$ git show -s HEAD
commit bcd7386fc517e8515e3f062f28196f33ff564e23 (HEAD -> master, upstream/master)
Author: Antoine Catton <devel@antoine.catton.fr>
Date:   Tue Jun 18 04:19:06 2019 +0200

    Add crates.io shield to the README (#572)

Run miri

petr_tik@merluza:~/Coding/tantivy$ cargo +nightly miri test -- -Zmiri-seed=42 -- -Zunstable-options --exclude-should-panic
   Compiling tantivy v0.10.0-dev (/home/petr_tik/Coding/tantivy)

running 322 tests
test collector::count_collector::tests::test_count_collect_does_not_requires_scoring ... ok
test collector::count_collector::tests::test_segment_count_collector ... ok
error[E0080]: Miri evaluation error: miri does not support dynamically loading libraries (requested symbol: __pthread_get_minstack)
  --> /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/weak.rs:65:5
   |
65 |     libc::dlsym(libc::RTLD_DEFAULT, name.as_ptr()) as usize
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Miri evaluation error: miri does not support dynamically loading libraries (requested symbol: __pthread_get_minstack)
   |
   = note: inside call to `std::sys::unix::weak::fetch` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/weak.rs:50:33
   = note: inside call to `std::sys::unix::weak::Weak::<unsafe extern "C" fn(*const libc::unix::notbsd::linux::other::b64::x86_64::pthread_attr_t) -> usize>::get` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/thread.rs:404:11
   = note: inside call to `std::sys::unix::thread::min_stack_size` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/sys/unix/thread.rs:48:42
   = note: inside call to `std::sys::unix::thread::Thread::new` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/mod.rs:488:26
   = note: inside call to `std::thread::Builder::spawn_unchecked::<[closure@src/directory/watch_event_router.rs:54:20: 58:14 callbacks:std::vec::Vec<std::sync::Arc<std::boxed::Box<dyn std::ops::Fn() + std::marker::Send + std::marker::Sync>>>], ()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/thread/mod.rs:382:18
note: inside call to `std::thread::Builder::spawn::<[closure@src/directory/watch_event_router.rs:54:20: 58:14 callbacks:std::vec::Vec<std::sync::Arc<std::boxed::Box<dyn std::ops::Fn() + std::marker::Send + std::marker::Sync>>>], ()>` at src/directory/watch_event_router.rs:52:25
  --> src/directory/watch_event_router.rs:52:25
   |
52 |           let spawn_res = std::thread::Builder::new()
   |  _________________________^
53 | |             .name("watch-callbacks".to_string())
54 | |             .spawn(move || {
55 | |                 for callback in callbacks {
56 | |                     callback();
57 | |                 }
58 | |             });
   | |______________^
note: inside call to `directory::watch_event_router::WatchCallbackList::broadcast` at src/directory/ram_directory.rs:185:13
  --> src/directory/ram_directory.rs:185:13
   |
185|             self.fs.write().unwrap().watch_router.broadcast();
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `<directory::ram_directory::RAMDirectory as directory::directory::Directory>::atomic_write` at src/directory/managed_directory.rs:227:9
  --> src/directory/managed_directory.rs:227:9
   |
227|         self.directory.atomic_write(path, data)
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `<directory::managed_directory::ManagedDirectory as directory::directory::Directory>::atomic_write` at src/indexer/segment_updater.rs:77:5
  --> src/indexer/segment_updater.rs:77:5
   |
77 |     directory.atomic_write(&META_FILEPATH, &buffer[..])?;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `indexer::segment_updater::save_metas` at src/indexer/segment_updater.rs:52:5
  --> src/indexer/segment_updater.rs:52:5
   |
52 | /     save_metas(
53 | |         &IndexMeta {
54 | |             segments: Vec::new(),
55 | |             schema,
...  |
59 | |         directory,
60 | |     )
   | |_____^
note: inside call to `indexer::segment_updater::save_new_metas` at src/core/index.rs:149:9
  --> src/core/index.rs:149:9
   |
149|         save_new_metas(schema.clone(), directory.borrow_mut())?;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `core::index::Index::from_directory` at src/core/index.rs:142:9
  --> src/core/index.rs:142:9
   |
142|         Index::from_directory(directory, schema)
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `core::index::Index::create::<directory::ram_directory::RAMDirectory>` at src/core/index.rs:92:9
  --> src/core/index.rs:92:9
   |
92 |         Index::create(ram_directory, schema).expect("Creating a RAMDirectory should never fail")
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `core::index::Index::create_in_ram` at src/collector/facet_collector.rs:529:21
  --> src/collector/facet_collector.rs:529:21
   |
529|         let index = Index::create_in_ram(schema);
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside call to `collector::facet_collector::tests::test_doc_unsorted_multifacet` at src/collector/facet_collector.rs:525:5
  --> src/collector/facet_collector.rs:525:5
   |
525| /     fn test_doc_unsorted_multifacet() {
526| |         let mut schema_builder = Schema::builder();
527| |         let facet_field = schema_builder.add_facet_field("facets");
528| |         let schema = schema_builder.build();
...  |
545| |         assert_eq!(facets[0].1, 1);
546| |     }
   | |_____^
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<[closure@src/collector/facet_collector.rs:525:5: 546:6] as std::ops::FnOnce<()>>::call_once - shim` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1521:5
   = note: inside call to `functional_test::test::__rust_begin_short_backtrace::<fn()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1512:30
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<[closure@DefId(123:368 ~ test[fa8f]::run_test[0]::{{closure}}[3]) 0:fn()] as std::ops::FnOnce<()>>::call_once - shim` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/ops/function.rs:231:5
   = note: inside call to `<[closure@DefId(123:368 ~ test[fa8f]::run_test[0]::{{closure}}[3]) 0:fn()] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/liballoc/boxed.rs:704:9
   = note: inside call to `<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:309:9
   = note: inside call to `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:293:40
   = note: inside call to `std::panicking::try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>>, ()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:289:5
   = note: inside call to `std::panicking::try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>>>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:388:9
   = note: inside call to `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::boxed::FnBox<(), Output = ()> + std::marker::Send>>, ()>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1467:26
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1489:13
   = note: inside call to `functional_test::test::run_test::run_test_inner` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1508:28
   = note: inside call to `functional_test::test::run_test` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:1167:13
   = note: inside call to `functional_test::test::run_tests::<[closure@DefId(123:317 ~ test[fa8f]::run_tests_console[0]::{{closure}}[2]) 0:&mut functional_test::test::ConsoleTestState, 1:&mut std::boxed::Box<dyn functional_test::test::formatters::OutputFormatter>]>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:972:5
   = note: inside call to `functional_test::test::run_tests_console` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:293:15
   = note: inside call to `functional_test::test::test_main` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libtest/lib.rs:327:5
   = note: inside call to `functional_test::test::test_main_static`
   = note: inside call to `main` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:64:34
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:52:53
   = note: inside call to closure at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:293:40
   = note: inside call to `std::panicking::try::do_call::<[closure@DefId(1:5981 ~ std[ef5b]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe], i32>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:289:5
   = note: inside call to `std::panicking::try::<i32, [closure@DefId(1:5981 ~ std[ef5b]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe]>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:388:9
   = note: inside call to `std::panic::catch_unwind::<[closure@DefId(1:5981 ~ std[ef5b]::rt[0]::lang_start_internal[0]::{{closure}}[0]) 0:&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe], i32>` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:52:25
   = note: inside call to `std::rt::lang_start_internal` at /home/petr_tik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:64:5
   = note: inside call to `std::rt::lang_start::<()>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0080`.
test collector::facet_collector::tests::test_doc_unsorted_multifacet ...
error: Could not compile `tantivy`.

To learn more, run the command again with --verbose.

RalfJung · 2019-06-24T21:43:03Z

Thanks for your report!

@oli-obk actually had some ideas for how to extend our shim system to handle dlsym without having to do full FFI... I believe that's part of #603. @oli-obk might be worth splitting the dlsym part (after removing the gross hack ;)? It's independent from the rest of that PR.

RalfJung · 2019-06-24T21:45:02Z

However, looking at your stacktrace, you are trying to access the file system. That's still way further of I think. See #653.

petr-tik · 2019-06-24T22:04:58Z

thanks for replying - hopefully my report might be useful.

However, looking at your stacktrace, you are trying to access the file system.

Our library offers several interfaces to interact with the index. We use the in-RAM directory abstraction for testing, so I thought we might wing it

RalfJung · 2019-06-24T22:12:41Z

Oh I misread... it's not the FS code you are coming from, it's thread spawning. That's ways off I am afraid.

RalfJung · 2019-06-30T14:47:30Z

This will be "fixed" by #808 insofar as the error message will get better:

  --> /home/r/src/rust/rustc/src/libstd/sys/unix/thread.rs:67:19
   |
67 |           let ret = libc::pthread_create(&mut native, &attr, thread_start,
   |  ___________________^
68 | |                                        &*p as *const _ as *mut _);
   | |_________________________________________________________________^ Miri evaluation error: Miri does not support threading

use Dlsym support to implement getentropy (and better thread spawn error) This is the Miri side of rust-lang/rust#62245. Fixes #789

RalfJung · 2019-07-06T09:07:15Z

Actual concurrency support is tracked at #811.

RalfJung added A-shims Area: This affects the external function shims C-proposal Category: a proposal for something we might want to do, or maybe not; details still being worked out labels Jun 24, 2019

RalfJung changed the title ~~miri does not support dynamically loading libraries - __pthread_get_minstack~~ dlsym not implemented for: __pthread_get_minstack Jun 24, 2019

RalfJung added C-enhancement Category: a PR with an enhancement or an issue tracking an accepted enhancement and removed C-proposal Category: a proposal for something we might want to do, or maybe not; details still being worked out labels Jun 29, 2019

RalfJung mentioned this issue Jun 30, 2019

use Dlsym support to implement getentropy (and better thread spawn error) #808

Merged

bors added a commit that referenced this issue Jul 6, 2019

Auto merge of #808 - RalfJung:extra-fn, r=RalfJung

226156f

use Dlsym support to implement getentropy (and better thread spawn error) This is the Miri side of rust-lang/rust#62245. Fixes #789

bors closed this as completed in #808 Jul 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dlsym not implemented for: __pthread_get_minstack #789

dlsym not implemented for: __pthread_get_minstack #789

petr-tik commented Jun 24, 2019

RalfJung commented Jun 24, 2019

RalfJung commented Jun 24, 2019

petr-tik commented Jun 24, 2019

RalfJung commented Jun 24, 2019

RalfJung commented Jun 30, 2019

RalfJung commented Jul 6, 2019

dlsym not implemented for: __pthread_get_minstack #789

dlsym not implemented for: __pthread_get_minstack #789

Comments

petr-tik commented Jun 24, 2019

RalfJung commented Jun 24, 2019

RalfJung commented Jun 24, 2019

petr-tik commented Jun 24, 2019

RalfJung commented Jun 24, 2019

RalfJung commented Jun 30, 2019

RalfJung commented Jul 6, 2019