Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: Failed to extract data from archive member #2285

Closed
jhugman opened this issue Oct 24, 2024 · 4 comments
Closed

Error: Failed to extract data from archive member #2285

jhugman opened this issue Oct 24, 2024 · 4 comments

Comments

@jhugman
Copy link
Contributor

jhugman commented Oct 24, 2024

I'm trying to use library_mode to generate_bindings from a reasonably large crate, the matrix-sdk-ffi bindings.

Also relevant: I've bumped uniffi to version 0.28.2.

Using cargo ndk, I'm building the staticlib version for aarc64-linux-android, in release mode only.

  cargo ndk \
  --manifest-path ./bindings/matrix-sdk-ffi/Cargo.toml \
  --target arm64-v8a \
  --platform 21 --no-strip -- \
  build --release

Then using print-repr to look at the metadata.

cargo run \
  --manifest-path uniffi-bindgen/Cargo.toml \
  print-repr \
  target/aarch64-linux-android/release/libmatrix_sdk_ffi.a 

Output is:

    Finished `dev` profile [unoptimized] target(s) in 0.13s
     Running `target/debug/uniffi-bindgen print-repr target/aarch64-linux-android/release/libmatrix_sdk_ffi.a`
Failed to extract data from archive member `matrix_sdk_ffi.matrix_sdk_ffi.bf8c76dbb4572677-cgu.07.rcgu.o`

A lightly instrumented uniffi which prints in extract_item that a symbol has been recognized, but is an unrecognized metadata code:

Metadata symbol UNIFFI_META_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION
thread 'main' panicked at uniffi_meta/src/metadata.rs:84:18:
XXXX UNKNOWN ITEM 88 XXXX
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/panicking.rs:647:5
   1: core::panicking::panic_fmt
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/panicking.rs:72:14
   2: uniffi_meta::metadata::codes::metadata_as_str
             at ./uniffi_meta/src/metadata.rs:84:18
   3: uniffi_meta::reader::MetadataReader::read_metadata
             at ./uniffi_meta/src/reader.rs:39:33
   4: uniffi_meta::reader::read_metadata
             at ./uniffi_meta/src/reader.rs:10:5
   5: uniffi_meta::Metadata::read
             at ./uniffi_meta/src/lib.rs:505:9
   6: uniffi_bindgen::macro_metadata::extract::ExtractedItems::extract_item
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:184:25
   7: uniffi_bindgen::macro_metadata::extract::extract_from_elf
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:53:13
   8: uniffi_bindgen::macro_metadata::extract::extract_from_bytes
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:29:29
   9: uniffi_bindgen::macro_metadata::extract::extract_from_archive
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:143:18
  10: uniffi_bindgen::macro_metadata::extract::extract_from_bytes
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:32:37
  11: uniffi_bindgen::macro_metadata::extract::extract_from_library
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:24:5
  12: uniffi_bindgen::library_mode::find_components
             at ./uniffi_bindgen/src/library_mode.rs:104:17
  13: uniffi_bindgen::library_mode::generate_bindings
             at ./uniffi_bindgen/src/library_mode.rs:46:26
  14: uniffi::cli::uniffi_bindgen::gen_library_mode
             at ./uniffi/src/cli/uniffi_bindgen.rs:207:38
  15: uniffi::cli::uniffi_bindgen::run_main
             at ./uniffi/src/cli/uniffi_bindgen.rs:297:17
  16: uniffi::cli::uniffi_bindgen_main
             at ./uniffi/src/cli/mod.rs:9:21
  17: uniffi_bindgen::main
             at ./cli/src/main.rs:2:5
  18: core::ops::function::FnOnce::call_once
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Unfortunately, I don't understand ELF, ndk or the process of it being written to move the investigation further.


Investigation to get to here: jhugman/uniffi-bindgen-react-native#122

@mhammond
Copy link
Member

That sounds a little like a mismatch between the bytes written by that item vs the bytes we think we should read. Can you share the definition of UNIFFI_META_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION as emitted by the macros?

@jhugman
Copy link
Contributor Author

jhugman commented Oct 24, 2024

Using cargo expand, I'm getting:

    const UNIFFI_META_CONST_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION: ::uniffi::MetadataBuffer = ::uniffi::MetadataBuffer::from_code(
            ::uniffi::metadata::codes::INTERFACE,
        )
        .concat_str("matrix_sdk_ffi")
        .concat_str("Encryption")
        .concat_long_str("");
    #[no_mangle]
    #[doc(hidden)]
    pub static UNIFFI_META_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION: [u8; UNIFFI_META_CONST_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION
        .size] = UNIFFI_META_CONST_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION.into_array();

@mhammond
Copy link
Member

huh, that seems correct. After we read ::uniffi::metadata::codes::INTERFACE here, we then read an object here, which tries to read 3 strings - which looks exactly like what we are emitting. So sorry, but that doesn't help me understand what's going wrong :(

@jhugman
Copy link
Contributor Author

jhugman commented Oct 24, 2024

Yeah, the reading/writing of the metadata works just fine in debug, and with other architectures.

I would guess (and I am guessing here), is that there is some subtlety in the extract_elf method that works in debug OR lib.so, but not in release AND lib.a.

@bendk pointed out a fix for a similar looking problem in MachO builds.

bendk added a commit to bendk/uniffi-rs that referenced this issue Oct 24, 2024
Symbol table entries store a 16-bit index for their section header.  If
the index overflows 16-bits then it's stored in a separate section
instead.

References:

- https://refspecs.linuxbase.org/elf/gabi4+/ch4.symtab.html
- https://docs.oracle.com/cd/E19683-01/817-3677/chapter6-94076/index.html
bendk added a commit to bendk/uniffi-rs that referenced this issue Oct 24, 2024
Symbol table entries store a 16-bit index for their section header.  If
the index overflows 16-bits then it's stored in a separate section
instead.

References:

- https://refspecs.linuxbase.org/elf/gabi4+/ch4.symtab.html
- https://docs.oracle.com/cd/E19683-01/817-3677/chapter6-94076/index.html
bendk added a commit to bendk/uniffi-rs that referenced this issue Oct 24, 2024
Symbol table entries store a 16-bit index for their section header.  If
the index overflows 16-bits then it's stored in a separate section
instead.

References:

- https://refspecs.linuxbase.org/elf/gabi4+/ch4.symtab.html
- https://docs.oracle.com/cd/E19683-01/817-3677/chapter6-94076/index.html
bendk added a commit to bendk/uniffi-rs that referenced this issue Oct 25, 2024
Symbol table entries store a 16-bit index for their section header.  If
the index overflows 16-bits then it's stored in a separate section
instead.

References:

- https://refspecs.linuxbase.org/elf/gabi4+/ch4.symtab.html
- https://docs.oracle.com/cd/E19683-01/817-3677/chapter6-94076/index.html
bendk added a commit that referenced this issue Oct 25, 2024
Fix metadata extraction for large ELF files (#2285)
TheLortex pushed a commit to wesprint-io/uniffi-rs that referenced this issue Oct 31, 2024
Symbol table entries store a 16-bit index for their section header.  If
the index overflows 16-bits then it's stored in a separate section
instead.

References:

- https://refspecs.linuxbase.org/elf/gabi4+/ch4.symtab.html
- https://docs.oracle.com/cd/E19683-01/817-3677/chapter6-94076/index.html
@bendk bendk closed this as completed Nov 8, 2024
bendk added a commit to bendk/uniffi-rs that referenced this issue Nov 8, 2024
Symbol table entries store a 16-bit index for their section header.  If
the index overflows 16-bits then it's stored in a separate section
instead.

References:

- https://refspecs.linuxbase.org/elf/gabi4+/ch4.symtab.html
- https://docs.oracle.com/cd/E19683-01/817-3677/chapter6-94076/index.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants