Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash in partition allocator on linux >= 6.9 #4

Closed
kxxt opened this issue Aug 17, 2024 · 5 comments
Closed

Crash in partition allocator on linux >= 6.9 #4

kxxt opened this issue Aug 17, 2024 · 5 comments
Labels
bug Something isn't working crash

Comments

@kxxt
Copy link

kxxt commented Aug 17, 2024

electron v30.1.2

Environment info:

[kxxt@talonflame 1.92.1]$ uname -a
Linux talonflame.felixc.at 6.10.2-arch1-1 #1 SMP PREEMPT_DYNAMIC Mon, 29 Jul 2024 04:58:02 +0000 riscv64 GNU/Linux
[kxxt@talonflame 1.92.1]$ cat /proc/cpuinfo 
processor	: 0
hart		: 2
isa		: rv64imafdc_zicntr_zicsr_zifencei_zihpm
mmu		: sv39
uarch		: sifive,bullet0
mvendorid	: 0x489
marchid		: 0x8000000000000007
mimpid		: 0x20181004
hart isa	: rv64imafdc_zicntr_zicsr_zifencei_zihpm

Crash Info:

Crash in partition allocator:

Line: https://chromium.googlesource.com/chromium/src/+/124.0.6367.243/base/allocator/partition_allocator/src/partition_alloc/partition_address_space.cc#258

Full backtrace:

(gdb) bt full
#0  partition_alloc::internal::(anonymous namespace)::HandlePoolAllocFailure () at ../../base/allocator/partition_allocator/src/partition_alloc/partition_address_space.cc:77
        line_number = 58
        kv60 = {k = "error\000  ", v = 12}
        alloc_page_error_code = <optimized out>
#1  0x0000002ab00314f2 in partition_alloc::internal::PartitionAddressSpace::Init () at ../../base/allocator/partition_allocator/src/partition_alloc/partition_address_space.cc:258
        regular_pool_size = 17179869184
        brp_pool_size = 17179869184
        regular_pool_fd = -1
        brp_pool_fd = <optimized out>
        kForbiddenZoneSize = <optimized out>
        base_address = <optimized out>
        requested_address = <optimized out>
        actual_address = <optimized out>
#2  0x0000002ab0037fde in partition_alloc::PartitionRoot::Init (this=0x2ab49ebac0 <(anonymous namespace)::g_root+64>, opts=...)
    at ../../base/allocator/partition_allocator/src/partition_alloc/partition_root.cc:970
        guard = {lock_ = @0x2ab49ebb00}
--Type <RET> for more, q to quit, c to continue without paging--
        bucket_index = <optimized out>
        lookup = {bucket_sizes_ = {282584257676671, 0, 4310892547, 33519840, 64, 167555120, 15762873573703685, 10977695894601741, 17179869190, 64, 64, 64, 728, 728, 8, 17179869187, 792, 
            792, 792, 33, 33, 1, 17179869185, 0, 0, 0, 33515744, 33515744, 4096, 21474836481, 33515744, 33519840, 33519840, 126588304, 126588304, 4096, 25769803777, 160104064, 160112256, 
            160112256, 6028864, 6029696, 4096, 25769803777, 166133760, 166146048, 166146048, 531920, 1752624, 4096, 17179869191, 160104064, 160112256, 160112256, 132, 1208, 64, 
            25769803778, 165952912, 165961104, 165961104, 944, 944, 8, 18865251666, 160104064, 160112256, 160112256, 6028864, 6029696, 1, 18865251664, 33424704, 33424704, 33424704, 15620, 
            15620, 4, 27455186257, 0, 0, 0, 0, 0, 0, 17179869188, 828, 828, 828, 100, 100, 4, 19058917379, 166665680, 0, 0, 83, 0, 1, 3270858180148161583, 7598185516716943724, 
            8100899532007957363, 3543892497277400118, 17179869184, 4294967312, 5590599, 64424509444, 38654705664, 5712339252805632008, 7233186070448403011, 714626500141252608, 17179869184, 
            12884901908, 18340355981446565447, 18095726453806617431, 17977689061250040375, 0, 0, 0, 77309411380, 0, 0, 137438953555, 0, 0, 137438953583, 0, 0}, bucket_index_lookup_ = {137, 
            0, 34, 0, 0, 0, 0, 0, 0, 0, 0, 0, 233, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 314, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 346, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 353, 0, 18, 0, 0, 0, 0, 
            0, 0, 0, 0, 0, 362, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 430, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 451, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 472, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 621, 
            0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 714, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1940, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1957, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2041, 0, 18, 0, 0, 0, 
            0, 0, 0, 0, 0, 0, 2092, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2104, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2112, 0, 18, 0, 0, 0, 0, 0...}}
#3  0x0000002ab00383ea in partition_alloc::PartitionRoot::PartitionRoot (this=0x2ab49ebac0 <(anonymous namespace)::g_root+64>, opts=...)
    at ../../base/allocator/partition_allocator/src/partition_alloc/partition_root.cc:1146
No locals.
#4  0x0000002ab00462e8 in (anonymous namespace)::MainPartitionConstructor::New (buffer=0x2ab49ebac0 <(anonymous namespace)::g_root+64>)
--Type <RET> for more, q to quit, c to continue without paging--
    at ../../base/allocator/partition_allocator/src/partition_alloc/shim/allocator_shim_default_dispatch_to_partition_alloc.cc:153
        new_root = 0x2ab49ebac0 <(anonymous namespace)::g_root+64>
        opts = {static kAllowed = partition_alloc::PartitionOptions::AllowToggle::kAllowed, static kDisallowed = partition_alloc::PartitionOptions::AllowToggle::kDisallowed, 
          static kDisabled = partition_alloc::PartitionOptions::EnableToggle::kDisabled, static kEnabled = partition_alloc::PartitionOptions::EnableToggle::kEnabled, 
          thread_cache = partition_alloc::PartitionOptions::EnableToggle::kDisabled, star_scan_quarantine = partition_alloc::PartitionOptions::AllowToggle::kAllowed, 
          backup_ref_ptr = partition_alloc::PartitionOptions::EnableToggle::kDisabled, use_configurable_pool = partition_alloc::PartitionOptions::AllowToggle::kDisallowed, 
          scheduler_loop_quarantine = partition_alloc::PartitionOptions::EnableToggle::kDisabled, scheduler_loop_quarantine_capacity_in_bytes = 0, 
          zapping_by_free_flags = partition_alloc::PartitionOptions::EnableToggle::kDisabled, memory_tagging = {enabled = partition_alloc::PartitionOptions::EnableToggle::kDisabled, 
            reporting_mode = partition_alloc::TagViolationReportingMode::kUndefined}, use_pool_offset_freelists = partition_alloc::PartitionOptions::EnableToggle::kDisabled}
#5  (anonymous namespace)::LeakySingleton<partition_alloc::PartitionRoot, (anonymous namespace)::MainPartitionConstructor>::GetSlowPath (this=<optimized out>)
    at ../../base/allocator/partition_allocator/src/partition_alloc/shim/allocator_shim_default_dispatch_to_partition_alloc.cc:136
        instance = 0x2ab49ebac0 <(anonymous namespace)::g_root+64>
        scoped_lock = <optimized out>
#6  (anonymous namespace)::LeakySingleton<partition_alloc::PartitionRoot, (anonymous namespace)::MainPartitionConstructor>::Get (this=0x2ab49eba80 <(anonymous namespace)::g_root>)
    at ../../base/allocator/partition_allocator/src/partition_alloc/shim/allocator_shim_default_dispatch_to_partition_alloc.cc:79
        instance = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
#7  (anonymous namespace)::Allocator () at ../../base/allocator/partition_allocator/src/partition_alloc/shim/allocator_shim_default_dispatch_to_partition_alloc.cc:162
No locals.
#8  allocator_shim::internal::PartitionMalloc (size=73728, context=<optimized out>)
    at ../../base/allocator/partition_allocator/src/partition_alloc/shim/allocator_shim_default_dispatch_to_partition_alloc.cc:206
        guard = {<No data fields>}
#9  0x0000002ab0045a94 in ShimMalloc (size=73728, context=0x0) at ../../base/allocator/partition_allocator/src/partition_alloc/shim/shim_alloc_functions.h:107
        ptr = 0xaaaaaaaaaaaaaaaa
        chain_head = 0xaaaaaaaaaaaaaaaa
#10 malloc (size=73728) at ../../base/allocator/partition_allocator/src/partition_alloc/shim/allocator_shim_override_libc_symbols.h:40
No locals.
#11 0x0000003ff2ca4ed4 in (anonymous namespace)::pool::pool (this=0x3ff2e54230 <(anonymous namespace)::emergency_pool>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_alloc.cc:235
        obj_count = <optimized out>
        str = <optimized out>
        ns_name = <optimized out>
        tunables = {{first = "obj_size", second = 0}, {first = "obj_count", second = 256}}
        obj_size = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
#12 __static_initialization_and_destruction_0 () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_alloc.cc:373
No locals.
#13 _GLOBAL__sub_I_eh_alloc.cc(void) () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_alloc.cc:456
No locals.
#14 0x0000003ff7fdc50a in ?? () from /lib/ld-linux-riscv64-lp64d.so.1
No symbol table info available.
#15 0x0000003ff7fdc5d4 in ?? () from /lib/ld-linux-riscv64-lp64d.so.1
No symbol table info available.
#16 0x0000003ff7fe98dc in ?? () from /lib/ld-linux-riscv64-lp64d.so.1
No symbol table info available.

Last mmap failure:

mprotect(0x3ff7ffc000, 8192, PROT_READ) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
munmap(0x3ff7fb9000, 106571)            = 0
futex(0x3ff2e5550c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
getrandom("\xe5\x64\x1f\x4a\x05\xfe\x05\xbc", 8, 0) = 8
getrandom("\x02\x3e\x78\xe9\x85\xe0\x59\x11", 8, 0) = 8
mmap(NULL, 17179869184, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3bf2c00000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x3bf2c00000, 17179869184, "partition_alloc") = 0
munmap(0x3bf2c00000, 17179869184)       = 0
mmap(0x400000000, 17179869184, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x400000000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x400000000, 17179869184, "partition_alloc") = 0
mmap(0x3fffff000, 17179873280, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
mmap(0x3fffff000, 17179873280, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPC, si_addr=0x2ab0031548} ---
+++ killed by SIGILL (core dumped) +++
@kxxt kxxt added bug Something isn't working help wanted Extra attention is needed crash labels Aug 17, 2024
@kxxt
Copy link
Author

kxxt commented Aug 17, 2024 •
edited
Loading

The successful mmap sequence on SG2042(sv39, downstream kernel 6.1.80-2-sophgo-11457-g83ab3eda46e6 #1) is

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3f9c4fa000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3f9c4f8000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3f9c4f6000
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3f9c4f3000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3f9c4f1000
munmap(0x3f9f6f7000, 279451)            = 0
mmap(NULL, 17179869184, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3b9a200000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x3b9a200000, 17179869184, "partition_alloc") = 0
munmap(0x3b9a200000, 17179869184)       = 0
mmap(0x400000000, 17179869184, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x400000000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x400000000, 17179869184, "partition_alloc") = 0
mmap(0x3fffff000, 17179873280, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3b9a1ff000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x3b9a1ff000, 17179873280, "partition_alloc") = 0
munmap(0x3b9a1ff000, 17179873280)       = 0
mmap(0x3fffff000, 17179873280, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3b9a1ff000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x3b9a1ff000, 17179873280, "partition_alloc") = 0
munmap(0x3b9a1ff000, 17179873280)       = 0
mmap(0x3fffff000, 17179873280, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3b9a1ff000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x3b9a1ff000, 17179873280, "partition_alloc") = 0
munmap(0x3b9a1ff000, 17179873280)       = 0
mmap(0x43878000, 34359738368, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x379a200000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x379a200000, 34359738368, "partition_alloc") = 0
munmap(0x379a200000, 1709174784)        = 0
munmap(0x3c00000000, 15470690304)       = 0

And on x86_64, 6.6.45-1-lts:

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x75f482b6d000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x75f482b6b000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x75f482b69000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x75f482b67000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x75f482b65000
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x75f482b62000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x75f482b60000
munmap(0x75f4860d1000, 176119)          = 0
mmap(0x214c00000000, 17179869184, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x214c00000000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x214c00000000, 17179869184, "partition_alloc") = 0
mmap(0x7a3fffff000, 17179873280, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7a3fffff000
prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x7a3fffff000, 17179873280, "partition_alloc") = 0

@kxxt
Copy link
Author

kxxt commented Aug 18, 2024 •
edited
Loading

This has something to do with the Linux kernel.

6.10.2-arch1-1  is bad
6.9.4-arch1-1   is bad
6.9.3-arch1-1   is bad
6.8.5-arch1-1   is good
6.8.2-arch2-1   is good
6.6.3-arch1-1   is good

@kxxt kxxt changed the title Crash on unmatched Crash in partition allocator on linux >= 6.9.3 Aug 18, 2024
@kxxt
Copy link
Author

kxxt commented Aug 18, 2024

It's likely caused by this patch: https://lore.kernel.org/all/20240130-use_mmap_hint_address-v3-1-8a655cfa8bcb@rivosinc.com/

And the way it crashes is similar to the program mentioned in https://lore.kernel.org/all/tencent_83E0AB36A9A3032E5A4C4AC864A311DF9406@qq.com/#t . Although partition allocator doesn't pass the same hint address twice but two hint address that are very close to each other(0x400000000 and 0x3fffff000).

@kxxt
Copy link
Author

kxxt commented Aug 21, 2024

This is a kernel regression and probably will be fixed in the kernel in the future, as suggested by:

https://lore.kernel.org/all/mhng-7d9e2b27-a53d-4579-b78e-0aec038290fb@palmer-ri-x1c9/

@kxxt kxxt closed this as not planned Won't fix, can't repro, duplicate, stale Aug 21, 2024
@kxxt kxxt removed the help wanted Extra attention is needed label Aug 21, 2024
@kxxt kxxt changed the title Crash in partition allocator on linux >= 6.9.3 Crash in partition allocator on linux >= 6.9 Sep 2, 2024
@kxxt
Copy link
Author

kxxt commented Sep 2, 2024

Two kernel patches for fixing this:

Relevant kernel patch:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working crash
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kxxt