Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replica instance crashes on REPLICAOF or when killed with ctrl+c #2714

Closed
kostasrim opened this issue Mar 11, 2024 · 5 comments
Closed

replica instance crashes on REPLICAOF or when killed with ctrl+c #2714

kostasrim opened this issue Mar 11, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@kostasrim
Copy link
Contributor

To reproduce:

First issue:

  1. start dragonfly master and replica
  2. connect to replica and issue replicaof
  3. ctr + c on the replica node
dragonfly: arena.c:1013: __malloc_arena_thread_freeres: Assertion `a->attached_threads > 0' failed.
*** SIGABRT received at time=1710163399 on cpu 7 ***
PC: @     0x7f468e8969fc  (unknown)  pthread_kill
    @     0x55f676347213        192  absl::lts_20230802::WriteFailureInfo()
    @     0x55f6763477e6        256  absl::lts_20230802::AbslFailureSignalHandler()
    @     0x7f468e842520  (unknown)  (unknown)

Second issue:

  1. start dragonfly master and replica
  2. connect to replica and issue replicaof (crashes immediately)
I20240311 15:20:49.671481 151109 server_family.cc:2225] Replicating localhost:6379
*** SIGSEGV received at time=1710163249 on cpu 6 ***
PC: @     0x7efff3ea6949  (unknown)  malloc_trim
    @     0x562e96f81213        192  absl::lts_20230802::WriteFailureInfo()
    @     0x562e96f817e6        256  absl::lts_20230802::AbslFailureSignalHandler()
    @     0x7efff3e42520  (unknown)  (unknown)
[1]    151108 segmentation fault (core dumped)  ./dragonfly --port=6380 --alsologtostderr

@kostasrim kostasrim added the bug Something isn't working label Mar 11, 2024
@kostasrim kostasrim changed the title replica instance crashes with ctrl+c replica instance crashes on REPLICAOF or when killed with ctrl+c Mar 11, 2024
@romange
Copy link
Collaborator

romange commented Mar 11, 2024

it's something new. @chakaz please check if it reproduces before and after the SSL commit.

@kostasrim
Copy link
Contributor Author

kostasrim commented Mar 11, 2024

yes I know, it happened when I merged main to one of my PR's so it should be one of the last commits.

I wonder how unlucky we were that reg tests did not catch this.

@chakaz
Copy link
Collaborator

chakaz commented Mar 12, 2024

Kostas and I discussed this today, and it looks like after a clean build this is not reproducible. Kostas will continue to try to repro it. Could you kindly update this bug with your findings?

@romange I'm not saying we shouldn't cleanup SSL, only that if it's reproducible, we should test with and without the fix.

@romange
Copy link
Collaborator

romange commented Mar 13, 2024

@chakaz consider compiling mimalloc in debug mode with assertions enabled to reproduce.

@chakaz
Copy link
Collaborator

chakaz commented Mar 19, 2024

Adi was able to reproduce it via pytest (even though it doesn't repro for me locally), so lets continue in #2737

@chakaz chakaz closed this as not planned Won't fix, can't repro, duplicate, stale Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants