Panic on rollback on slot number not recorded in the Cardano transactions store

[jpraynaud]

Why

While investigating issue #1840, we have received panics from nodes operating a rollback on Cardano transactions under specific circumstances: the problem occurs when the slot number on which the chain must be rolled back does not exist in the Cardano transactions store.

Here is the error message received:

{"msg":"⟳ Preload Cardano Transactions - Started","v":0,"name":"slog-rs","level":30,"time":"2024-09-04T15:46:19.608177483Z","hostname":"jp","pid":1848856}
{"msg":"⟳ Locking signed entity type","v":0,"name":"slog-rs","level":20,"time":"2024-09-04T15:46:19.608196899Z","hostname":"jp","pid":1848856,"entity_type":"CardanoTransactions"}
{"msg":"TransactionsImporter will retrieve Cardano transactions between block_number '368' and '370'","v":0,"name":"slog-rs","level":20,"time":"2024-09-04T15:46:19.608740328Z","hostname":"jp","pid":1848856}
{"msg":"ChainReaderBlockStreamer polls next","v":0,"name":"slog-rs","level":20,"time":"2024-09-04T15:46:19.608762249Z","hostname":"jp","pid":1848856}
{"msg":"ChainScannedBlocks triggered a full RollBackward(SlotNumber(369))","v":0,"name":"slog-rs","level":20,"time":"2024-09-04T15:46:19.615742363Z","hostname":"jp","pid":1848856}
{"msg":"⟳ Releasing signed entity type","v":0,"name":"slog-rs","level":20,"time":"2024-09-04T15:46:19.615875753Z","hostname":"jp","pid":1848856,"entity_type":"CardanoTransactions"}
{"msg":"⟳ Preload Cardano Transactions - Finished","v":0,"name":"slog-rs","level":30,"time":"2024-09-04T15:46:19.615908905Z","hostname":"jp","pid":1848856}
thread 'tokio-runtime-worker' panicked at /home/jp/Works/Cardano/Mithril/mithril/mithril-aggregator/src/commands/serve_command.rs:131:64:
called `Result::unwrap()` on an `Err` value: No block number found for slot number 369

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: mithril_aggregator::database::repository::cardano_transaction_repository::<impl mithril_aggregator::services::cardano_transactions_importer::TransactionStore for mithril_persistence::database::repository::cardano_transaction_repository::CardanoTransactionRepository>::remove_rolled_back_transactions_and_block_range::{{closure}}
   2: tokio::runtime::park::CachedParkThread::block_on
   3: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
   4: tokio::runtime::task::core::Core<T,S>::poll
   5: tokio::runtime::task::harness::Harness<T,S>::poll
   6: tokio::runtime::blocking::pool::Inner::run
   7: std::sys_common::backtrace::__rust_begin_short_backtrace
   8: core::ops::function::FnOnce::call_once{{vtable.shim}}
   9: std::sys::pal::unix::thread::Thread::new::thread_start
  10: start_thread
             at ./nptl/pthread_create.c:442:8
  11: __GI___clone3
             at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
stack backtrace:
   0:     0x612ec189ef05 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hd736fd5964392270
   1:     0x612ec18cd6cb - core::fmt::write::hc6043626647b98ea
   2:     0x612ec189b07f - std::io::Write::write_fmt::h0d24b3e0473045db
   3:     0x612ec189ecde - std::sys_common::backtrace::print::h45eb8174d25a1e76
   4:     0x612ec18a02b9 - std::panicking::default_hook::{{closure}}::haf3f0170eb4f3b53
   5:     0x612ec18a005a - std::panicking::default_hook::hb5d3b27aa9f6dcda
   6:     0x612ec18a0753 - std::panicking::rust_panic_with_hook::h6b49d59f86ee588c
   7:     0x612ec18a0634 - std::panicking::begin_panic_handler::{{closure}}::hd4c2f7ed79b82b70
   8:     0x612ec189f3c9 - std::sys_common::backtrace::__rust_end_short_backtrace::h2946d6d32d7ea1ad
   9:     0x612ec18a0367 - rust_begin_unwind
  10:     0x612ec09fefc3 - core::panicking::panic_fmt::ha02418e5cd774672
  11:     0x612ec09ff4b6 - core::result::unwrap_failed::h55f86ada3ace5ed2
  12:     0x612ec0b9ad0e - mithril_aggregator::commands::serve_command::ServeCommand::execute::{{closure}}::{{closure}}::hac1c9395e2262808
  13:     0x612ec0bba3a0 - tokio::runtime::task::core::Core<T,S>::poll::h4933a132dc62d608
  14:     0x612ec0a94fa3 - tokio::runtime::task::raw::poll::hcad783a865fa91f1
  15:     0x612ec171e9d9 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h09099db2069213cd
  16:     0x612ec171df39 - tokio::runtime::scheduler::multi_thread::worker::Context::run::h6a3e524afbc25909
  17:     0x612ec1720d37 - tokio::runtime::context::scoped::Scoped<T>::set::h989babefc757debc
  18:     0x612ec1739a7d - tokio::runtime::context::runtime::enter_runtime::hd3cd638f002d8ee7
  19:     0x612ec171d756 - tokio::runtime::scheduler::multi_thread::worker::run::h96aaf4a1c0c15929
  20:     0x612ec17299d3 - tokio::runtime::task::core::Core<T,S>::poll::hf3bfbb316f2031e0
  21:     0x612ec1743292 - tokio::runtime::task::harness::Harness<T,S>::poll::h3db188a34a22ddef
  22:     0x612ec173e550 - tokio::runtime::blocking::pool::Inner::run::h5ed182888c2ac2ce
  23:     0x612ec173efef - std::sys_common::backtrace::__rust_begin_short_backtrace::haf72fd192fff9945
  24:     0x612ec172b892 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h626a9490c5f36d81
  25:     0x612ec18a727b - std::sys::pal::unix::thread::Thread::new::thread_start::hb85dbfa54ba503d6
  26:     0x7da5cce94ac3 - start_thread
                               at ./nptl/pthread_create.c:442:8
  27:     0x7da5ccf26850 - __GI___clone3
                               at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
  28:                0x0 - <unknown>
{"msg":"⇄ HTTP SERVER: certificate_pending","v":0,"name":"slog-rs","level":20,"time":"2024-09-04T15:46:19.625674481Z","hostname":"jp","pid":1848856}
{"msg":"⇄ HTTP SERVER: certificate_pending","v":0,"name":"slog-rs","level":20,"time":"2024-09-04T15:46:19.630882062Z","hostname":"jp","pid":1848856}
{"msg":"================================================================================","v":0,"name":"slog-rs","level":30,"time":"2024-09-04T15:46:19.656635617Z","hostname":"jp","pid":1848856}

What

Fix the panic and make rollback operate on the correct translated block number in the Cardano transactions store.

How

• Fix the panic when a rollback occurs if the slot number is not recorded in the Cardano transactions store