-
Notifications
You must be signed in to change notification settings - Fork 160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
db-sync crashes when MIR cert transaction to unregistered stake address is submitted #845
Comments
This crash is rewards related. Rewards are currently known to be broken. |
This is a |
What network is this? |
It's on local cluster, no testnet. |
Supposedly a transaction that was supposed to trigger this issue was submitted to the When running on that network I ran into some other problems (#882) and after figuring out that problem was relatively benign and adding code to skip that issue I find that
Stopping |
I traced this hang back to where it reads blocks from queue. Blocking (ie hanging) it the correct behaviour when the queue is and remains empty. Looking at the other end of the queue, it turns out that nothing was being fed into the queue, because no blocks were being received from the This is all correct and expected behavior as far as Rerunning both the |
On resync from geneisis, Someone from devops has confirmed that the |
@mkoura your comment:
does not seem to be correct. That "warning" is not a crash. Its a log message. If WHile it does not seem to have "crashed" it does seem to have halted, but its doing that because its waiting for more input (ie more blocks from the |
|
That doesn't seem like the problem I'm observing. My node is running with no issues and keeps creating blocks even after db-sync crashes. Db-sync process exits with exit code 1 (i.e. not hanging), every single time, 100% reproducible. Hence it is a crash. My comment
means the warning is the last message before the crash (there's no error in the log file etc.) |
Its reproducible by you, but I have not yet been able to, despite sowing several days of effort into the attempt. I cannot fix this, if I cannot reproduce it. |
I can think of two ways
|
With assistance of QA I tried to reproduce this with
test with flying colours. No hang, no crash. |
I'm still able to reproduce it with up-to-date master ( I ran db-sync in the foreground so it dumps it's
|
@mkoura The original bug report claimed that The above obviously does have information about why it is exiting (ie the lines starting at "Epoch No: 39") . However, this is almost certainly due to #882 which is a known issue that needs to be fixed in If the original claim is correct, that |
If #882 doesn't crash, it's different issue. You can reproduce this issue either by following the steps in description, or by using the node DB and configuration where the issue was already reproduced that I have provided to you. |
#882 prints an error message and then exits. Exiting is a deliberate action, not a crash |
#882 doesn't mention anything about exit or crash (October 19, 2021, 9:10 UTC) |
Doesn't mention it, but it does exit. |
This was never a "crash without any error message". After updating the dependencies to fix the issue in #882, I still see:
Looking at that address: select stake_registration.* from stake_registration inner join stake_address
on stake_address.id = stake_registration.addr_id
where hash_raw = '\xe0b1d7a6db1859b8d24def65f87554e4549546186e4cb65b55a5e51399' ;
id | addr_id | cert_index | epoch_no | tx_id
----+---------+------------+----------+-------
8 | 21 | 0 | 7 | 14
(1 row) and almost immediately deregistered: select stake_deregistration.* from stake_deregistration inner join stake_address
on stake_address.id = stake_deregistration.addr_id
where hash_raw = '\xe0b1d7a6db1859b8d24def65f87554e4549546186e4cb65b55a5e51399' ;
id | addr_id | cert_index | epoch_no | tx_id | redeemer_id
----+---------+------------+----------+-------+-------------
2 | 21 | 0 | 7 | 16 |
(1 row) but then does receive a payment from the treasury: select reward.* from reward
inner join stake_address on reward.addr_id = stake_address.id
where stake_address.hash_raw = '\xe0b1d7a6db1859b8d24def65f87554e4549546186e4cb65b55a5e51399' ;
id | addr_id | type | amount | earned_epoch | spendable_epoch | pool_id
----+---------+----------+---------------+--------------+-----------------+---------
17 | 21 | treasury | 1500000000000 | 8 | 9 |
(1 row) If the address has been de-registered it should not be receiving rewards. The important factor here is not that has been registered and is currently unregistered, rather than it never having been registered at all. Looks like this is yet another indication that trying to have two implementations of the same logc (ie in |
Fixing this in
which shows the stake address receiving a MIR payment after it is deregistered. This points to an issue in The |
Finally fixed this in Have not been able to recreate any crash or hang without an error message. Closing this. |
OS
Linux - Fedora 34
Versions
The
db-sync
version:(yesterday's master)
PostgreSQL version: 11.11
Build/Install Method
The method you use to build or install
cardano-db-sync
: nix-buildRun method
The method you used to run
cardano-db-sync
(eg Nix/Docker/systemd/none): supervisordAdditional context
Latest master node running with latest master db-sync, Alonzo era, Protocol Version 6.
Problem Report
I have a test that tries to send funds from reserves or treasury pot to an unregistered stake address.
The test:
After the transaction is submitted, db-sync crashes at the epoch boundary without any error in the log files, there's just warning about rewards earned:
After restarting it keeps crashing with the same warning.
The text was updated successfully, but these errors were encountered: