Epoch 285 catalyst rewards mismatch (bug) #769
Comments
|
This is almost certainly related to #696 .
|
|
This was actually caused by faulty catalyst rewards, where in epoch 285, two catalyst rewards showed up for all participants, but the first of the two did not make it through to node and cli, but it appeared in db-sync, which threw balances off. Samuel Leathers can tell you more if interested. |
The |
Let's say stake1u9schrkl2ed8daympxqxd38z72kx00y69ky8udd7g4v53kcp5kkyg. The cardano-cli reward amount was But if you calculate the rewards from db-sync, the outstanding reward balance is 10793839. The difference is 43604, which corresponds to the first of the two catalyst rewards for epoch 285. |
Quoting Samuel Leathers:
|
|
@xdzurman Unfortunately, your
On current master (after merging of the reward validation) I get: select reward.* from reward inner join stake_address on reward.addr_id = stake_address.id
where stake_address.view = 'stake1u9schrkl2ed8daympxqxd38z72kx00y69ky8udd7g4v53kcp5kkyg'
order by earned_epoch ;
id | addr_id | type | amount | pool_id | earned_epoch | spendable_epoch
----------+---------+----------+---------+---------+--------------+-----------------
13040354 | 1895674 | member | 353659 | 7699 | 273 | 275
13651004 | 1895674 | member | 238593 | 7699 | 274 | 276
14271526 | 1895674 | member | 940700 | 7699 | 275 | 277
14900823 | 1895674 | member | 1352813 | 7699 | 276 | 278
15547990 | 1895674 | treasury | 39108 | | 277 | 279
15547989 | 1895674 | member | 493762 | 7699 | 277 | 279
16210475 | 1895674 | member | 1371629 | 7699 | 278 | 280
16867777 | 1895674 | member | 1236043 | 7699 | 279 | 281
17533273 | 1895674 | member | 1004487 | 7699 | 280 | 282
18204762 | 1895674 | member | 1112734 | 7699 | 281 | 283
18883060 | 1895674 | member | 895610 | 7699 | 282 | 284
19567131 | 1895674 | member | 851842 | 7699 | 283 | 285
20268768 | 1895674 | member | 845867 | 7699 | 284 | 286
20268769 | 1895674 | reserves | 13388 | | 284 | 286
20986850 | 1895674 | member | 973891 | 7699 | 285 | 287
21694133 | 1895674 | member | 225910 | 7699 | 286 | 288
23036583 | 1895674 | member | 518683 | 7699 | 287 | 289
(17 rows)and select withdrawal.* from withdrawal inner join stake_address on withdrawal.addr_id = stake_address.id
where stake_address.view = 'stake1u9schrkl2ed8daympxqxd38z72kx00y69ky8udd7g4v53kcp5kkyg';
id | addr_id | amount | tx_id | redeemer_id
----+---------+--------+-------+-------------
(0 rows)Does that seem correct? |
|
Yes, this seems correct. Until now, we queried ITN and catalyst rewards from one of which should not be there. That was the whole issue. |
Also, can you check rewards for Rewards from db sync 9.0.0: Rewards from db sync 11.0.0 Could the fix be patch released until the hard fork? |
|
@xdzurman how do you know |
|
That is the version everyone rolled back to after 10.0.0 introduced rewards issues. It has been deployed to production for a long time and stood the test of time (except a minor tweak regarding epoch 285's catalyst rewards). |
|
Running master against testnet, there are couple of these as well and one needs to disable |
It may well have stood the test of time, but it may still be wrong. |
If the |
Created #805 |
Yes, they may have rolled back to |
select hash_raw from stake_address where view = 'stake1ux025zswj96df8esu78wqdhvfc56yg9yzztwhj4hgkp0rqswd7rl7' ;
hash_raw
--------------------------------------------------------------
\xe19eaa0a0e9174d49f30e78ee036ec4e29a220a41096ebcab74582f182
(1 row)and then using the stake history tool in the shows that yes indeed, rewards for this address should start in epoch 238. I think this is basically a duplicate of #769 . |
I do not think that is the case. I mean, at least it was much more "correct" (like a fraction of a percent might have been incorrect). Otherwise we would see a lot more failed reward withdrawal transactions in Yoroi, as we would construct the transaction with incorrect reward in place and get rejected by the ledger. With 9.0.0, I really hardly recall any of it. And as far as I know this has been the case with Adalite too, as they have upgraded from 9.0.0. |
|
Same here. Users of adalite were always able to withdraw their funds while we were using 9.0.0 (we compute the reward amounts as a sum of all types of rewards minus withdrawals). This always worked, except the ones that retired their pools and got the deposit back, for which I created an issue more that 9 months ago. This issue (catalyst's epoch 285 double rewards) was the second major issue, but apart from that version 9.0.0 was correct (based on the Adalite's historical user logs and reported issues). However, when switching to db synx 10.0.0, a huge portion of users had wrong rewards. These issues are still unresolved in 11.0.0, which forces us to show warning banners to our users that their reward history is most probably wrong. We also had to resort to ignore the reward sum from db sync and query it via ogmios (bloating our backends just for this one piece of information) so that the users can at least withdraw their funds. Ever since having to move from db sync 9.0.0, we are impatiently pleading for fixes to the major issues, so that we don't have to compromise the truthfulness of the data we serve to our users. |
|
Hopefully this is fixed on https://github.com/input-output-hk/cardano-db-sync/releases/tag/13.0.0. Feel free to reopen if the issue persists. |
There is a mismatch between cardano-db-sync catalyst rewards and cardano-cli (ledger). The db sync states 2 catalyst rewards which appeared in epoch 285. However, the sooner one of them is invisible to cardano-cli. A few examples:
https://cardanoscan.io/stakekey/05c6aa84de1fd5d5d15fa2cc0f30a5c804f52fb19725525aab54a839?tab=instantaneousrewards
Here you can see the two catalyst rewards which appeared 5 days ago. AFAIK cardanoscan simply sums up the rewards minus withdrawals. The reward sum with the two catalyst rewards for this address is 2649355 Lovelaces. According to cardano-cli,
"rewardAccountBalance": 2629950. The difference is 19405 Lovelace, which corresponds to the sooner catalyst reward of the two.Same here - https://cardanoscan.io/stakekey/stake1uyzuqsnynkxkqs848y8mgylywwezrgcrmd6xmqspx4kfk7gqtlzjv?tab=instantaneousrewards
Reward sum: 103826843
Cardano-cli reward: 103654676
Diff: 172167 = equal to the sooner of the two rewards
The same goes for basically any address you look up.
Db sync version: 9.0.0
This caused Adalite users not being to withdraw rewards until we started filtering out the first of epoch 285's rewards.
The text was updated successfully, but these errors were encountered: