Track confirmation block hash and return via `Confirm::get_relevant_txids` #1796

tnull · 2022-10-24T12:18:35Z

Previously, Confirm::get_relevant_txids() only returned a list of transactions that have to be monitored for reorganization out of the chain. This interface however required a double bookkeeping: while we internally keep track of the best block, height, etc, it would also require the user to keep track which transaction was previously confirmed in which block and to take actions based on any change, e.g, to reconfirm them when the block would be reorged-out and the transactions had been reconfirmed in another block.

Here, we track the confirmation block hash internally and return it via Confirm::get_relevant_txids() to the user, which alleviates the requirement for double bookkeeping: the user can now simply check whether the given transactions are still confirmed via the given blocks, and take action if they are not.

tnull · 2022-10-24T12:19:20Z

Marked draft for now, will switch after concept ACK.

codecov-commenter · 2022-10-24T12:38:08Z

Codecov Report

Base: 90.78% // Head: 91.23% // Increases project coverage by +0.44% 🎉

Coverage data is based on head (de9cd52) compared to base (e55e0d5).
Patch coverage: 89.39% of modified lines in pull request are covered.

❗ Current head de9cd52 differs from pull request most recent head 9685d6c. Consider uploading reports for the commit 9685d6c to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1796      +/-   ##
==========================================
+ Coverage   90.78%   91.23%   +0.44%     
==========================================
  Files          87       89       +2     
  Lines       47604    50973    +3369     
  Branches    47604    50973    +3369     
==========================================
+ Hits        43218    46503    +3285     
- Misses       4386     4470      +84

Impacted Files	Coverage Δ
lightning/src/chain/chainmonitor.rs	`97.78% <ø> (ø)`
lightning/src/chain/mod.rs	`68.18% <ø> (ø)`
lightning/src/chain/channelmonitor.rs	`90.69% <80.64%> (+0.04%)`	⬆️
lightning/src/chain/onchaintx.rs	`95.08% <90.00%> (+0.07%)`	⬆️
lightning/src/ln/channel.rs	`88.71% <100.00%> (+<0.01%)`	⬆️
lightning/src/ln/channelmanager.rs	`88.85% <100.00%> (+3.44%)`	⬆️
lightning/src/ln/reorg_tests.rs	`100.00% <100.00%> (ø)`
lightning-net-tokio/src/lib.rs	`76.73% <0.00%> (-0.31%)`	⬇️
lightning/src/ln/functional_tests.rs	`96.95% <0.00%> (-0.11%)`	⬇️
lightning/src/lib.rs	`100.00% <0.00%> (ø)`
... and 12 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

tnull · 2022-10-24T17:28:51Z

Marked draft for now, will switch after concept ACK.

Just discussed this with Matt in Discord, I'll count this as concept ACK.

arik-so · 2022-10-27T02:09:49Z

Would it be helpful to also return the confirmation height, so a user could trivially verify if a transaction has been dropped by knowing the depth of recent reorgs?

lightning/src/chain/channelmonitor.rs

lightning/src/chain/onchaintx.rs

lightning/src/ln/channelmanager.rs

lightning/src/ln/reorg_tests.rs

tnull · 2022-10-27T07:15:40Z

Would it be helpful to also return the confirmation height, so a user could trivially verify if a transaction has been dropped by knowing the depth of recent reorgs?

Not sure, as the height to compare it to is always due to change meaning it may be ambiguous what 'recent reorgs' are. So including height might invite users to do some comparisons that are likely gonna be race-y again?

wpaulino

LGTM, feel free to squash.

lightning/src/chain/channelmonitor.rs

TheBlueMatt · 2022-10-31T19:17:01Z

This really needs some kind of testing. It should be pretty straightforward to just add relevant assertions in some of the existing reorg tests, I hope.

tnull · 2022-11-03T12:02:05Z

This really needs some kind of testing. It should be pretty straightforward to just add relevant assertions in some of the existing reorg tests, I hope.

Now added a test that we're indeed getting the expected confirmation block hash with c937807. Let me know if it's fine to squash.

TheBlueMatt · 2022-11-04T19:31:13Z

lightning/src/chain/onchaintx.rs

@@ -548,7 +552,7 @@ impl<ChannelSigner: Sign> OnchainTxHandler<ChannelSigner> {
 	/// `conf_height` represents the height at which the transactions in `txn_matched` were
 	/// confirmed. This does not need to equal the current blockchain tip height, which should be
 	/// provided via `cur_height`, however it must never be higher than `cur_height`.
-	pub(crate) fn update_claims_view<B: Deref, F: Deref, L: Deref>(&mut self, txn_matched: &[&Transaction], requests: Vec<PackageTemplate>, conf_height: u32, cur_height: u32, broadcaster: &B, fee_estimator: &LowerBoundedFeeEstimator<F>, logger: &L)
+	pub(crate) fn update_claims_view<B: Deref, F: Deref, L: Deref>(&mut self, txn_matched: &[&Transaction], requests: Vec<PackageTemplate>, conf_height: u32, conf_hash: BlockHash, cur_height: u32, broadcaster: &B, fee_estimator: &LowerBoundedFeeEstimator<F>, logger: &L)


I know its not really your fault, but man this is confusing - update_claims_view takes block hash and height parameters for "what block/height this tx was included in" but is often called without any txn and dummy (current block) values. Can we split the function in two and have the part that operates on txn_matched be separate? Should be a straightforward change and would make this PR much more clearly correct.

Now split the method into update_claims_view_from_requests and update_claims_view_from_matched_txn in 0f9fc8a. I also DRYed the onchain event processing that would always run into process_onchain_events_awaiting_threshold_conf.

TheBlueMatt · 2022-11-07T18:35:16Z

lightning/src/chain/onchaintx.rs

 	/// `conf_height` represents the height at which the transactions in `txn_matched` were
 	/// confirmed. This does not need to equal the current blockchain tip height, which should be
 	/// provided via `cur_height`, however it must never be higher than `cur_height`.
-	pub(crate) fn update_claims_view<B: Deref, F: Deref, L: Deref>(&mut self, txn_matched: &[&Transaction], requests: Vec<PackageTemplate>, conf_height: u32, cur_height: u32, broadcaster: &B, fee_estimator: &LowerBoundedFeeEstimator<F>, logger: &L)
+	pub(crate) fn update_claims_view_from_requests<B: Deref, F: Deref, L: Deref>(&mut self,
+		requests: Vec<PackageTemplate>, conf_height: u32, cur_height: u32, broadcaster: &B,


~~Should drop the conf_height - nothing is being confirmed. Can also drop the corresponding docs which talk about txn_matched.~~

Can you fix the above docs on conf_height which mentions txn_matched?

Done hope I'm not misinterpreting something here:

$ git diff-tree -U2 0f9fc8a 84073d diff --git a/lightning/src/chain/onchaintx.rs b/lightning/src/chain/onchaintx.rs index ee557eb7..8fbfa215 100644 --- a/lightning/src/chain/onchaintx.rs +++ b/lightning/src/chain/onchaintx.rs @@ -552,7 +552,7 @@ impl<ChannelSigner: Sign> OnchainTxHandler<ChannelSigner> { /// preimage after force-close. /// - /// `conf_height` represents the height at which the transactions in `txn_matched` were - /// confirmed. This does not need to equal the current blockchain tip height, which should be - /// provided via `cur_height`, however it must never be higher than `cur_height`. + /// `conf_height` represents the minimal height at which the request could get confirmed. This + /// does not need to equal the current blockchain tip height, which should be provided via + /// `cur_height`, however it must never be higher than `cur_height`. pub(crate) fn update_claims_view_from_requests<B: Deref, F: Deref, L: Deref>(&mut self, requests: Vec<PackageTemplate>, conf_height: u32, cur_height: u32, broadcaster: &B,

TheBlueMatt · 2022-11-07T18:41:05Z

Basically LGTM, feel free to squash from my PoV, but will need re-review from multiple reviewers here.

tnull · 2022-11-07T18:48:54Z

Squashed commits.

wpaulino

LGTM

lightning/src/chain/onchaintx.rs

jkczyz

Just some nits.

lightning/src/chain/onchaintx.rs

lightning/src/ln/reorg_tests.rs

tnull · 2022-11-08T18:19:37Z

Addressed nits, squashed again (and accidentally rebased on main in the process... Sorry 😢 )

tnull · 2022-11-08T18:33:25Z

Recovered the most recent base before the accident. So the latest squash diff is:

> git diff-tree -U2  1a9fec5 1a80eac
diff --git a/lightning/src/chain/onchaintx.rs b/lightning/src/chain/onchaintx.rs
index aa1116ae..14b5a298 100644
--- a/lightning/src/chain/onchaintx.rs
+++ b/lightning/src/chain/onchaintx.rs
@@ -555,10 +555,11 @@ impl<ChannelSigner: Sign> OnchainTxHandler<ChannelSigner> {
 	/// does not need to equal the current blockchain tip height, which should be provided via
 	/// `cur_height`, however it must never be higher than `cur_height`.
-	pub(crate) fn update_claims_view_from_requests<B: Deref, F: Deref, L: Deref>(&mut self,
-		requests: Vec<PackageTemplate>, conf_height: u32, cur_height: u32, broadcaster: &B,
-		fee_estimator: &LowerBoundedFeeEstimator<F>, logger: &L)
-		where B::Target: BroadcasterInterface,
-			  F::Target: FeeEstimator,
-			  L::Target: Logger,
+	pub(crate) fn update_claims_view_from_requests<B: Deref, F: Deref, L: Deref>(
+		&mut self, requests: Vec<PackageTemplate>, conf_height: u32, cur_height: u32,
+		broadcaster: &B, fee_estimator: &LowerBoundedFeeEstimator<F>, logger: &L
+	) where
+			B::Target: BroadcasterInterface,
+			F::Target: FeeEstimator,
+			L::Target: Logger,
 	{
 		log_debug!(logger, "Updating claims view at height {} with {} claim requests", cur_height, requests.len());
@@ -654,10 +655,11 @@ impl<ChannelSigner: Sign> OnchainTxHandler<ChannelSigner> {
 	/// confirmed. This does not need to equal the current blockchain tip height, which should be
 	/// provided via `cur_height`, however it must never be higher than `cur_height`.
-	pub(crate) fn update_claims_view_from_matched_txn<B: Deref, F: Deref, L: Deref>(&mut self,
-		txn_matched: &[&Transaction], conf_height: u32, conf_hash: BlockHash, cur_height: u32,
-		broadcaster: &B, fee_estimator: &LowerBoundedFeeEstimator<F>, logger: &L)
-		where B::Target: BroadcasterInterface,
-			  F::Target: FeeEstimator,
-			  L::Target: Logger,
+	pub(crate) fn update_claims_view_from_matched_txn<B: Deref, F: Deref, L: Deref>(
+		&mut self, txn_matched: &[&Transaction], conf_height: u32, conf_hash: BlockHash,
+		cur_height: u32, broadcaster: &B, fee_estimator: &LowerBoundedFeeEstimator<F>, logger: &L
+	) where
+			B::Target: BroadcasterInterface,
+			F::Target: FeeEstimator,
+			L::Target: Logger,
 	{
 		log_debug!(logger, "Updating claims view at height {} with {} matched transactions in block {}", cur_height, txn_matched.len(), conf_height);
diff --git a/lightning/src/ln/reorg_tests.rs b/lightning/src/ln/reorg_tests.rs
index 6d4fe16f..1e26d339 100644
--- a/lightning/src/ln/reorg_tests.rs
+++ b/lightning/src/ln/reorg_tests.rs
@@ -285,7 +285,6 @@ fn do_test_unconf_chan(reload_node: bool, reorg_after_reload: bool, use_funding_
 			assert_eq!(relevant_txids.len(), 1);
 			let block_hash_opt = relevant_txids[0].1;
-			assert!(block_hash_opt.is_some());
 			let expected_hash = nodes[0].get_block_header(chan_conf_height).block_hash();
-			assert_eq!(block_hash_opt.unwrap(), expected_hash);
+			assert_eq!(block_hash_opt, Some(expected_hash));
 			let txid = relevant_txids[0].0;
 			assert_eq!(txid, chan.3.txid());
@@ -364,7 +363,6 @@ fn do_test_unconf_chan(reload_node: bool, reorg_after_reload: bool, use_funding_
 			assert_eq!(relevant_txids.len(), 1);
 			let block_hash_opt = relevant_txids[0].1;
-			assert!(block_hash_opt.is_some());
 			let expected_hash = nodes[0].get_block_header(chan_conf_height).block_hash();
-			assert_eq!(block_hash_opt.unwrap(), expected_hash);
+			assert_eq!(block_hash_opt, Some(expected_hash));
 			let txid = relevant_txids[0].0;
 			assert_eq!(txid, chan.3.txid());

lightning/src/chain/mod.rs

lightning/src/chain/onchaintx.rs

TheBlueMatt · 2022-11-08T18:46:16Z

FWIW, in the future, the update_claims_view split could be in a separate commit. Not worth trying to split it now that its done, though.

tnull · 2022-11-08T19:28:55Z

FWIW, in the future, the update_claims_view split could be in a separate commit. Not worth trying to split it now that its done, though.

Alright, split it out again.

TheBlueMatt

Two doc nits, feel free to ignore unless/until you have more review comments from others.

lightning/src/chain/mod.rs

lightning/src/chain/onchaintx.rs

Previously, `Confirm::get_relevant_txids()` only returned a list of transactions that have to be monitored for reorganization out of the chain. This interface however required double bookkeeping: while we internally keep track of the best block, height, etc, it would also require the user to keep track which transaction was previously confirmed in which block and to take actions based on any change, e.g, to reconfirm them when the block would be reorged-out and the transactions had been reconfirmed in another block. Here, we track the confirmation block hash internally and return it via `Confirm::get_relevant_txids()` to the user, which alleviates the requirement for double bookkeeping: the user can now simply check whether the given transaction is still confirmed and in the given block, and take action if not. We also split `update_claims_view`: Previously it was one, now it's two methods: `update_claims_view_from_matched_txn` and `update_claims_view_from_requests`.

tnull · 2022-11-09T10:14:11Z

Addressed nits and squashed again, CI should be happy now.

jkczyz · 2022-11-09T18:08:17Z

Addressed nits and squashed again, CI should be happy now.

Didn't we go with two commits earlier?

TheBlueMatt · 2022-11-09T18:37:40Z

Yea, we did split it before, but not sure its worth bothering to hold this up more for that - the total diff is only +129 −60 so its not like its big enough that its hard to read.

tnull marked this pull request as draft October 24, 2022 12:18

tnull mentioned this pull request Oct 24, 2022

Initial chain access implementation lightningdevkit/ldk-node#9

Closed

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from c1d82ac to 0ff16b4 Compare October 24, 2022 13:27

jkczyz self-requested a review October 24, 2022 13:55

tnull added Seeking Code Review Seeking Conceptual Review and removed Seeking Conceptual Review labels Oct 24, 2022

tnull marked this pull request as ready for review October 24, 2022 17:28

tnull added this to the 0.0.113 milestone Oct 25, 2022

tnull mentioned this pull request Oct 25, 2022

Migration to rust-lightning 0.0.113 lightningdevkit/ldk-node#28

Closed

3 tasks

jkczyz reviewed Oct 27, 2022

View reviewed changes

wpaulino reviewed Oct 31, 2022

View reviewed changes

lightning/src/chain/channelmonitor.rs Outdated Show resolved Hide resolved

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from cd2378a to c937807 Compare November 3, 2022 11:59

TheBlueMatt removed the Seeking Code Review label Nov 3, 2022

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from c937807 to 24c5dde Compare November 4, 2022 10:46

TheBlueMatt reviewed Nov 4, 2022

View reviewed changes

TheBlueMatt reviewed Nov 7, 2022

View reviewed changes

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from 0f9fc8a to 84073d6 Compare November 7, 2022 18:48

tnull requested review from jkczyz and wpaulino November 7, 2022 19:29

wpaulino reviewed Nov 7, 2022

View reviewed changes

lightning/src/chain/onchaintx.rs Outdated Show resolved Hide resolved

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from 84073d6 to 1a9fec5 Compare November 8, 2022 08:59

jkczyz previously approved these changes Nov 8, 2022

View reviewed changes

lightning/src/chain/onchaintx.rs Outdated Show resolved Hide resolved

lightning/src/chain/onchaintx.rs Outdated Show resolved Hide resolved

lightning/src/ln/reorg_tests.rs Outdated Show resolved Hide resolved

lightning/src/ln/reorg_tests.rs Outdated Show resolved Hide resolved

tnull dismissed jkczyz’s stale review via e6b7bc2 November 8, 2022 18:13

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from 1a9fec5 to e6b7bc2 Compare November 8, 2022 18:13

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from e6b7bc2 to 1a80eac Compare November 8, 2022 18:30

TheBlueMatt reviewed Nov 8, 2022

View reviewed changes

lightning/src/chain/mod.rs Show resolved Hide resolved

lightning/src/chain/onchaintx.rs Outdated Show resolved Hide resolved

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from 1a80eac to ee34469 Compare November 8, 2022 19:28

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from ee34469 to 2084f11 Compare November 8, 2022 20:46

TheBlueMatt previously approved these changes Nov 8, 2022

View reviewed changes

lightning/src/chain/mod.rs Outdated Show resolved Hide resolved

lightning/src/chain/onchaintx.rs Outdated Show resolved Hide resolved

jkczyz previously approved these changes Nov 8, 2022

View reviewed changes

lightning/src/chain/onchaintx.rs Outdated Show resolved Hide resolved

lightning/src/chain/onchaintx.rs Outdated Show resolved Hide resolved

tnull dismissed stale reviews from jkczyz and TheBlueMatt via de9cd52 November 9, 2022 08:27

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from 2084f11 to de9cd52 Compare November 9, 2022 08:27

tnull force-pushed the 2022-10-track-confirmation-block-hash branch from de9cd52 to 9685d6c Compare November 9, 2022 10:13

TheBlueMatt approved these changes Nov 9, 2022

View reviewed changes

jkczyz approved these changes Nov 9, 2022

View reviewed changes

wpaulino approved these changes Nov 9, 2022

View reviewed changes

TheBlueMatt merged commit b6fce3d into lightningdevkit:main Nov 9, 2022

tnull mentioned this pull request Jan 31, 2023

Add transaction sync crate #1870

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Track confirmation block hash and return via `Confirm::get_relevant_txids` #1796

Track confirmation block hash and return via `Confirm::get_relevant_txids` #1796

tnull commented Oct 24, 2022 •

edited

Loading

tnull commented Oct 24, 2022

codecov-commenter commented Oct 24, 2022 •

edited

Loading

tnull commented Oct 24, 2022 •

edited

Loading

arik-so commented Oct 27, 2022

tnull commented Oct 27, 2022

wpaulino left a comment

TheBlueMatt commented Oct 31, 2022

tnull commented Nov 3, 2022

TheBlueMatt Nov 4, 2022

tnull Nov 7, 2022 •

edited

Loading

TheBlueMatt Nov 7, 2022 •

edited

Loading

tnull Nov 7, 2022 •

edited

Loading

TheBlueMatt commented Nov 7, 2022

tnull commented Nov 7, 2022

wpaulino left a comment

jkczyz left a comment

tnull commented Nov 8, 2022

tnull commented Nov 8, 2022

TheBlueMatt commented Nov 8, 2022

tnull commented Nov 8, 2022

TheBlueMatt left a comment

tnull commented Nov 9, 2022 •

edited

Loading

jkczyz commented Nov 9, 2022

TheBlueMatt commented Nov 9, 2022

Track confirmation block hash and return via Confirm::get_relevant_txids #1796

Track confirmation block hash and return via Confirm::get_relevant_txids #1796

Conversation

tnull commented Oct 24, 2022 • edited Loading

tnull commented Oct 24, 2022

codecov-commenter commented Oct 24, 2022 • edited Loading

Codecov Report

tnull commented Oct 24, 2022 • edited Loading

arik-so commented Oct 27, 2022

tnull commented Oct 27, 2022

wpaulino left a comment

Choose a reason for hiding this comment

TheBlueMatt commented Oct 31, 2022

tnull commented Nov 3, 2022

TheBlueMatt Nov 4, 2022

Choose a reason for hiding this comment

tnull Nov 7, 2022 • edited Loading

Choose a reason for hiding this comment

TheBlueMatt Nov 7, 2022 • edited Loading

Choose a reason for hiding this comment

tnull Nov 7, 2022 • edited Loading

Choose a reason for hiding this comment

TheBlueMatt commented Nov 7, 2022

tnull commented Nov 7, 2022

wpaulino left a comment

Choose a reason for hiding this comment

jkczyz left a comment

Choose a reason for hiding this comment

tnull commented Nov 8, 2022

tnull commented Nov 8, 2022

TheBlueMatt commented Nov 8, 2022

tnull commented Nov 8, 2022

TheBlueMatt left a comment

Choose a reason for hiding this comment

tnull commented Nov 9, 2022 • edited Loading

jkczyz commented Nov 9, 2022

TheBlueMatt commented Nov 9, 2022

Track confirmation block hash and return via `Confirm::get_relevant_txids` #1796

Track confirmation block hash and return via `Confirm::get_relevant_txids` #1796

tnull commented Oct 24, 2022 •

edited

Loading

codecov-commenter commented Oct 24, 2022 •

edited

Loading

tnull commented Oct 24, 2022 •

edited

Loading

tnull Nov 7, 2022 •

edited

Loading

TheBlueMatt Nov 7, 2022 •

edited

Loading

tnull Nov 7, 2022 •

edited

Loading

tnull commented Nov 9, 2022 •

edited

Loading