sweep: add wallet inputs to reach dust limit #3814

joostjager · 2019-12-10T16:45:04Z

This PR enables the sweeper to sweep inputs that are positively-yielding, but by itself wouldn't reach the dust limit for the sweep transaction output value at the chosen fee rate.

They way to deal with this is to attach additional wallet input(s) to bring up the output value of the transaction.

This is a preparation for sweeping anchor outputs that are too small to reach the dust limit of the sweep tx output.

There are also a few commits in this PR that aren't strictly necessary for the functionality that this PR adds, but are included anyway to prevent touching the code again in the follow-up.

Next step after this PR is to add forced sweeping of negatively yielding inputs.

We need it to be set in order to properly test the sweeper handling the dust limit on regtest.

Using a fee rate just as an identifier can be confusing.

Fixes a bug where bucket sizes were not the configured size, but the configured size plus the min relay fee.

Prepares for adding more input-specific sweep parameters.

Get rid of needless referencing of the embedded object.

halseth

Good PR! Excellent commit structure, and a nice cleanup :)

sweep/sweeper.go

sweep/tx_input_set.go

sweep/sweeper.go

sweep/tx_input_set.go

sweep/txgenerator.go

sweep/tx_input_set.go

sweep/txgenerator.go

joostjager · 2019-12-13T13:07:02Z

Unit test added

sweep/sweeper.go

sweep/tx_input_set.go

sweep/sweeper.go

sweep/tx_input_set.go

halseth · 2019-12-17T09:55:52Z

sweep/tx_input_set.go

+	t.inputTotal = newInputTotal
+	t.outputValue = newOutputValue
+	t.inputs = append(t.inputs, input)
+	t.weightEstimate = newWeightEstimate


looks like all these counters could be calculated on demand instead (just pointing it out, still up for discussion whether that is better than just keeping running counters).

In a later commit, i would also need the fromWallet state. I am not too convinced of it. When you add 20 inputs, 200 (virtual) add operations will be executed. (1+2+3+...+20).

@wpaulino what do you think?

Adding a new input would make us recalculate everything, but the performance impact is likely negligible unless there are lots of inputs. I'm fine with leaving things as is given that the code is already there.

sweep/txgenerator.go

sweep/sweeper.go

sweep/tx_input_set.go

halseth · 2019-12-17T10:15:35Z

sweep/tx_input_set.go

+
+	// Retrieve wallet utxos. Only consider confirmed utxos to prevent
+	// problems around RBF rules for unconfirmed inputs.
+	utxos, err := t.wallet.ListUnspentWitness(1, math.MaxInt32)


to avoid having the wallet as a field on the input set, the list of utxos can be passed into this method.

yes, but then we'd always get utxos, even if the dust limit is already reached. could also move that out then. but not sure if it is better

sweep/interface.go

wpaulino · 2019-12-17T18:27:00Z

sweep/tx_input_set.go

+	t.inputTotal = newInputTotal
+	t.outputValue = newOutputValue
+	t.inputs = append(t.inputs, input)
+	t.weightEstimate = newWeightEstimate


Adding a new input would make us recalculate everything, but the performance impact is likely negligible unless there are lots of inputs. I'm fine with leaving things as is given that the code is already there.

sweep/tx_input_set.go

sweep/sweeper.go

sweep/tx_input_set_test.go

A refactoring that introduces no functional changes. This prepares for the addition of wallet utxos to push the sweep tx above the dust limit. It also enabled access to input-specific sweep parameters during tx generation. This will be used in later commits to control the sweep process.

We need access to additional wallet functionality. This commit creates an interface to prevent passing in multiple function pointers.

Prepares for adding another level of nesting.

This commit allows sweeper to sweep inputs that on its own are not able to form a sweep transaction that meets the dust limit. This functionality is useful for sweeping small outputs. In the future, this will be particularly important to sweep anchors. Anchors will typically be spent with a relatively large fee to pay for the parent tx. It will then be necessary to attach an additional wallet utxo.

halseth

LGTM 👍

joostjager requested review from cfromknecht and Roasbeef as code owners December 10, 2019 16:45

joostjager changed the title ~~sweep: add wallet inputs to reach dust limit~~ sweep: add wallet inputs to reach dust limit [do not review] Dec 10, 2019

joostjager mentioned this pull request Dec 12, 2019

sweep: allow force sweeping of negatively yielding inputs #3809

Merged

chainregistry: set static min relay fee

8c43232

We need it to be set in order to properly test the sweeper handling the dust limit on regtest.

joostjager force-pushed the sweeper-add-utxo branch 3 times, most recently from bd3d6c8 to 215c322 Compare December 12, 2019 13:17

joostjager removed request for Roasbeef and cfromknecht December 12, 2019 14:14

joostjager force-pushed the sweeper-add-utxo branch 2 times, most recently from 26a6329 to 3dea473 Compare December 13, 2019 09:58

joostjager added 2 commits December 13, 2019 11:08

sweep: use bucket ids

5007821

Using a fee rate just as an identifier can be confusing.

sweep: fix bucket clustering

e2bf6b4

Fixes a bug where bucket sizes were not the configured size, but the configured size plus the min relay fee.

joostjager force-pushed the sweeper-add-utxo branch from 3dea473 to 86121bb Compare December 13, 2019 10:08

halseth added the anchors label Dec 13, 2019

joostjager force-pushed the sweeper-add-utxo branch 4 times, most recently from c15f2ff to aa898b2 Compare December 13, 2019 10:55

joostjager added 2 commits December 13, 2019 12:02

sweep: create sweep parameters struct

38adfd7

Prepares for adding more input-specific sweep parameters.

sweep: embed input.Input interface

071c57d

Get rid of needless referencing of the embedded object.

joostjager force-pushed the sweeper-add-utxo branch from aa898b2 to 3db8d0b Compare December 13, 2019 11:06

halseth suggested changes Dec 13, 2019

View reviewed changes

joostjager force-pushed the sweeper-add-utxo branch 3 times, most recently from d98a1d1 to fb6377d Compare December 13, 2019 13:02

joostjager requested a review from halseth December 13, 2019 13:06

joostjager requested a review from wpaulino December 13, 2019 13:51

joostjager added this to the 0.9.0 milestone Dec 13, 2019

joostjager added v0.9.0 chain handling channel closing Related to the closing of channels cooperatively and uncooperatively utxo sweeping labels Dec 13, 2019

joostjager changed the title ~~sweep: add wallet inputs to reach dust limit [do not review]~~ sweep: add wallet inputs to reach dust limit Dec 13, 2019

joostjager force-pushed the sweeper-add-utxo branch from fb6377d to b142690 Compare December 13, 2019 14:20

wpaulino reviewed Dec 16, 2019

View reviewed changes

joostjager force-pushed the sweeper-add-utxo branch 2 times, most recently from 2187e18 to 61bdf32 Compare December 16, 2019 21:13

joostjager requested a review from wpaulino December 16, 2019 21:13

joostjager force-pushed the sweeper-add-utxo branch from 61bdf32 to 600e059 Compare December 16, 2019 21:35

halseth reviewed Dec 17, 2019

View reviewed changes

joostjager force-pushed the sweeper-add-utxo branch from 600e059 to 997a67e Compare December 17, 2019 12:02

joostjager requested a review from halseth December 17, 2019 12:02

wpaulino reviewed Dec 17, 2019

View reviewed changes

joostjager force-pushed the sweeper-add-utxo branch from 997a67e to e70fd44 Compare December 17, 2019 20:57

joostjager added 4 commits December 17, 2019 22:00

sweep: create wallet interface

34c9193

We need access to additional wallet functionality. This commit creates an interface to prevent passing in multiple function pointers.

sweep: extract sweepCluster method

8353b6f

Prepares for adding another level of nesting.

joostjager requested a review from wpaulino December 17, 2019 21:00

joostjager force-pushed the sweeper-add-utxo branch from e70fd44 to e01600f Compare December 17, 2019 21:00

wpaulino approved these changes Dec 17, 2019

View reviewed changes

halseth approved these changes Dec 18, 2019

View reviewed changes

joostjager merged commit 7ecbe22 into lightningnetwork:master Dec 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sweep: add wallet inputs to reach dust limit #3814

sweep: add wallet inputs to reach dust limit #3814

joostjager commented Dec 10, 2019 •

edited

Loading

halseth left a comment

joostjager commented Dec 13, 2019

halseth Dec 17, 2019

joostjager Dec 17, 2019

wpaulino Dec 17, 2019

halseth Dec 17, 2019

joostjager Dec 17, 2019

wpaulino Dec 17, 2019

halseth left a comment

sweep: add wallet inputs to reach dust limit #3814

sweep: add wallet inputs to reach dust limit #3814

Conversation

joostjager commented Dec 10, 2019 • edited Loading

halseth left a comment

Choose a reason for hiding this comment

joostjager commented Dec 13, 2019

halseth Dec 17, 2019

Choose a reason for hiding this comment

joostjager Dec 17, 2019

Choose a reason for hiding this comment

wpaulino Dec 17, 2019

Choose a reason for hiding this comment

halseth Dec 17, 2019

Choose a reason for hiding this comment

joostjager Dec 17, 2019

Choose a reason for hiding this comment

wpaulino Dec 17, 2019

Choose a reason for hiding this comment

halseth left a comment

Choose a reason for hiding this comment

joostjager commented Dec 10, 2019 •

edited

Loading