Add the option on path creator to specify the incoming channel on blinded path #9127

MPins · 2024-09-21T04:16:26Z

Change Description

Add the option on path creator to specify the incoming chained channel list

Steps to Test

lncli addinvoice --blind --blinded_path_incoming_channel_list channel_id1 amount
lncli addinvoice --blind --blinded_path_incoming_channel_list channel_id1,channel_id2 amount

Pull Request Checklist

Testing

Your PR passes all CI checks.
Tests covering the positive and negative (error paths) are included.
Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

The change obeys the Code Documentation and Commenting guidelines, and lines wrap at 80.
Commits follow the Ideal Git Commit Structure.
Any new logging statements use an appropriate subsystem and logging level.
Any new lncli commands have appropriate tags in the comments for the rpc in the proto file.
There is a change description in the release notes, or [skip ci] in the commit message for small changes.

📝 Please see our Contribution Guidelines for further guidance.

MPins · 2024-09-21T04:22:06Z

Hello @ellemouton if you think I'm on the right way I can add the option to choose a channel also.

MPins · 2024-09-22T13:42:18Z

Maybe change the name to "incoming_node" is a better idea! But I'll wait for others opinion also.

ellemouton · 2024-09-23T01:16:34Z

thanks @MPins - in tokyo at the moment for the LN summit so will take a look at this a bit later this week or next week

ziggie1984 · 2024-10-03T20:14:21Z

I think it makes sense to just specify the incoming channel, I don't see a lot of use cases for the whole blinded route, but maybe we already prepare the code so that it will be easy to just upgrade to the whole blinded path if it has benefits.

ziggie1984

Thank you for your first contribution 🎉

Please squash all commits related to the lnrpc change into one.

Moreover while reviewing this PR I found out that we have a bug in your probabiliy calculation. We basically should have the prob. of full certainty for the last hop however we need to change the logic in our probability estimator which still seems to not account for the blinded usecase see here (only when from == Self do we set the full local probability, that needs to now be inverted for the last hop as well):

lnd/routing/missioncontrol.go

Lines 509 to 526 in 2d33317

    
           func (m *MissionControl) GetProbability(fromNode, toNode route.Vertex, 
        
           	amt lnwire.MilliSatoshi, capacity btcutil.Amount) float64 { 
        
           	m.mu.Lock() 
        
           	defer m.mu.Unlock() 
        
           	now := m.cfg.clock.Now() 
        
           	results, _ := m.state.getLastPairResult(fromNode) 
        
           	// Use a distinct probability estimation function for local channels. 
        
           	if fromNode == m.cfg.selfNode { 
        
           		return m.estimator.LocalPairProbability(now, results, toNode) 
        
           	} 
        
           	return m.estimator.PairProbability( 
        
           		now, results, toNode, amt, capacity, 
        
           	) 
        
           }

lnrpc/lightning.proto

routing/pathfind.go

cmd/commands/cmd_invoice.go

MPins · 2024-10-04T02:50:19Z

I think it makes sense to just specify the incoming channel, I don't see a lot of use cases for the whole blinded route, but maybe we already prepare the code so that it will be easy to just upgrade to the whole blinded path if it has benefits.

In most cases it would be sufficient to specify the node, the channel must be specified in cases where there is more than one channel with the same node. Of course, specifying the channel covers both use cases. I'm not sure if we should keep both options for the user, perhaps it would be more intuitive to specify the income node for most users.

ziggie1984 · 2024-10-04T09:57:12Z

I prefer the channel, because than you have more control rather than the pubkey. But at the end not sure if its really worth it to treat different channels to the same peer differently because we have non-strict forwarding. So probably both options are ok ...

MPins · 2024-10-07T12:58:32Z

Thank you for your first contribution 🎉

Please squash all commits related to the lnrpc change into one.

Moreover while reviewing this PR I found out that we have a bug in your probabiliy calculation. We basically should have the prob. of full certainty for the last hop however we need to change the logic in our probability estimator which still seems to not account for the blinded usecase see here (only when from == Self do we set the full local probability, that needs to now be inverted for the last hop as well):

lnd/routing/missioncontrol.go

Lines 509 to 526 in 2d33317

func (m *MissionControl) GetProbability(fromNode, toNode route.Vertex,

amt lnwire.MilliSatoshi, capacity btcutil.Amount) float64 {

m.mu.Lock()

defer m.mu.Unlock()

now := m.cfg.clock.Now()

results, _ := m.state.getLastPairResult(fromNode)

// Use a distinct probability estimation function for local channels.

if fromNode == m.cfg.selfNode {

return m.estimator.LocalPairProbability(now, results, toNode)

}

return m.estimator.PairProbability(

now, results, toNode, amt, capacity,

)

}

Thank you for your carefully revision. I'll be working on it some time on the following days. About the probability, I invest some time trying to understanding it, but I didn't get the bug on it! From my understanding, probability is about sending the payment, right? How the payment node would even know that we are choosing the income channel/node? I'm probably missing something, I would appreciate if you give me directions to expand my understanding.

MPins · 2024-10-07T13:37:56Z

I prefer the channel, because than you have more control rather than the pubkey. But at the end not sure if its really worth it to treat different channels to the same peer differently because we have non-strict forwarding. So probably both options are ok ...

I think we can move forward with both options, but in my opinion they should be exclusive. I mean, if the channel is chosen, the node cannot be chosen. (just to avoid redundant info)

ziggie1984 · 2024-10-08T06:29:07Z

So my initial idea was, that we can know the probability of the last channel in a blinded path with very high accuracy because its our own channel and we know the liquidity distribution when creating the blinded path. However looking into the codebase I think we should keep it as is, we do allow MPP payments for blinded paths so the amount can be splitted among the incoming channels in general. Moreover when we specify the incoming channel to receive on (via this PR) we already made the decision that this is the right channel to receive on the full amount, so maybe we can keep the probability estimation of the last hop/channel as is. Given the fact that we include all routes with a minimum route prob. of 1% we are good as is I think.

MPins · 2024-10-08T09:26:11Z

So my initial idea was, that we can know the probability of the last channel in a blinded path with very high accuracy because its our own channel and we know the liquidity distribution when creating the blinded path. However looking into the codebase I think we should keep it as is, we do allow MPP payments for blinded paths so the amount can be splitted among the incoming channels in general. Moreover when we specify the incoming channel to receive on (via this PR) we already made the decision that this is the right channel to receive on the full amount, so maybe we can keep the probability estimation of the last hop/channel as is. Given the fact that we include all routes with a minimum route prob. of 1% we are good as is I think.

Got it ... besides that, as it is blinded I can't think a way of considering that it is our own node without breaking the main reason for using a blinded path, that is not revealing our own node on a invoice.

ziggie1984 · 2024-10-08T09:54:21Z

Got it ... besides that, as it is blinded I can't think a way of considering that it is our own node without breaking the main reason for using a blinded path, that is not revealing our own node on a invoice.

Hmm not sure what you mean, but I was just referring to the creator of the blinded path taking the incoming channel distribution directly into account without really relying on the MC data. The path would still be blinded for the sender so the sender would just have a sorted list of blinded paths in the bolt11 invoice.

MPins · 2024-10-08T10:00:29Z

Got it ... besides that, as it is blinded I can't think a way of considering that it is our own node without breaking the main reason for using a blinded path, that is not revealing our own node on a invoice.

Hmm not sure what you mean, but I was just referring to the creator of the blinded path taking the incoming channel distribution directly into account without really relying on the MC data. The path would still be blinded for the sender so the sender would just have a sorted list of blinded paths in the bolt11 invoice.

Never mind ... I was thinking that the GetProbability func would be called when the payment is being processed by the sender.

MPins · 2024-10-08T12:10:52Z

So my initial idea was, that we can know the probability of the last channel in a blinded path with very high accuracy because its our own channel and we know the liquidity distribution when creating the blinded path. However looking into the codebase I think we should keep it as is, we do allow MPP payments for blinded paths so the amount can be splitted among the incoming channels in general. Moreover when we specify the incoming channel to receive on (via this PR) we already made the decision that this is the right channel to receive on the full amount, so maybe we can keep the probability estimation of the last hop/channel as is. Given the fact that we include all routes with a minimum route prob. of 1% we are good as is I think.

I just saw that it is called by FindBlindedPaths ... Thank you, I have a better understanding of the whole process now.

I think you right ... maybe it should be changed ... not sure if it should be included in this PR or maybe on a specific one for that.

go.mod

saubyk · 2024-10-13T17:25:55Z

Have a basic question here.
The problem statement in the linked issue is regarding specifying an incoming channel...

to have better control over payment success probability
to better control the liquidity distribution available on the node

So why are we then giving the user the control of selecting a node in the cli, instead of directly specifying a channel?
I believe the user should only specify the channel not the node (which I find confusing)

MPins · 2024-10-13T19:25:37Z

Have a basic question here. The problem statement in the linked issue is regarding specifying an incoming channel...

to have better control over payment success probability

to better control the liquidity distribution available on the node

So why are we then giving the user the control of selecting a node in the cli, instead of directly specifying a channel? I believe the user should only specify the channel not the node (which I find confusing)

The idea to have the option to specify the node instead of the channel is because in the most cases it would be enough and from the node runner perspective it would be more intuitive specifying just the income node.

ziggie1984 · 2024-10-13T20:26:05Z

So why are we then giving the user the control of selecting a node in the cli, instead of directly specifying a channel?
I believe the user should only specify the channel not the node (which I find confusing)

Good question, I also tend to only support the incoming channel id why:

When creating the blinded path we do not create a Unified Edge meaning that we do NOT take all the different policies for all the channels with the same peer into account. Moreover we cannot be sure that all policies are the same, other implementations iirc can have different policies for different channels, therefore I recommend only using the channel ID here.
Although the lightning network supports non-strict forwarding which means the forwarder can basically select another channel if he wants to, but as mentioned before, we are not unifying the edges to create the worst case policy to the last node. So I think we should stick to the incoming channel id here and should not allow the node id.

ziggie1984 · 2024-10-13T20:47:09Z

Imagine your peer has 2 channels with you and different policies, now when creating the blinded path and using the nodeid, you cannot control the particular channel and the sender will probably use the route with the better constraints. We already have something similar when sending a payment where we can select the outgoing channel. We should prob. stick to this form.

MPins · 2024-10-13T21:14:34Z

I think that

Imagine your peer has 2 channels with you and different policies, now when creating the blinded path and using the nodeid, you cannot control the particular channel and the sender will probably use the route with the better constraints. We already have something similar when sending a payment where we can select the outgoing channel. We should prob. stick to this form.

When sending payment you can also choose the last hop (penultimate node in the path) to route through for the payment, but it is other use case. So I think you both are right, I'm going to take the node option out.

ziggie1984 · 2024-10-13T21:20:27Z

Good observation that we also allow the last hop in the payment flow, but LND does unify the edges in the sending flow and selects the most expensive policy that's why it makes sense to have this last hop setting.

MPins · 2024-10-13T21:34:06Z

Good observation that we also allow the last hop in the payment flow, but LND does unify the edges in the sending flow and selects the most expensive policy that's why it makes sense to have this last hop setting.

Perfect ... I'm going to take the node option out. Thank you @saubyk for starting this exchange of ideas.

saubyk · 2024-10-13T21:37:54Z

Good observation that we also allow the last hop in the payment flow, but LND does unify the edges in the sending flow and selects the most expensive policy that's why it makes sense to have this last hop setting.

Perfect ... I'm going to take the node option out. Thank you @saubyk for starting this exchange of ideas.

Thanks. Not to add the confusion, but I do understand the logic of selecting the node from a UX standpoint and especially when you imagine a user doing it in a UI. Selecting a pub key (with an alias) is much easier than selecting a channel ID. But I think that problem can be addressed at the application level, where the UI can establish that node-channel linkage and present a user friendly way for the user to make a selection.

ellemouton · 2024-10-16T09:28:21Z

I prefer the channel, because than you have more control rather than the pubkey.

The problem is that you dont actually every have this control due to non-strict forwarding. At the end of the day, the peer may choose any channel to fwd on as long as the peer on the other end is the same peer.

Regarding the impl here and if we only want to specify final hop or list of hops, I think things should be made general enough for both:

so if user specifies 1 peer, then LND takes that as a fixed final hop and uses its current logic to build on that given the other config values
use can specify a list of hops and LND will just do the same thing as above.

MPins · 2024-10-16T10:37:31Z

use can specify a list of hops and LND will just do the same thing as above.

The way it is done here the user can specify the last hop many times (peers or channels, not both). If I understand you correctly it should be kept this way. Right?

ziggie1984 · 2024-10-16T11:04:41Z

@ellemouton what's your take on this, we cannot really know whether peers have a universal policy to us?

Imagine your peer has 2 channels with you and different policies, now when creating the blinded path and using the nodeid, you cannot control the particular channel and the sender will probably use the route with the better constraints. We already have something similar when sending a payment where we can select the outgoing channel. We should prob. stick to this form.

ziggie1984

Pending comments and CI run then gtg from my side.

Take a look at the commit msg, and try to explain what's happening in that respective commit and avoid general descriptions. Sometimes you have the same description for different commits which is missing the point of the description.

cmd/commands/cmd_invoice.go

routing/router.go

rpcserver.go

itest/lnd_route_blinding_test.go

ziggie1984 · 2024-11-28T08:13:42Z

itest/lnd_route_blinding_test.go

+// inserted into an invoice, sent to an invoice with a blinded path and
+// forwarded payment in a blinded route and finally, receiving the payment.
+func testGivenBlindedRouteInvoices(ht *lntest.HarnessTest) {
+	// Create a three hop network: Alice -> Bob -> Carol.


still not changed ?

docs/release-notes/release-notes-0.19.0.md

cmd/commands/cmd_invoice.go

routing/pathfind.go

MPins · 2024-11-28T14:07:30Z

Hello @ziggie1984 thank you so much for your carefully review and guidance.

ziggie1984

LGTM 🎉

Had some last nits but it's gtg from my side.

lnrpc/invoicesrpc/invoices.swagger.json

lnwire/short_channel_id.go

cmd/commands/cmd_invoice.go

rpcserver.go

Add an option to the addinvoice rpc which allows to specify the blinded path via a list of chained channels.

Including the blinded path incoming channel list argument to the addinvoice command, parsing and verifying that it has one or a comma separeted list of channels id.

Adding the incoming channel list parameter to FindBlindedPaths function on router.go, and adding to processNodeForBlindedPath func on pathfind.go the ability to restrict the blinded route to the chained channel list for incoming channel.

This commit makes sure that when an incoming channel list is supplied for the blinded route, that it changes the other blinding restrictions accordingly. Moreover it introduces a sanity check when the number of paths for the blinded route is set to 0.

lightninglabs-deploy · 2024-12-06T22:23:35Z

@saubyk: review reminder
@ellemouton: review reminder

docs/release-notes/release-notes-0.19.0.md

MPins changed the title ~~Add the option on path creator to specify the incoming node~~ Add the option on path creator to specify the incoming node on blinded path Sep 21, 2024

saubyk assigned MPins Oct 3, 2024

ziggie1984 requested changes Oct 3, 2024

View reviewed changes

lnrpc/lightning.proto Show resolved Hide resolved

routing/pathfind.go Outdated Show resolved Hide resolved

routing/pathfind.go Outdated Show resolved Hide resolved

cmd/commands/cmd_invoice.go Outdated Show resolved Hide resolved

cmd/commands/cmd_invoice.go Outdated Show resolved Hide resolved

MPins force-pushed the issue-8993 branch from 65a9c7f to adc5c80 Compare October 12, 2024 00:49

MPins commented Oct 12, 2024

View reviewed changes

go.mod Outdated Show resolved Hide resolved

MPins changed the title ~~Add the option on path creator to specify the incoming node on blinded path~~ Add the option on path creator to specify the incoming node or channel on blinded path Oct 13, 2024

MPins force-pushed the issue-8993 branch 5 times, most recently from eb7b1ee to 3598716 Compare November 28, 2024 04:08

MPins requested a review from ziggie1984 November 28, 2024 04:29

ziggie1984 reviewed Nov 28, 2024

View reviewed changes

MPins force-pushed the issue-8993 branch 2 times, most recently from 8db12b4 to b393833 Compare November 28, 2024 10:19

MPins requested review from ziggie1984 and saubyk November 28, 2024 14:06

MPins force-pushed the issue-8993 branch from b393833 to 4aaad35 Compare November 29, 2024 11:50

ziggie1984 approved these changes Nov 29, 2024

View reviewed changes

lnrpc/invoicesrpc/invoices.swagger.json Show resolved Hide resolved

lnwire/short_channel_id.go Show resolved Hide resolved

cmd/commands/cmd_invoice.go Outdated Show resolved Hide resolved

rpcserver.go Show resolved Hide resolved

MPins added 8 commits November 29, 2024 19:39

lnrpc: add the option to set the incoming channel on a blinded path

f7e1441

Add an option to the addinvoice rpc which allows to specify the blinded path via a list of chained channels.

lnwire: add a function which parses the human-readable short channel id.

6d44bad

commands: add option to set the incoming channel on blinded path

4607bba

Including the blinded path incoming channel list argument to the addinvoice command, parsing and verifying that it has one or a comma separeted list of channels id.

lndtest: Added function RPC call to asserts errors on AddInvoice

fd9a5c9

itest: add the test for given incoming channel on blinded invoice

0a7302f

routing: add the test for incoming chained channels on blinded path

c316753

MPins force-pushed the issue-8993 branch from 4aaad35 to 92fafc0 Compare November 29, 2024 22:49

MPins requested a review from ellemouton November 29, 2024 23:41

saubyk reviewed Dec 7, 2024

View reviewed changes

docs/release-notes/release-notes-0.19.0.md Show resolved Hide resolved

docs: release-notes-0.19.0

8280e71

MPins force-pushed the issue-8993 branch from 92fafc0 to 8280e71 Compare December 9, 2024 12:01

MPins requested a review from saubyk December 9, 2024 12:03

ellemouton removed the request for review from saubyk December 12, 2024 10:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the option on path creator to specify the incoming channel on blinded path #9127

Add the option on path creator to specify the incoming channel on blinded path #9127

MPins commented Sep 21, 2024 •

edited

Loading

MPins commented Sep 21, 2024

MPins commented Sep 22, 2024

ellemouton commented Sep 23, 2024

ziggie1984 commented Oct 3, 2024

ziggie1984 left a comment •

edited

Loading

MPins commented Oct 4, 2024

ziggie1984 commented Oct 4, 2024

MPins commented Oct 7, 2024

MPins commented Oct 7, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

MPins commented Oct 8, 2024 •

edited

Loading

saubyk commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

saubyk commented Oct 13, 2024 •

edited

Loading

ellemouton commented Oct 16, 2024

MPins commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024 •

edited

Loading

ziggie1984 left a comment

ziggie1984 Nov 28, 2024

MPins commented Nov 28, 2024

ziggie1984 left a comment

lightninglabs-deploy commented Dec 6, 2024

	func (m *MissionControl) GetProbability(fromNode, toNode route.Vertex,
	amt lnwire.MilliSatoshi, capacity btcutil.Amount) float64 {

	m.mu.Lock()
	defer m.mu.Unlock()

	now := m.cfg.clock.Now()
	results, _ := m.state.getLastPairResult(fromNode)

	// Use a distinct probability estimation function for local channels.
	if fromNode == m.cfg.selfNode {
	return m.estimator.LocalPairProbability(now, results, toNode)
	}

	return m.estimator.PairProbability(
	now, results, toNode, amt, capacity,
	)
	}

Add the option on path creator to specify the incoming channel on blinded path #9127

Are you sure you want to change the base?

Add the option on path creator to specify the incoming channel on blinded path #9127

Conversation

MPins commented Sep 21, 2024 • edited Loading

Change Description

Steps to Test

Pull Request Checklist

Testing

Code Style and Documentation

MPins commented Sep 21, 2024

MPins commented Sep 22, 2024

ellemouton commented Sep 23, 2024

ziggie1984 commented Oct 3, 2024

ziggie1984 left a comment • edited Loading

Choose a reason for hiding this comment

MPins commented Oct 4, 2024

ziggie1984 commented Oct 4, 2024

MPins commented Oct 7, 2024

MPins commented Oct 7, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

ziggie1984 commented Oct 8, 2024

MPins commented Oct 8, 2024

MPins commented Oct 8, 2024 • edited Loading

saubyk commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

ziggie1984 commented Oct 13, 2024

MPins commented Oct 13, 2024

saubyk commented Oct 13, 2024 • edited Loading

ellemouton commented Oct 16, 2024

MPins commented Oct 16, 2024

ziggie1984 commented Oct 16, 2024 • edited Loading

ziggie1984 left a comment

Choose a reason for hiding this comment

ziggie1984 Nov 28, 2024

Choose a reason for hiding this comment

MPins commented Nov 28, 2024

ziggie1984 left a comment

Choose a reason for hiding this comment

lightninglabs-deploy commented Dec 6, 2024

MPins commented Sep 21, 2024 •

edited

Loading

ziggie1984 left a comment •

edited

Loading

MPins commented Oct 8, 2024 •

edited

Loading

saubyk commented Oct 13, 2024 •

edited

Loading

ziggie1984 commented Oct 16, 2024 •

edited

Loading