routing: extend trackpayment with htlc events #3972

joostjager · 2020-01-29T17:48:35Z

Currently there is inconsistency between the payment status as reported via ListPayment and the status notified through routerrpc.SendPayment/TrackPayment. With this PR, the same full payment struct is returned by both rpc calls.

In addition to that, routerrpc notifications are now sent out for every change to the payment. This will allow users to track progress of their (multi-parts) payments, either via lncli (use the newly added --show_inflight flag) or via graphical user interfaces/apps if support is added there. Previously only the final state of the payment was reported.

Note: The HTLCAttempt_IN_FLIGHT update will be sent out before the htlc is actually in flight on the link. Launching a QueryRoutes at that point will not see the balance change yet. This is relevant for applications that use the output of TrackPayment to implement their own payment loop.

joostjager · 2020-04-06T11:16:34Z

Ready for concept ack

Todo: add ConcurrentQueue for notifications, to prevent rpc users from blocking the payment process

joostjager · 2020-04-06T15:00:57Z

ConcurrentQueue added. Also used multimutex to prevent a global lock around db tx and notification. Initially I had the global lock, but that also caused the async payments benchmark to fail. Probably too slow.

routing/control_tower.go

halseth

Excellent PR, really makes it easier to reason about the control tower subscription logic 👍

lnrpc/routerrpc/router_backend.go

lnrpc/rpc.proto

halseth · 2020-04-07T08:18:17Z

queue/queue.go

+		}
+
+		// Close outgoing channel.
+		close(cq.chanOut)


maybe this should be done on quit also?

I don't think so. It would signal a clean end to the queue, which isn't the case for quit

queue/queue.go

halseth · 2020-04-07T08:21:06Z

multimutex/hash_mutex.go

+	// a given hash will hold the mutex to be used by all
+	// callers requesting access for the hash, in addition to
+	// the count of callers.
+	mutexes map[lntypes.Hash]*cntMutex


can't we just take 4 bytes of the hash and use the existing uint64 mutex? Alternatively use interface{} as the key type.

Interesting suggestion. That will work, unless people use non-random hashes (for invoices that are never settled)? There is some cost in terms of understandability. Not sure if it weighs up.

Worst case it will just have to wait a bit longer before accessing the DB in those cases. I think it is nice to avoid the code duplication, as long as we don't have generics.

Do avoid the scenario you mention, could also a add a simple method

func (h Hash) Uint64Key() uint64 { return uint64(sha256(h[:])[0:4]) }

but maybe that is taking it too far

I will leave this thread open for others to comment on.

no strong preference fro me, tho i think a hash map would be a useful addition to multimutex as a library

leaving hash map in then

routing/control_tower.go

lnrpc/routerrpc/router_backend.go

channeldb/payment_control.go

lnrpc/routerrpc/router_server.go

routing/control_tower.go

cfromknecht · 2020-04-07T23:07:14Z

multimutex/hash_mutex.go

+	// a given hash will hold the mutex to be used by all
+	// callers requesting access for the hash, in addition to
+	// the count of callers.
+	mutexes map[lntypes.Hash]*cntMutex


no strong preference fro me, tho i think a hash map would be a useful addition to multimutex as a library

Roasbeef

LGTM 🦊

Re the breaking proto changes: as mentioned offline, in the end we haven't committed to making this a stable API yet. I think we also envisioned from the start that AMP/MPP would likely result in a series of changes in the API as things became more concrete through development. With 0.10, as we're now moving to always compile it in, and we may also have the API docs render the sub-servers as well, we can now start to publicly commit to gradual freezing of the interface.

As mentioned offline, a blog post explaining how to interact with and track MPP for 0.10 will go a long way.

Similar to what is done for SettleAttempt. Co-authored-by: Johan T. Halseth <johanth@gmail.com>

This commit fixes the inconsistency between the payment state as reported by routerrpc.SendPayment/routerrpc.TrackPayment and the main rpc ListPayments call. In addition to that, payment state changes are now sent out for every state change. This opens the door to user interfaces giving more feedback to the user about the payment process. This is especially interesting for multi-part payments.

joostjager · 2020-04-08T15:13:12Z

This PR replaces the messages in the stream of SendPayment and TrackPayment with a completely different message. Running an application that isn't updated against this new rpc yields:

grpc: failed to unmarshal the received message unexpected EOF

alexbosworth · 2020-04-08T17:01:56Z

This PR replaces the messages in the stream of SendPayment and TrackPayment with a completely different message. Running an application that isn't updated against this new rpc yields:

grpc: failed to unmarshal the received message unexpected EOF

That's pretty rough, it is because it is not a new RPC, it's an existing RPC that was overwritten in a way that gRPC was not designed to support

Roasbeef · 2020-04-08T22:01:25Z

That's pretty rough

That's perfectively acceptable as this is an experimental RPC server, which was only envisioned to only near a final form after MPP/AMP was implemented, which is now. Compare this to the external sendpayment and llist/subscribe invoice calls which haven't changed significantly since they were created.

cfromknecht

LGTM 🔥

The main SendPayment rpcs have been in place for 3+ years now w/o breaking changes. IMO it's better to break the API before moving routerrpc out of experimental than end up in a mediocre middle ground. If we're lucky routerrpc.SendPayment will be more Lindy than the original, so in the grand scheme of things the amortized costs are pretty marginal for users who happen to be using the experimental RPCs early. Seems a little cart before the horse to have experimental integrations prevent shipping a polished product for the masses.

alexbosworth · 2020-04-08T22:21:34Z

By rough I mean that it's more difficult to deal with as a consumer. So if you were building on this, you will now have a harder job adapting to the change than otherwise might be possible

Roasbeef added gRPC mpp payments Related to invoices/payments routing rpc Related to the RPC interface labels Feb 3, 2020

joostjager force-pushed the notify-htlc-arrival branch from 9415ddd to 097b5db Compare April 3, 2020 21:31

joostjager added the incomplete WIP PR, not fully finalized, but light review possible label Apr 3, 2020

joostjager mentioned this pull request Apr 3, 2020

routing: multi-part mpp send #3967

Merged

joostjager force-pushed the notify-htlc-arrival branch 4 times, most recently from 6152911 to 874c0e1 Compare April 6, 2020 10:46

joostjager marked this pull request as ready for review April 6, 2020 10:48

joostjager requested review from halseth and Roasbeef as code owners April 6, 2020 10:48

joostjager requested review from cfromknecht and removed request for halseth April 6, 2020 10:48

joostjager removed the incomplete WIP PR, not fully finalized, but light review possible label Apr 6, 2020

joostjager force-pushed the notify-htlc-arrival branch 5 times, most recently from 313d220 to 40e8ac4 Compare April 6, 2020 14:43

joostjager force-pushed the notify-htlc-arrival branch from 40e8ac4 to d5daad5 Compare April 6, 2020 15:10

joostjager modified the milestones: 0.10.1, 0.10.0 Apr 6, 2020

Roasbeef reviewed Apr 6, 2020

View reviewed changes

routing/control_tower.go Outdated Show resolved Hide resolved

joostjager requested a review from Roasbeef April 7, 2020 05:49

joostjager force-pushed the notify-htlc-arrival branch from d5daad5 to 5929f0e Compare April 7, 2020 05:49

halseth suggested changes Apr 7, 2020

View reviewed changes

joostjager force-pushed the notify-htlc-arrival branch from 5929f0e to 35cbdde Compare April 7, 2020 09:04

joostjager requested a review from halseth April 7, 2020 11:10

cfromknecht reviewed Apr 7, 2020

View reviewed changes

Roasbeef approved these changes Apr 8, 2020

View reviewed changes

routing: fix log

5d0a117

joostjager force-pushed the notify-htlc-arrival branch from 35cbdde to 97fa277 Compare April 8, 2020 06:17

joostjager requested a review from cfromknecht April 8, 2020 06:17

joostjager force-pushed the notify-htlc-arrival branch 2 times, most recently from 3cdbd28 to 10919c4 Compare April 8, 2020 06:33

joostjager and others added 5 commits April 8, 2020 08:53

routerrpc: move payment marshalling

327634e

lnrpc: expose payment failure reason

351d8e1

channeldb: return updated payment on attempt update

278915e

Similar to what is done for SettleAttempt. Co-authored-by: Johan T. Halseth <johanth@gmail.com>

queue: detect close of incoming channel

f907fbc

multimutex: add hash mutex

5bebda8

joostjager force-pushed the notify-htlc-arrival branch from 10919c4 to a086a48 Compare April 8, 2020 06:54

joostjager added 2 commits April 8, 2020 09:26

lncli: add show_inflight flag

3ed3c90

joostjager force-pushed the notify-htlc-arrival branch from a086a48 to 3ed3c90 Compare April 8, 2020 07:26

cfromknecht approved these changes Apr 8, 2020

View reviewed changes

Roasbeef merged commit 3ad7ab2 into lightningnetwork:master Apr 8, 2020

guggero mentioned this pull request May 18, 2020

mod: update to queue/v1.0.4 #4295

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

routing: extend trackpayment with htlc events #3972

routing: extend trackpayment with htlc events #3972

joostjager commented Jan 29, 2020 •

edited

Loading

joostjager commented Apr 6, 2020

joostjager commented Apr 6, 2020

halseth left a comment

halseth Apr 7, 2020

joostjager Apr 7, 2020

halseth Apr 7, 2020

joostjager Apr 7, 2020

halseth Apr 7, 2020 •

edited

Loading

joostjager Apr 7, 2020

cfromknecht Apr 7, 2020

joostjager Apr 8, 2020

cfromknecht Apr 7, 2020

Roasbeef left a comment

joostjager commented Apr 8, 2020

alexbosworth commented Apr 8, 2020 •

edited

Loading

Roasbeef commented Apr 8, 2020

cfromknecht left a comment

alexbosworth commented Apr 8, 2020

routing: extend trackpayment with htlc events #3972

routing: extend trackpayment with htlc events #3972

Conversation

joostjager commented Jan 29, 2020 • edited Loading

joostjager commented Apr 6, 2020

joostjager commented Apr 6, 2020

halseth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

halseth Apr 7, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Roasbeef left a comment

Choose a reason for hiding this comment

joostjager commented Apr 8, 2020

alexbosworth commented Apr 8, 2020 • edited Loading

Roasbeef commented Apr 8, 2020

cfromknecht left a comment

Choose a reason for hiding this comment

alexbosworth commented Apr 8, 2020

joostjager commented Jan 29, 2020 •

edited

Loading

halseth Apr 7, 2020 •

edited

Loading

alexbosworth commented Apr 8, 2020 •

edited

Loading