Lightning Dev Summit Topics

[vincenzopalazzo]

LN Summit 2024 Topics

Tracking issue to collect the topics that folks are interested in discussing during the lightning dev summit in September.

Please submit topics that you would like to discuss in the comments.

General Guidelines:

• Where possible, please try to post topics in advance so that other attendees have time to prepare!
• Short summaries and links to additional resources relevant to your topic are very helpful.
• Feel free to post a topic you're interested in even if you don't want to present/lead a discussion on it.

The notes from last year's summit can be found here.

[moneyball]

How do folks feel about spending 1 if not 2 of the 3 days discussing some of the biggest fundamental challenges with the LN protocol as it is today and then potential long-term solutions?

Example fundamental challenges with LN:

1. Onboarding costs for new users (due to on-chain costs and cost of capital)
2. Small balances aren't economically feasible (due to on-chain costs)
3. Regulatory/trust challenges with channel management for mobile users
4. LN today is a step in the wrong direction to achieve p2p digital cash as it needs numerous cloud servers/services (block/txn data, LN state backup, RGS-like service for performant pathfinding, probing scoring data service, LSP/JIT channel partner, async payments channel partner)

Nothing would be off limits for solutions including long-proposed ideas such as eltoo/LN Symmetry, channel factories, and coinpools as well as more recent proposals such as sidepools and timeout trees. Protocol changes that require bitcoin consensus changes around covenants or GSR would be within scope of discussion.

Thoughts?

[t-bast]

I completely agree with @moneyball's proposal and think it's worth spending 2 days on that topic. We're at a stage right now where we're all working on implementing a few large improvements for which the spec is somewhat final (dual-funding, splicing, taproot, bolt 12, liquidity ads). These features have been discussed at length already and simply need engineering effort to get to the finish line, without any significant blocker. It's a great time to brainstorm what happens next, after all of these features, and ensure that we're working towards a satisfying long-term network!

And we can keep 1 day to iron out the details of some of the short / medium term work.

[vincenzopalazzo]

From my perspective, there are a few points I'd like to discuss that can fit well inside 2h of the 3th day after we discuss the fundamental limitation of the current lightning, and also could help to improve the "engineering effort" required to implement features proposed like dual-funding, splicing, taproot, bolt 12, liquidity ads.

• If I manage to finish a Proof of Concept (PoC) for the new lnprototest by the summit, I'd like to give a demo preview to gather overall feedback.
• I'd also like to discuss our decision to use Delving Bitcoin and gather feedback on whether we should continue with it or consider alternative ML approaches. See for initial discussion meta: update the link to the ML #1170
• Regarding Delving Bitcoin, I believe it introduces another issue concerning how we handle significant pull requests (PRs) like offers or splicing. As one of the implementors of these features, I find it challenging to track PRs without resorting to "pushing my version" and resolving issues later.
• Last but not least, I would like to discuss the possibility of introducing a user story section inside the BOLTs proposal, for context: When we introduce new features would be helpful to have a description of the feature as a user story. An example is the route blinding and I think we need one for 0-conf and anchor outputs. IMHO there is no reason to have a speck if it is difficult for external people to understand it.

[carlaKC]

If anyone wants nightmares ideas for topics I tried to map out open problems here a while ago: https://miro.com/app/board/uXjVN6hnZj8=/

[carlaKC]

I'd also like to discuss our decision to use Delving Bitcoin and gather feedback on whether we should continue with it or consider alternative ML approaches.

Is there a reason we need to cover this at the summit? Seems like an easy enough topic for a regular meeting, we should just add it to the agenda sometime.

[rustyrussell]

Unordered list:

• Gossip minisketch. Some fun bitstuffing to fit things into 64 bits, particularly for node_announcements.
• Putting a separate comms key for BOLT 8 in node_announcement.
  • This means you can rotate what key you use to talk to peers.
  • Would allow a proxy handshake fronting without (permanent) trust.
• Adding a "who I'm trying to reach" to the initial BOLT 8 handshake.
  • This would be good for providers who would no longer have to dedicate a TCP port to each client.
• BOLT 12 extensions, like contacts and recurrence.

[t-bast]

I'd like to spend some time brainstorming our not-so-future use of TRUC transactions and 0-fee commitments, the simplifications it provides and the trade-offs it creates for wallets that thus need to keep on-chain funds available in case their LSP cheats.

I'd also like to discuss (quickly) a few current topics that we should be able to finally close:

• channel reserve requirements with splicing
• trampoline and blinded paths (see https://github.com/t-bast/trampoline-blinded-paths/blob/master/Trampoline%20with%20blinded%20paths.pdf)

[Roasbeef]

Some items on my mind lately (mostly future stuff):

• Off chain channel on boarding:
  • A class of protocol extensions designed to enable more than one channel to exist within a single output. This includes ideas like channel factories, variants of ark, etc, etc.
    • Ark is a bit different, since all outputs expire after a period of time, so all channels need to be rolled over or the funds are forfeited. As a result, all channels created in an ark end up being ephemeral channels.
  • The goal here is to be able to reduce the last mile on boarding cost for new users, packing as many channels into a single output as possible.
  • A few months back I sketched out a very minimal version of such an extension, dubbed "self funded channel factories": https://gist.github.com/Roasbeef/8aa7ff10b5a132c126e047a613d4b075:
    • It's similar to the type of batch channel funding people do today with PSBTs (N channels N outputs in 1 transaction), but uses a root musig2 output to create a 1 level tree of channels.
    • As is, there's always a single funder to the factory that puts in all the capital (simplifies things a lot)
• Alloy BOLT Specifications:
  • Alloy is a language for lightweight formal specifications. Compared to something like Coq which is a proof assistant for formal specification and verification (used to prove that somethign is correct for all inputs, Alloy is simpler and has friendly syntax. Instead of generating proofs for the spec, you write the Spec in Alloy, then write certain assertions that must hold, then Alloy tries to find counterexamples, so it's bounded model checking.
  • I've been dabbling in it over the past few months, so far I've used it to create a model of the way time delay based fee bumping works, in lnd: docs/alloy-models: add new folder for Alloy models along w/ model for Linear Fee Function bug fix  lightningnetwork/lnd#8943. In the PR I make a model of how the logic works today, then use it to reproduce an off-by-one bug in the design.
    • My goal from here is to make a basic ping/pong model (trivial but will get a better hang of modeling message passing), then use the message passing module made from that to write a spec for stfu, before doing the final boss of the channel state machine.
    • For a simpler example, here's a simple model that increments a counter over time.
  • Alloy specs can serve as complement to our implementations and also the written spec. We can use it to further specify the murky parts of the spec, like the channel state machine, allowing us to more precisely document certain subtle cases (concurrent updates) or even uncover certain invariant violations that already exist, but haven't popped up yet in our implementations.
• PTLCs
  • Picking up where instagibbs left off re spec design. We're close to having 2 implementations of taproot channels, and after gossip 1.5/1.75/2.0, deploying PTLCs is the next logical step.
  • Along the way we need to decide if we're going in the direction of the simplified channel state machine (a great candidate for an Alloy spec!), and also if we'd also like to attempt to finally introduce an unadd, which allows channels to actually recover from state transition violations/bugs.
  • There's also some overlap here re redundant payments.
• Payment Reliability Gadgets:
  • What low hanging fruit exist that could potentially be used to further increase payment reliability on the network?
  • Redundant Payments:
    • Talked about in passing last time. The general idea is that w/ PTLCs you can take a point in an invoice, then mix in your own secret, forcing the receiver to contact how (somehow) before they settle.
    • This allows for "spray and pray" where you can send many HTLCs, knowing that you as the sender will only reveal the preimages for only up to the payment amount, with the others timing out or being cncelled back.
    • This allows MPP implementations to be a bit more aggressive, as they don't need to send payments with an exact "shape".
    • Onion messaging can potentially be used to handle the communication between the sender and receiver.
  • Payment Level ACKs:
    • Today when you send an HTLC, you have no idea if it actually got to the other side or not. You only know if it fails (cancel), or when everything gets there and the receiver finally settles.
    • What if we had an ACK? So when you send the HTLC, the other side ACKs if they got it or not, allowing you to update your MPP plan, and also potentially work out some forwarding latency related metrics.
    • Poor man's payment ACK is to send two HTLCs, one min(1 mSAT, max(route.min_htlcs)) and the other the actual payment. This ends up running into a two general's like problem: what if the ACK gets stuck...?
    • Onion messages could be used to transmit the ACKs, giving a sender a better view into what actually go to the other side. Combined with redundant payments, you can continue to send, know which ones got there, and not fear an over payment.
  • Hybrid Reactive/Proactive Routing:
    • Structurally, today we use a proactive routing protocol. All nodes have some version of the graph, and then use that for payments. Other than new links and policy updates, no other information is exchanged.
    • Overtime, people started to probe more as a way to figure out if a route can even work in the first place, leading to a hybrid protocol wherein nodes proactively maintain the channel graph, but then at times reactively send out a probe to gauge current network conditions.
    • Should we consider a more concrete cut out for experimentation with a better specified hybrid protocol? So something similar to old reservation systems people have proposed in the past, where a non-binding intent to route is sent out a feeler (no HTLC) which can be used to gain up to date information re internal network conditions.
    • This isn't without tradeoffs ofc:
      • Non-binding so nodes can lie, or even overpromise (say yes to everything beyond your capacity).
      • Enshrines probing as a thing, potential source of additional balance/privacy leak.

[renepickhardt]

@moneyball wrote:

Example fundamental challenges with LN:

1. Onboarding costs for new users (due to on-chain costs and cost of capital)
2. Small balances aren't economically feasible (due to on-chain costs)
3. Regulatory/trust challenges with channel management for mobile users
4. LN today is a step in the wrong direction to achieve p2p digital cash as it needs numerous cloud servers/services (block/txn data, LN state backup, RGS-like service for performant pathfinding, probing scoring data service, LSP/JIT channel partner, async payments channel partner)

Extending my last post on delvingbitcoin about the likelihood for payments to be infeasible I have recently published the current draft (first 10 pages are pretty stable) of my research entitled "A mathematical theory of payment channel networks". From this I'd like to add a 5th point to @moneyball's list:

5. The current 2-party channels put too heavy constraints on liquidity and reduce the reliability guarantees of payment delivery.

The model and results of the linked research suggests that 2-party channels are too constraint to leave users enough flexibility to reliably conduct payments. The geometric properties of the model also explains why we observe so many depleted channels in practice and that this is to be expected and can hardly be mitigated through liquidity management (neither off-chain nor on-chain).

The positive side of those results indicate that the direction that @Roasbeef proposes seems to have more advantages than those mentioned by him:

• Off chain channel on boarding:

  • A class of protocol extensions designed to enable more than one channel to exist within a single output. This includes ideas like channel factories, variants of ark, etc, etc.

    • Ark is a bit different, since all outputs expire after a period of time, so all channels need to be rolled over or the funds are forfeited. As a result, all channels created in an ark end up being ephemeral channels.

  • The goal here is to be able to reduce the last mile on boarding cost for new users, packing as many channels into a single output as possible.

I fully agree with the intuition that more channels per output are useful as we gain more flexibility for the liquidity without the necessity to conduct expensive on chain transactions. However at this time I belief we have sufficient evidence that the number of channels per output is not as important as the number of participants per output.

In the same repository I have a somewhat outdated notebook that shows that the more peers share an output (with all the downsides that are attached to it) the better the liquidity can be used by the participants of the network. This results in better payment flows and higher reliability. Please take this notebook carefully as there is still a methodological error included. In reality the results will be less optimistic. Yet I do belief that the trend of the following two diagrams from the notebook should still hold true even if the systematic error is fixed:

This shows rates of infeasible payments of a current lightning network snapshot for payments of 1M sats. The multi party cases are constructed by starting from the current topology of the 2-party channel graph and randomly adding existing peers to the channels.

The end points of the diagram when the curves have converged are put to this diagram:

This is rather strong evidence that multi party channel networks seem to have the additional advantage to reduce the rate of infeasible payments. As far as I know this property of multi party channel networks was not explicitly known / discussed before and I think it is useful to keep this in mind when discussing @Roasbeef's ideas / proposal.

• Hybrid Reactive/Proactive Routing:

  • Structurally, today we use a proactive routing protocol. All nodes have some version of the graph, and then use that for payments. Other than new links and policy updates, no other information is exchanged.

  • Overtime, people started to probe more as a way to figure out if a route can even work in the first place, leading to a hybrid protocol wherein nodes proactively maintain the channel graph, but then at times reactively send out a probe to gauge current network conditions.

  • Should we consider a more concrete cut out for experimentation with a better specified hybrid protocol? So something similar to old reservation systems people have proposed in the past, where a non-binding intent to route is sent out a feeler (no HTLC) which can be used to gain up to date information re internal network conditions.

  • This isn't without tradeoffs ofc:

    • Non-binding so nodes can lie, or even overpromise (say yes to everything beyond your capacity).
    • Enshrines probing as a thing, potential source of additional balance/privacy leak.

The afore mentioned research indicates that the rate of infeasible payments cannot be reduced through a better routing protocol. However a reduction in uncertainty and improvements in routing protocols can help us in the good case to deliver the payment more quickly and in the bad case to fail quicker. Thus I generally support this direction and will add the lowish hanging fruit of optionally sharing liquidity information within the friend of a friend network.

[jkczyz]

I'd like to discuss the BOLT12 Payment Notification proposal, which aims to extend BOLT12 to cover the LNURL-verify use case.

• Design alternatives: https://docs.google.com/document/d/1MbOnzl-IiBBw95qnaLr9JeRqecfFCF9ehoQ8KZb9Tpk/edit?usp=sharing
• Background discussion: https://discord.com/channels/981260788004184144/981260788004184147/1174493444144648293

[jkczyz]

I'd like to discuss the BOLT12 Payment Notification proposal, which aims to extend BOLT12 to cover the LNURL-verify use case.

More broadly, we should discuss "next steps" for after the BOLT12 spec is merged. In addition to payment notifications, we've seen interest in requesting a refund given an invoice + preimage. And the notification proposal includes a proof-of-payment mechanism, so we may want to keep that in mind when thinking about this.

[ZmnSCPxj-jr]

Forwardable peerswaps. Yes, it is not as cool as multiparty (N > 2) offchain cryptocurrency mechanisms, but it is effectively multiparty in that a single onchain transaction (potentially) affects multiple channels, can be deployed with less code than actual N > 2 cryptocurrency mechanisms, and is simpler even than batched splicing.

[ZmnSCPxj-jr]

• gossip feedback loops that are observed in the wild and how we can mitigate or destroy them.

[carlaKC]

Channel Jamming

We've posted some results and updates to our proposal on delving for discussion at the summit. It's a long post (for a long flight perhaps!), but if you only read one thing please start reading at the sink attack section. We've tested the mitigation against various types of attacks, and would like to do a whiteboard session to refine the changes to the proposal we're looking at making, and discuss tradeoffs.

V3 and Upgrade Mechanism

Spend some time discussing the V3 commitment format, and how we're going to go about upgrading (upgrade for anchors / upgrade for taproot / both?). Would be great to leave the summit with agreement on the new commitment format and a solid plan for upgrade mechanism.

Bolt Issue/PR Cleanup

I think it will be worthwhile to do a "shill, kill or merge" session to clean up the issues and PRs on the BOLTs repo. Can be done in a low-brainpower (/high jetlag) hour, where we run through everything that's open and either:

• Shill: make the case for picking up work, find a volunteer to move the work forward
• Kill: close anything that's outdated or won't be picked up by a volunteer
• Merge: commit to merging anything that's ready

We're generally quite good at merging in our online meetings, so we'd mostly be killin and shillin 🔪

[moneyball]

Would be great to discuss the SuperScalar proposal which is perhaps more actionable than John Law's timeout trees as it doesn't require a consensus change. Although one can argue this isn't appropriate for this dev meetup since it doesn't require a BOLT change as any LSP could implement this. (but, there still might be broad interest among this group as a way to address some of LN's drawbacks) https://delvingbitcoin.org/t/superscalar-laddered-timeout-tree-structured-decker-wattenhofer-factories/1143/5

[niftynei]

I'd be interested in having some time dedicated to discussing LN symmetry and the covenant proposal landscape as it applies/impacts the ability to add symmetry.

More of an informational and status session on the current symmetry project and what list/set of updates onchain are needed to unlock it.

[t-bast]

It was great to see everyone, thanks everyone for all the energy and ideas!
And thanks for Vincenzo for organizing everything!

Sharing my notes from the topics I was involved with:

• 0-fee commitment transactions:
  • bitcoind 29.x (target release April 2025) will contain everything we need (0-amount anchors and v3 1-parent-1-child packages)
  • lets us remove the CSV 1 on outputs and CPFP from to_remote or HTLC txs (endogenous fees FTW)
  • single anchor instead of two anchors (with amount matching the pending dust HTLCs)
  • @t-bast will open a draft BOLT PR for the new commitment type that uses that
  • splice transactions will keep using v2: pinning risk still exists while the splice is unconfirmed
    • but we'll add an option in interactive-tx to set the transaction's version
    • useful when you use a splice transaction to spend an anchor outputs and bump a v3 commit tx
  • still need to discuss on the BOLT PR whether the anchor should be keyed or not
  • and whether using v3 impacts the maximum number of HTLCs
  • relies on bitcoin core's submitpackage RPC, which isn't available for electrum client (e.g. mobile wallets)
    • @tnull mentioned working on a project to fix this?
  • upgrade path through @ProofOfKeags's dynamic commitment upgrade protocol
  • maybe in X years with cluster mempool we'll be able to go back to v2 transactions without pinning risk (unknown so far)
• existing anchor channels:
  • automatically benefits from package propagation with bitcoind 28.x (to be released very soon)
  • allows us to set commit feerate at 1 sat/byte, stop sending update_fee and use package relay to force-close
  • once we're confident enough bitcoin nodes have upgraded of course
• option_simple_close:
  • clarify requirements when receiving a new shutdown: we don't need to re-send ours, can just send signatures immediately for the remote tx
  • interop ongoing between eclair and lnd, should be ready soon
• trampoline:
  • interaction with Bolt 12 has a concept ACK (see https://github.com/t-bast/trampoline-blinded-paths/blob/master/Trampoline%20with%20blinded%20paths.pdf)
  • @t-bast to update the spec PR's last commit to reflect that
  • @t-bast to clarify requirements in spec PR
• channel jamming:
  • reputation algorithm seems to work well in practice
  • need to run simulations with the outgoing reputation added
  • looks promising so far
• onion messages back pressure:
  • @t-bast to turn the latest write-up into a spec (BOLT or bLIP?)
• Bolt 12 contacts:
  • we mustn't include additional information in offers for this (private by default)
  • when paying an offer, the payer may include one of their own offers and a contact_id in their invoice_request if they wish to add the recipient to their contacts
  • the recipient can use this to pay them back and add them to its contacts
  • it's unclear yet how to reconcile concurrent payments in both directions (different contact_ids for the same contact pair)
  • we don't use the payer_id field of the invoice_request
  • @t-bast to update bLIP 42 to match those requirements and clarify edge cases

[yyforyongyu]

and whether using v3 impacts the maximum number of HTLCs

@t-bast I think so. Here's a draft-ish calculation. Also think the keyless anchor gives us the benefit to use key path spending in tap chans, just FYI.

[ariard]

@rustyrussell Can I ask you a simple question in a cold and polite fashion ? As I think you're still the older active contributor to the lightning protocol so far.

If you could give to the general audience what were the technical criterias and how many people were invited to this Lightning Dev Summit. Not asking the names of the attendees, just if a formal process was followed in constituting the invitation list to ensure consistency, neutrality and bright lines.

In the past, you published such piece about the “Corruption of Ethics in Cryptocurrencies" and I must say it was a good one.
Though more recently you had strong (hostile ?) comments on twitter (here) about the behavior of some other contributors, and now I’m very confused if you're not acting with double-standard of ethics, or more concerning deliberately doing delusional statements in this industry.

I don’t think there is no need to yell about your past experiences about the Linux kernel of decades ago. Used to do linux hacking and familiar how to debug a syscall table and more frankly, I don't think a lot of things are great about Linux culture.

Here, I guess I should finish by quoting some Aussie writers, philosophers or historians, as well people on the internet are driven by different cultural norms. But the only one I know is Samuel Alexander (through Whitehead) and I must confess I have not read it.

[TheBlueMatt]

I can answer that because I originally invited people. The list was "people who actively contributed to the spec (or were invited by a major lightning implementation cause they regularly work on it) or were otherwise at the last one".

Also, closing this issue now since the summit is over. Thanks everyone for attending and thanks again to the folks who organized it!

[Roasbeef]

Here's the raw notes I took during the summit (working on typing up a longer summary): https://docs.google.com/document/d/1erQfnZjjfRBSSwo_QWiKiCZP5UQ-MR53ZWs4zIAVcqs/edit?usp=sharing