Welcome to Mastering the Lightning Network!
The Lightning Network, often abbreviated as "LN", is changing the way people exchange value online, and it’s one of the most exciting advancements to happen in Bitcoin’s history. Today, in 2021, the Lightning Network is still in its infancy. The Lightning Network is a protocol for using Bitcoin in a smart and non-obvious way. It is a second layer technology on top of Bitcoin.
The concept of the Lightning Network was proposed in 2015, and the first implementation was launched in 2018. As of 2021, we’re only beginning to see the opportunities the Lightning Network provides to Bitcoin including improved privacy, speed, and scale. With core knowledge of the Lightning Network, you can help shape the future of the network while also building opportunities for yourself.
We assume you already have some basic knowledge about Bitcoin, but if not, don’t worry – we will explain the most important Bitcoin concepts, those you must know to understand the Lightning Network, in [bitcoin_fundamentals_review]. If you want to learn more about Bitcoin, you can read Mastering Bitcoin, available for free online.
While the bulk of this book is written for programmers, the first few chapters are written to be approachable by anyone regardless of technical experience. In this chapter, we’ll start with some terminology, then move to look at trust and its application in these systems, and finally we’ll discuss the history and future of the Lightning Network. Let’s get started.
As we explore how the Lightning Network actually works, we will encounter some technical terminology that might, at first, be a bit confusing. While all of these concepts and terms will be explained in detail as we progress through the book and are defined in the glossary, some basic definitions now will make it easier to understand the concepts in the next two chapters. If you don’t understand all of the words in these definitions yet, that’s okay. You’ll understand more as you move through the text.
- Blockchain
-
A distributed transaction ledger, produced by a network of computers. Bitcoin, for example, is a system that produces a blockchain. The Lightning Network is not itself a blockchain, nor does it produce a blockchain. It is a network that relies on an existing external blockchain for its security.
- Digital Signature
-
A digital signature is a mathematical scheme for verifying the authenticity of digital messages or documents. A valid digital signature gives a recipient reason to believe that the message was created by a known sender, that the sender cannot deny having sent the message, and that the message was not altered in transit.
- Hash Function
-
A cryptographic hash function is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function which is infeasible to invert.
- Node
-
A computer that participates in a network. A Lightning node is a computer that participates in the Lightning Network. A Bitcoin node is a computer that participates in the Bitcoin Network. Typically a Lightning Network user will run a Lightning node and a Bitcoin node.
- On-Chain vs. Off-Chain
-
A payment is "on-chain" if it is recorded as a transaction on the Bitcoin (or other underlying) blockchain. Payments sent via payment channels between Lightning nodes, and which are not visible in the underlying blockchain, are called "off-chain" payments. Usually in the Lightning Network, the only on-chain transactions are those used to open and close a Lightning payment channel. A third type of channel modifying transaction exists, called splicing, which can be used to add/remove the amount of funds committed in a channel.
- Payment
-
When value is exchanged on the Lightning Network, we call this a "payment" as compared to a "transaction" on the Bitcoin blockchain.
- Payment Channel
-
A financial relationship between two nodes on the Lightning Network, typically implemented by multi-signature Bitcoin transactions that share control over bitcoin between the two Lightning nodes.
- Routing vs Sending
-
Unlike Bitcoin where transactions are "sent" by broadcasting them to everyone, Lightning is a routed network where payments are "routed" across one or more payment channels following a path from sender to recipient.
- Transaction
-
A data structure that records the transfer of control over some funds (e.g. some bitcoin). The Lightning Network relies on Bitcoin transactions (or those of another blockchain) to track control of funds.
More detailed definitions of these and many other terms can be found in the [glossary]. Throughout this book, we will explain what these concepts mean and how these technologies actually work.
|
Tip
|
Throughout this book, you will see "Bitcoin" with the first letter capitalized, which refers to the Bitcoin System and is a proper noun. You will also see "bitcoin", with a lower-case b, which refers to the currency unit. Each bitcoin is further subdivided into 100 million units each called a "satoshi" (singular) or "satoshis" (plural) |
Now that you’re familiar with these basic terms, let’s move to a concept you are already comfortable with: trust.
You will often hear people calling Bitcoin and the Lightning Network "trustless". At first glance this is confusing. After all, isn’t trust a good thing? Banks even use it in their names! Isn’t a "trustless" system, a system devoid of trust, a bad thing?
The use of the word "trustless" is intended to convey the ability to operate without needing trust in the other participants in the system. In a decentralized system like Bitcoin, you can always choose to transact with someone you trust. However, the system ensures you can’t be cheated even if you can’t trust the other party in a transaction. Trust is a nice-to-have instead of a must-have property of the system.
Contrast that to traditional systems like banking where you must place your trust in a third party, since it controls your money. If the bank violates your trust, you may be able to find some recourse from a regulator or court, but at an enormous cost of time, money, and effort.
Trustless does not mean devoid of trust. It means that trust is not a necessary prerequisite to all transactions and that you can transact even with people you don’t trust because the system prevents cheating.
Before we get into how the Lightning Network works, it’s important to understand one basic concept that underlies Bitcoin, the Lightning Network and many other such systems: something we call a fairness protocol. A fairness protocol is a way to achieve fair outcomes between participants, who do not need to trust each other, without the need for a central authority and it is the backbone of decentralized systems like Bitcoin.
When people have competing interests, how can they establish enough trust to engage in some cooperative or transactional behavior? The answer to this question lies at the core of several scientific and humanistic disciplines, such as economics, sociology, behavioral psychology, and mathematics. Some of those disciplines give us "soft" answers that depend on concepts such as reputation, fairness, morality, and even religion. Other disciplines give us concrete answers that depend only on the assumption that the participants in these interactions will act rationally, with their self-interest as the main objective.
In broad terms, there are a handful of ways to ensure fair outcomes in interactions between individuals who may have competing interests:
-
Require trust - you only interact with people whom you already trust, due to prior interactions, reputation, or familial relationships. This works well enough at small scale, especially within families and small groups, that it is the most common basis for cooperative behavior. Unfortunately, it doesn’t scale and it suffers from tribalist (in-group) bias.
-
Rule of law - establish rules for interactions that are enforced by an institution. This scales better, but it cannot scale globally due to differences in customs and traditions, as well as the inability to scale the institutions of enforcement. One nasty side-effect of this solution is that the institutions become more and more powerful as they get bigger and that may lead to corruption.
-
Trusted third parties - put an intermediary in every interaction to enforce fairness. Combined with the "rule of law" to provide oversight of intermediaries, this scales better, but suffers from the same imbalance of power: the intermediaries get very powerful and may attract corruption. Concentration of power leads to systemic risk and systemic failure ("Too big to fail").
-
Game theoretical fairness protocols - this last category emerges from the combination of the internet and cryptography and is the subject of this section. Let’s see how it works and what its advantages and disadvantages are.
Cryptographic systems like Bitcoin and the Lightning Network are systems that allow you to transact with people (and computers) that you don’t trust. This is often referred to as "trustless" operation, even though it is not actually trustless. You have to trust in the software that you run, and you have to trust that the protocol implemented by that software will result in fair outcomes.
The big distinction between a cryptographic system like this and a traditional financial system is that in traditional finance you have a trusted third party, for example a bank, to ensure that outcomes are fair. A significant problem with such systems is that they give too much power to the third party, and they are also vulnerable to a single point of failure. If the trusted third party itself violates trust or attempts to cheat, the basis of trust breaks.
As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the protocol, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is two-fold: Not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
The use of incentives and disincentives to achieve fair outcomes is one aspect of a branch of mathematics called game theory, which studies "models of strategic interaction among rational decision makers" [1]. Cryptographic systems that control financial interactions between participants, such as Bitcoin and the Lightning Network, rely heavily on game theory to prevent participants from cheating and allow participants who don’t trust each other to achieve fair outcomes.
While game theory and its use in cryptographic systems may appear confounding and unfamiliar at first, chances are you’re already familiar with these systems in your everyday life; you just don’t recognize them yet. Below we’ll use a simple example from childhood to help us identify the basic pattern. Once you understand the basic pattern, you will see it everywhere in the blockchain space and you will come to recognize it quickly and intuitively.
In this book, we call this pattern a Fairness Protocol defined as a process that uses a system of incentives and/or disincentives to ensure fair outcomes for participants who don’t trust each other. Enforcement of a fairness protocol is only necessary to ensure that the participants can’t escape the incentives or disincentives.
Let’s look at an example of a fairness protocol, which you may already be familiar with.
Imagine a family lunch, with a parent and two children. The children are fussy eaters and the only thing they will agree to eat is fried potatoes. The parent has prepared a bowl of fried potatoes ("french fries" or "chips" depending on which English dialect you use). The two siblings must share the plate of chips. The parent must ensure a fair distribution of chips to each child, otherwise the parent will have to hear constant complaining (maybe all day) and there’s always a possibility of an unfair situation escalating to violence. What is a parent to do?
There are a few different ways that fairness can be achieved in this strategic interaction between two siblings that do not trust each other and have competing interests. The naive but commonly used method is for the parent to use their authority as a trusted third party: they split the bowl of chips into two servings. This is similar to traditional finance, where a bank, accountant or lawyer acts as a trusted third party to prevent any cheating between two parties who want to transact.
The problem with this scenario is that it vests a lot of power and responsibility in the hands of the trusted third party. In this example, the parent is fully responsible for the equal allocation of chips, the parties merely wait, watch, and complain. The children accuse the parent of playing favorites and not allocating the chips fairly. The siblings fight over the chips, yelling "that chip is bigger!", and dragging the parent into their fight. It sounds awful, doesn’t it? Should the parent yell louder? Take all of the chips away? Threaten to never make chips again and let those ungrateful children go hungry?
A much better solution exists: the siblings are taught to play a game called "split and choose". At each lunch one sibling splits the bowl of chips into two servings and the other sibling gets to choose which serving they want. Almost immediately, the siblings figure out the dynamic of this game. If the one splitting makes a mistake or tries to cheat, the other sibling can "punish" them by choosing the bigger bowl. It is in the best interest of both siblings, but especially the one splitting the bowl, to play fair. Only the cheater loses in this scenario. The parent doesn’t even have to use their authority or enforce fairness. All the parent has to do is enforce the protocol; as long as the siblings cannot escape their assigned roles of "splitter" and "chooser", the protocol itself ensures a fair outcome without the need for any intervention. The parent can’t play favorites or distort the outcome.
|
Warning
|
While the infamous chip battles of the 1980’s neatly illustrate the point, any similarity between the scenario above and any of the authors' actual childhood experiences with their cousins is entirely coincidental… Or is it? |
In order for a fairness protocol like this to work, there need to be certain guarantees, or security primitives, that can be combined to ensure enforcement. The first security primitive is strict time ordering/sequencing: the "splitting" action must happen before the "choosing" action. It’s not immediately obvious, but unless you can guarantee that action A happens before action B, then the protocol falls apart. The second security primitive is commitment with non-repudiation. Each sibling must commit to their choice of role: either splitter or chooser. Also, once the splitting has been completed, the splitter is committed to the split they created - they cannot repudiate that choice and go try again.
Cryptographic systems offer a number of security primitives that can be combined in different ways to construct a fairness protocol. In addition to sequencing and commitment, we can also use many other tools:
-
Hash functions to fingerprint data, as a form of commitment, or as the basis for a digital signature.
-
Digital signatures for authentication, non-repudiation, and proof of ownership of a secret.
-
Encryption/decryption to restrict access to information to authorized participants only.
This is only a small list of a whole "menagerie" of security and cryptographic primitives that are in use. More basic primitives and combinations are invented all the time.
In our real-life example, we saw one form of fairness protocol called "split and choose". This is just one of a myriad different fairness protocols that can be built by combining the building blocks of security primitives in different ways. But the basic pattern is always the same: two or more participants interact without trusting each other, by engaging in a series of steps that are part of an agreed protocol. The protocol’s steps arrange incentives and disincentives to ensure that if the participants are rational, cheating is counter-productive and fairness is the automatic outcome. Enforcement is not necessary to get fair outcomes - it is only necessary to keep the participants from breaking out of the agreed protocol.
Now that you understand this basic pattern, you will start seeing it everywhere in Bitcoin, the Lightning Network and many other systems. Let’s look at some specific examples next.
The most prominent example of a "fairness protocol" is Bitcoin’s consensus algorithm, Proof of Work (PoW). In Bitcoin, miners compete to verify transactions and aggregate them in blocks. To ensure that the miners do not cheat, without entrusting them with authority, Bitcoin uses a system of incentives and disincentives. Miners have to use electricity and dedicate hardware doing "work" that is embedded as a "proof" inside every block. This is achieved because of a property of hash functions where the output value is randomly distributed across the entire range of possible outputs. If miners succeed in producing a valid block fast enough, they are rewarded by earning the block reward for that block. Forcing miners to use a lot of electricity before the network considers their block means that they have an incentive to correctly validate the transactions in the block. If they cheat or make any kind of mistake, their block is rejected and the electricity they used to "prove" it is wasted. No one needs to force miners to produce valid blocks; the reward and punishment incentivize them to do so. All the protocol needs to do is ensure that only valid blocks with proof of work are accepted.
The "fairness protocol" pattern can also be found in many different aspects of the Lightning Network:
-
Those who fund channels make sure that they have a refund transaction signed before they publish the funding transaction.
-
Whenever a channel is moved to a new state, the old state is "revoked" by ensuring that if anyone tries to broadcast it, they lose the entire balance and get punished.
-
Those who forward payments know that if they commit funds forward, they can either get a refund or they get paid by the node preceding them.
Again and again, we see this pattern. Fair outcomes are not enforced by any authority. They emerge as the natural consequence of a protocol that rewards fairness and punishes cheating. A fairness protocol that harnesses self-interest by directing it towards fair outcomes.
Bitcoin and the Lightning Network are both implementations of fairness protocols. So why do we need the Lightning Network? Isn’t Bitcoin enough?
Bitcoin is a system that records transactions on a globally replicated public ledger. Every transaction is seen, validated and stored by every participating computer. As you can imagine, this generates a lot of data and is difficult to scale.
As Bitcoin and the demand for transactions grew, the number of transactions in each block increased until it eventually reached the block size limit. Once blocks are "full", excess transactions are left to wait in a queue. Many users will increase the fees they’re willing to pay in order to buy space for their transactions in the next block.
If demand continues to outpace the capacity of the network, an increasing number of users' transactions are left waiting unconfirmed. Competition for fees also increases the cost of each transaction, making many smaller-value transactions (e.g. micro-transactions) completely uneconomical during periods of particularly high demand.
To solve this problem, we could increase the block size limit to create space for more transactions. An increase in the "supply" of block space will lead to a lower price equilibrium for transaction fees.
However, increasing block size shifts the cost to node operators and requires them to expend more resources to validate and store the blockchain. Because blockchains are gossip protocols, each node is required to know and validate every single transaction that occurs on the network. Furthermore, once validated, each transaction and block must be propagated to the node’s "neighbors", multiplying the bandwidth requirements. As such, the greater the block size, the greater the bandwidth, processing, and storage requirements for each individual node. Increasing transaction capacity in this way has the undesirable effect of centralizing the system by reducing the number of nodes and node operators. Since node operators are not compensated for running nodes, if nodes are very expensive to run, only a few well-funded node operators will continue to run nodes.
The side effects of increasing the block size or decreasing the block time with respect to centralization of the network are severe as a few calculations with the numbers show.
Let us assume the usage of Bitcoin grows so that the network has to process 40,000 transactions per second, which is the approximate transaction processing level of the VISA network during peak usage.
Assuming 250 bytes on average per transaction, this would result in a data stream of 10 megabytes per second or 80 Mbit/s just to be able to receive all the transactions. This does not include the traffic overhead of forwarding the transaction information to other peers. While 10 MB/s does not seem extreme in the context of high-speed fiber optic and 5G mobile speeds, it would effectively exclude anyone who cannot meet this requirement from running a node, especially in countries where high-performance internet is not affordable or widely available.
Users also have many other demands on their bandwidth and cannot be expected to expend this much only to receive transactions.
Furthermore, storing this information locally would result in 864 gigabytes per day. This is roughly one terabyte of data, or the size of a hard drive.
Verifying 40,000 ECDSA signatures per second is also barely feasible (c.f.: https://bitcoin.stackexchange.com/questions/95339/how-many-bitcoin-transactions-can-be-verified-per-second) making the Initial Blockchain Download (IBD) of the Bitcoin blockchain (synchronizing and verifying everything starting from the genesis block) almost impossible without very expensive hardware.
While 40,000 transactions per second seems like a lot, it only achieves parity with traditional financial payment networks at peak times. Innovations in machine-to-machine payments, micro-transactions and other applications are likely to push demand to many orders higher than that.
Simply put: You can’t scale a blockchain to validate the entire world’s transactions in a decentralized way.
But what if each node wasn’t required to know and validate every single transaction? What if there was a way to have scalable off-chain transactions, without losing the security of the Bitcoin network?
In February 2015, Joseph Poon and Thaddeus Dryja proposed a possible solution to the Bitcoin Scalability Problem, with the publication of "The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments" [2]
In the (now outdated) whitepaper, Poon and Dryja estimate that in order for Bitcoin to reach the 47,000 transactions per second processed at peak by Visa, it would require 8 GB blocks. This would make running a node completely untenable for anyone but large scale enterprises and industrial grade operations. The result would be a network in which only a few users could actually validate the state of the ledger. Bitcoin relies on users validating the ledger for themselves, without explicitly trusting third parties, in order to stay decentralized. Pricing users out of running nodes would force the average user to trust third parties to discover the state of the ledger, ultimately breaking the trust model of Bitcoin.
The Lightning Network proposes a new network, a "second layer", where users can make payments to each other peer-to-peer, without the necessity of publishing a transaction to the Bitcoin blockchain for each payment. Users may pay each other on the Lightning Network as many times as they want, without creating additional Bitcoin transactions or incurring on-chain fees. They only make use of the Bitcoin blockchain in order to load bitcoin onto the Lightning network initially and to "settle", that is, to remove bitcoin from the Lightning Network. The result is that many more Bitcoin payments can take place "off-chain", with only the initial loading and final settlement transactions needing to be validated and stored by Bitcoin nodes. Aside from reducing the burden on nodes, payments on the Lightning Network are cheaper for users as they do not need to pay blockchain fees, and more private for users as they are not published to all participants of the network and furthermore not stored permanently.
While the Lightning Network was initially conceived for Bitcoin, it can be implemented on any blockchain that meets some basic technical requirements. Other blockchains, such as Litecoin, already support the Lightning Network. Additionally, several other blockchains are developing similar "second layer" or "layer 2" solutions to help them scale.
The Lightning Network is a network that operates as a "second layer" protocol on top of Bitcoin and other blockchains. The Lightning Network enables fast, secure, private, trustless, and permissionless payments. Here are some of the features of the Lightning Network:
-
Users of the Lightning Network can route payments to each other for low cost and in real time.
-
Users who exchange value over the Lightning Network do not need to wait for block confirmations for payments.
-
Once a payment on the Lightning Network has completed, usually within a few seconds, it is final and cannot be reversed. Like a Bitcoin transaction, a payment on the Lightning Network can only be refunded by the recipient.
-
While "on-chain" Bitcoin transactions are broadcast and verified by all nodes in the network, payments routed on the Lightning Network are transmitted between pairs of nodes and are not visible to everyone, resulting in much greater privacy.
-
Unlike transactions on the Bitcoin Network, payments routed on the Lightning Network do not need to be stored permanently. Lightning thus uses fewer resources and hence is cheaper. This property also has benefits for privacy.
-
The Lightning Network uses onion routing, similar to the protocol used by The Onion Router (Tor) privacy network, so that even the nodes involved in routing a payment are only directly aware of their predecessor and successor in the payment route.
-
When used on top of Bitcoin, the Lightning Network uses real bitcoin which is always in the possession (custody) and full control of the user. Lightning is not a separate token or coin, it is Bitcoin.
In order to better understand how the Lightning Network actually works, and why people use it, we’ll be following a number of users and their stories.
In our examples, some of the people have already used Bitcoin and others are completely new to the Bitcoin network. Each person and their story, as listed here, illustrate one or more specific use cases. We’ll be revisiting them throughout this book:
- consumer
-
Alice is a Bitcoin user who wants to make fast, secure, cheap, and private payments for small retail purchases. She buys coffee with bitcoin, using the Lightning Network.
- merchant
-
Bob owns a coffee shop, "Bob’s Cafe". "On-chain" Bitcoin payments don’t scale for small amounts like a cup of coffee, so he uses the Lightning Network to accept Bitcoin payments almost instantaneously and for low fees.
- software service business
-
Chan is a Chinese entrepreneur who sells information services related to the Lightning Network, as well as Bitcoin and other cryptocurrencies. Chan is selling these information services over the Internet by implementing micro-payments over the Lightning Network. Additionally, Chan has implemented a liquidity provider service that rents inbound channel capacity on the Lightning Network, charging a small bitcoin fee for each rental period.
- gamer
-
Dina is a teenage gamer from Russia. She plays many different computer games, but her favorite ones are those that have an "in-game economy" based on real money. As she plays games, she also earns money by acquiring and selling virtual in-game items. The Lightning Network allows her to transact in small amounts for in-game items as well as earn small amounts for completing quests.
In this chapter, we talked about the fundamental concept that underlies both Bitcoin and the Lightning Network: the fairness protocol.
We looked at the history of the Lightning Network and the motivations behind second layer scaling solutions for Bitcoin and other blockchain-based networks.
We learned basic terminology including node, payment channel, on-chain transactions, and off-chain payments.
Finally, we met Alice, Bob, Chan and Dina, who we’ll be following throughout the rest of the book. In the next chapter, we’ll meet Alice and walk through her thought process as she selects a Lightning wallet and prepares to make her first Lightning payment to buy a cup of coffee from Bob’s Cafe.